This study examines the impact of using a mitosis-specific antibody (phospho-histone H3, or PHH3) on the accuracy of annotating mitotic figures (MFs) in tissue samples stained with hematoxylin and eosin (H&E). While PHH3 staining is considered a reliable method for identifying mitosis, it may introduce biases that affect the performance of machine learning models trained on these annotations. Specifically, PHH3-assisted labeling could lead to the inclusion of more mitotic figures than would be identified using H&E alone, potentially causing false positives that could reduce the model’s accuracy.
Tumor diagnosis often depends on examining tissue samples to assess features like the proliferation rate, which is evaluated by counting mitotic figures (MFs). MFs are crucial for tumor grading and provide insights into the aggressiveness of the tumor. However, identifying MFs accurately is challenging due to low inter-rater agreement among pathologists. Advances in digital pathology, including the use of whole-slide imaging (WSI) and deep learning models, offer opportunities to improve annotation consistency and automate the detection of MFs.
The mitotic count (MC) is an important factor in grading tumors, and immunohistochemistry (IHC) using PHH3 staining is a well-established method for detecting MFs. Although PHH3 staining offers improved sensitivity for mitosis detection, it is more expensive than traditional H&E staining, making it less practical for cost-sensitive environments, like veterinary pathology.
To address this, some studies have proposed combining the sensitivity of PHH3 with machine learning tools to assist clinicians in annotating MFs more accurately. This approach has been explored in various cancer types, including colon cancer, prostate cancer, and canine breast cancer. However, the impact of PHH3-assisted labeling on inter-rater agreement and model performance has not been thoroughly studied. The authors hypothesized that using PHH3 might lead to a hindsight bias, where more MFs are annotated than would be with H&E alone, potentially affecting model performance during training.
To test this hypothesis, the study involved 13 pathologists who annotated MFs in 20 regions of interest (ROIs) across four tumor types. The pathologists annotated the slides both with and without PHH3 assistance. The resulting annotations were then used to train machine learning models, challenging the assumption that PHH3-assisted labeling provides the best possible ground truth for H&E-stained slides.
The study used the MIDOG dataset (from the 2022 MIDOG challenge) to train detectors. This dataset included 10 samples representing various cancers (e.g., canine hemangiosarcoma, feline lymphoma, human astrocytoma). The models were trained with annotations from both H&E-only slides and PHH3-assisted slides. The results showed that while PHH3-assisted labeling led to higher object-level agreement between pathologists, it did not improve the performance of H&E-based models. In fact, the H&E-only models performed better when trained with annotations from H&E-stained slides alone. Additionally, a dual-stain detector—trained on both H&E and PHH3 annotations—revealed that the additional information from PHH3 led to mismatches between the two stain types, further emphasizing the need for better labeling procedures.
The study concludes that while PHH3-assisted labeling may offer some benefits in terms of annotation consistency, it does not automatically improve model performance when applied to H&E-stained slides. The researchers suggest that an improved approach to PHH3-assisted labeling is needed to optimize model training and reduce biases introduced by PHH3.
