Prasath Lab @ CCHMC

Benchmarking HEp-2 Cell Segmentation Methods in Indirect Immuno-Fluorescence (IIF) Images - Standard Models to Deep Learning

Abstract:

In this work, we provide a systematic review of different human epithelial (HEp-2) cell segmentation methods on indirect immunofluresence (IIF) images. Our review covers traditional image processing (thresholding, Otsu, watershed, active contours), machine learning (auto-learning, random forest), and deep learning (DL) methods for segmenting HEp-2 cells. We extensively benchmark various DL cell segmentation models such as convolutional neural networks (CNNs), generative adversarial networks (GANs) for segmenting HEp-2 cells from IIF images. Our results indicate that DL models achieve better segmentation accuracy than traditional image processing, machine learning models with ResNet50-U-Net being the top model overall.

HE-2 Cell Segmentation - Existing Image Processing/Machine Learning/Deep Learning Models

We consider the standard image processing (thresholding, Otsu, watershed), machine learning (auto-learning, random forest), and deep learning (CNN, GAN) models for HEp-2 cell segmentation.

Table 1

Abbreviations used for metric evaluation: PVO - Percent Volume Overlap, PVD - Percent Volume Difference, ME - Misclassification Error, RAE - Relative Area Error, MHD - Modified Housdorff Distance, RDE - Relative Distance Error, SEG - Segmentation accuracy, SE - Sensitivity, JA - Jaccard index. Abbreviations used for Methods: SNFM - Stomped Normal Distribution and Finite Mixture based segmentation, RPrCM - Rough-probabilistic clustering and hidden Markov random field based segmentation method, FCRN-88 - Fully Convolutional Residual Network 88 layers, RouFS - Rough-fuzzy segmentation, cC-GAN - continuous conditional generative adversarial network, CSC - Cell Segmentation and Counting, DSFCN - Deeply Supervised Full Convolutional Network, sRFCM - spatially constrained rough-fuzzy c-means, SBF - Sliding Band Filter.

HE-2 Cell Segmentation - Benchmarking CNN/GAN Models

For cell segmentation, we benchmarked the different DL models: U-Net, VGG, ResNet, SegNet, MobileNet etc along with GAN-based approaches. Our results are evaluated in terms of Accuracy, Dice, F1-score, Jaccard, Precision, Sensitivity, Specificity, AUPR, AUROC.

Example segmentations

Classwise segmentation results on selected good GT HEp-2 images. (1) Input grayscale, (2) contrast enhanced (CLAHE), and corresponding (3) ground truth (GT) masks. Segmentation results of (4) HRNet, (5) MobileNet-UNet*, and (6) ResNet50-UNet models. (*Pretrained).

Table 2

Comparison of different DL models (no pretraining).

Bibliography: (28 papers so far)

Since the topic is new it is possible to trace everything that has been published. To the best of our knowledge this bibliography and systematic review represents all available publications on this topic to date, we will revise this document and bring it up to date once in six months.

Methods: deep learning, image processing/machine learning

2022 (1)

H. Xie, Y. He, D. Xu, J. Y. Kuo, H. Lei, and B. Lei. Joint segmentation and classification task via adversarial network: Application to HEp-2 cell images. Applied Soft Computing, vol. 114, p. 108156, 2022.

2021 (1)

G.-T. Jiang, Y.-D. Wu, T.-Y. Hsieh, and Y.-L. Huang. Automatic hep- 2 cell segmentation in indirect immunofluorescence images using deep learning. International Forum on Medical Imaging in Asia, vol. 11792. International Society for Optics and Photonics, 2021.

2020 (2)

I. Ul Islam, K. Ullah, M. Afaq, J. Iqbal, and A. Ali. Towards the automatic segmentation of HEp-2 cells in indirect immunofluorescence images using an efficient filtering based approach. Multimedia Tools and Applications, vol. 79, no. 45, pp. 34 325–34 337, 2020.

S. Roy and P. Maji. Medical image segmentation by partitioning spatially constrained fuzzy approximation spaces. IEEE Transactions on Fuzzy Systems, vol. 28, no. 5, pp. 965–977, 2020.

2019 (3)

H. Xie, H. Lei, Y. He, and B. Lei. Deeply supervised full convolution network for HEp-2 specimen image segmentation. Neurocomputing, vol. 351, pp. 77–86, 2019.

H. Xie, Y. He, H. Lei, J. Y. Kuo, and B. Lei. Segmentation guided HEp-2 cell classification with adversarial networks. Computing, Communications and IoT Applications (ComComAp). IEEE, 2019, pp. 374–379.

K. Gupta, A. Bhavsar, and A. K. Sao. A CNN based HEp-2 specimen image segmentation and identification of mitotic spindle type specimens. International Conference on Computer Analysis of Images and Patterns (CAIP). Springer, 2019, pp. 564–575.

2018 (2)

Y. Li and L. Shen. cC-GAN: A robust transfer-learning framework for HEp-2 specimen image segmentation. IEEE Access, vol. 6, pp. 14 048–14 058, 2018.

D. Riccio, N. Brancati, M. Frucci, and D. Gragnaniello. A new unsupervised approach for segmenting and counting cells in highthroughput microscopy image sets. IEEE Journal of Biomedical and Health Informatics, vol. 23, no. 1, pp. 437–448, 2018.

2017 (2)

Y. Li, L. Shen, and S. Yu. HEp-2 specimen image segmentation and classification using very deep fully convolutional network. IEEE Transactions on Medical Imaging, vol. 36, no. 7, pp. 1561–1572, 2017.

S. Roy and P. Maji. Rough-fuzzy segmentation of HEp-2 cell indirect immunofluorescence images. International Journal of Data Mining and Bioinformatics, vol. 17, no. 4, pp. 311–340, 2017.

2016 (6)

V. B. S. Prasath, Y. M. Kassim, Z. A. Oraibi, J.-B. Guiriec, A. Hafiane, G. Seetharaman, and K. Palaniappan. HEp-2 cell classification and segmentation using motif texture patterns and spatial features with random forests. International Conference on Pattern Recognition (ICPR), Cancun, Mexico, December 2016, pp. 90–95.

S. Roy and P. Maji. A modified rough-fuzzy clustering algorithm with spatial information for HEp-2 cell image segmentation. 2016 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). IEEE, 2016, pp. 383–388.

M. Merone and P. Soda. On using active contour to segment HEp-2 cells. IEEE 29th International Symposium on Computer-Based Medical Systems (CBMS), June 2016, pp. 118–123.

A.-D. Khamael, J. Banks, I. Tomeo-Reyes, and V. Chandran. Automatic segmentation of HEp-2 cell fluorescence microscope images using level set method via geometric active contours. 23rd International Conference on Pattern Recognition (ICPR), IEEE, 2016, pp. 81– 83.

A. Banerjee and P. Maji. Rough-probabilistic clustering and hidden Markov random field model for segmentation of HEp-2 cell and brain MR images. Applied Soft Computing, vol. 46, pp. 558–576, 2016.

X. Zhou, Y. Li, and L. Shen. A novel adaptive local thresholding approach for segmentation of HEp-2 cell images. IEEE International Conference on Signal and Image Processing (ICSIP). IEEE, 2016, pp. 174–178.

2015 (3)

A. Banerjee and P. Maji. Rough sets for finite mixture model based hep-2 cell segmentation. International Conference on Rough Sets and Knowledge Technology. Springer, 2015, pp. 459–469.

X. Jiang, G. Percannella, and M. Vento. A verification-based multithreshold probing approach to HEp-2 cell segmentation. International Conference on Computer Analysis of Images and Patterns. Springer, 2015, pp. 266–276.

S. Tonti, S. Di Cataldo, A. Bottino, and E. Ficarra. An automated approach to the segmentation of HEp-2 cells for the indirect immunofluorescence ana test. Computerized Medical Imaging and Graphics, vol. 40, pp. 62–69, 2015.

2014 (1)

Y.-K. Chan, D.-C. Huang, K.-C. Liu, R.-T. Chen, and X. Jiang. An automatic indirect immunofluorescence cell segmentation system. Mathematical Problems in Engineering, vol. 2014, 2014.

2013 (1)

C.-C. Cheng, T.-Y. Hsieh, J.-S. Taur, and Y.-F. Chen. An automatic segmentation and classification framework for anti-nuclear antibody images. Biomedical Engineering Online, vol. 12, no. 1, p. S5, 2013.

2012 (2)

G. Percannella, P. Soda, and M. Vento. A classification-based approach to segment HEp-2 cells. 25th IEEE International Symposium on Computer-Based Medical Systems (CBMS), June 2012, pp. 1–5.

B. Divya, H. Nanjundaswamy, and N. Muniraj. Segmentatation of IIF HEp-2 ANA cells based on thresholding and connectivity. Int. J. Microsyst. Technol. Appl., vol. 1, no. 1, pp. 26–30, 2012.

2011 (1)

C. Creemers, K. Guerti, S. Geerts, K. Van Cotthem, A. Ledda, and V. Spruyt. HEp-2 cell pattern segmentation for the support of autoimmune disease diagnosis. 4th International Symposium on Applied Sciences in Biomedical and Communication Technologies, 2011.

2010 (0)

2009 (1)

P. Elbischger, S. Geerts, K. Sander, G. Ziervogel-Lukas, and P. Sinah. Algorithmic framework for HEp-2 fluorescence pattern classification to aid auto-immune diseases diagnosis. IEEE International Symposium on Biomedical Imaging: From Nano to Macro (ISBI), IEEE, 2009, pp. 562–565.

2008 (2)

Y. Huang, C. Chung, T. Hsieh, and Y. Jao. Outline detection for the HEp-2 cell in indirect immunofluorescence images using watershed segmentation. IEEE International Conference on Sensor Networks, Ubiquitous, and Trustworthy Computing (SUTC), June 2008, pp. 423–427.

Y. Huang, Y. Jao, T. Hsieh, and C. Chung. Adaptive automatic segmentation of HEp-2 cells in indirect immunofluorescence images. IEEE International Conference on Sensor Networks, Ubiquitous, and Trustworthy Computing (SUTC), June 2008, pp. 418–422.

Available downloads:

1. PDF of the review paper version 1 (2023): TBA

2. Bibliography: BibTex (Please email us if you require a copy)

3. Each reference in PDF - Zip file ~100 Mb (Too big to attach here, please email us if you require a copy, we can send a Google Drive link for the whole data-set, results and send a particular paper's PDF via email)

4. Supplementary - Download - TBA

This page is continuously updated with relevant references/links.

Last updated: January 2025

Please kindly cite the main Reference below if you use any of the materials here.

Reference:

B. Iyer, S. Deoghare, K. Ranjan, B. J. Aronow, V. B. S. Prasath. Benchmarking HEp-2 cell segmentation methods in indirect immunofluorescence images - Standard models to deep learning. Computers in Biology and Medicine, 2025. doi:TBA [Code]

Notes:

This page (at:https://www.prasathlab.com/research/microscopy/HEp-2SegZoo) is a, chronologically ordered, bibliography of scientific publications on HEp-2 Cell Segmentation, compiled and continuously updated by Prasath Lab. If you know of a related work in any form (preprint, reprint, journal publication, conference proceedings, technical report, abstract or poster, book chapter, thesis, patent, unpublished report, etc.) that should be included here kindly write to us on: prasatsa at uc dot edu (or at smruti.deoghare at cchmc dot org) with full bibliographic details, a DOI if available, and a PDF copy of the work if possible. If any publication in this area is missed/overlooked please kindly let us know via email and our sincere apologies for missing it.

Copyright notice: Downloads are supplied for personal academic use only. A download is considered equivalent to a preprint or reprint request. Use is granted consistent with fair-use of a preprint or reprint. By downloading any of the materials here you are agreeing to these terms.

This page was last updated on January 2024.