3D Mitochondria Instance Segmentation with Spatio-Temporal Transformers

Medical Image Computing and Computer Assisted Intervention – MICCAI 2023: Proceedings of 26th International Conference, pp. 613-623, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) ; Volume 14227 LNCS

Abstract

Accurate 3D mitochondria instance segmentation in electron microscopy (EM) is a challenging problem and serves as a prerequisite to empirically analyze their distributions and morphology. Most existing approaches employ 3D convolutions to obtain representative features. However, these convolution-based approaches struggle to effectively capture long-range dependencies in the volume mitochondria data, due to their limited local receptive field. To address this, we propose a hybrid encoder-decoder framework based on a split spatio-temporal attention module that efficiently computes spatial and temporal self-attentions in parallel, which are later fused through a deformable convolution. Further, we introduce a semantic foreground-background adversarial loss during training that aids in delineating the region of mitochondria instances from the background clutter. Our extensive experiments on three benchmarks, Lucchi, MitoEM-R and MitoEM-H, reveal the benefits of the proposed contributions achieving state-of-the-art results on all three datasets. Our code and models are available at https://github.com/OmkarThawakar/STT-UNET.

Description

Keywords

Electron Microscopy, Hybrid CNN-Transformers, Mitochondria instance segmentation, Spatio-Temporal Transformer

DOI

10.1007/978-3-031-43993-3_59

Citation

Thawakar, O, Anwer, R M, Laaksonen, J, Reiner, O, Shah, M & Khan, F S 2023, 3D Mitochondria Instance Segmentation with Spatio-Temporal Transformers . in H Greenspan, H Greenspan, A Madabhushi, P Mousavi, S Salcudean, J Duncan, T Syeda-Mahmood & R Taylor (eds), Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 : Proceedings of 26th International Conference . Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 14227 LNCS, Springer, pp. 613-623, International Conference on Medical Image Computing and Computer Assisted Intervention, Vancouver, British Columbia, Canada, 08/10/2023 . https://doi.org/10.1007/978-3-031-43993-3_59

Permanent link to this item

https://urn.fi/URN:NBN:fi:aalto-202401041120

Collections

[article-cris] Perustieteiden korkeakoulu / SCI

Show all metadata

3D Mitochondria Instance Segmentation with Spatio-Temporal Transformers

Access rights

URL

Journal Title

Journal ISSN

Volume Title

Authors

Date

Department

Major/Subject

Mcode

Degree programme

Language

Pages

Series