Large-Scale Multi-Modal Visualization

Large-Scale Multi-Modal Visualization

​This research theme comprises two research projects, Multi-Modal Biological Understanding and Visualization, and Large-Scale Connectomics.

Multi-Modal Biological Understanding and Visualization

This multi-disciplinary project investigates brain cells and tissues using image analysis of biological cells and tissues by applying a Multi-Modality imaging and visualization platform ("MMI platform"). This imaging platform is implemented by jointly developing ideas and setups supporting multimodality optical microscope imaging strategies. Cutting-edge technologies are conjugated to scientific competencies in order to support innovative analyses of neuronal structure and function. Historically, optical microscopy has been one of the most productive scientific instruments in technology and medicine. Nevertheless, some limitations of optical microscopy appeared soon. In particular the lack of resolution formulated by the well-known Abbe law, as well as the lack of quantitative information.

Various approaches have been developed in order to get over these basic limitations: electron or optical microscopy, with wavelengths extending from infrared to EUV (extreme UV and X-ray) have been applied to all modern micro- and nanotechnologies. Nowadays, we observe a strong revival of optical microscopy in biology and medicine. The techniques developed in image understanding of multi-modal microscopy data is expected to impact neurosciences. In particular, they are expected to aid therapies in combatting against neurodegeneration, dementia and depression, which is of prime importance in contemporary civilization. Prof. Magistretti (KAUST) has internationally shown that failure of the neuron-glia system can lead to severe nervous system diseases and chronic degeneracies. The benefit of this research is important to the extent that it affects a focal point for advances in neuroscience that could directly impact the therapies of tomorrow neurology, in particular as far as social welfare is concerned. In this context, imaging the details of the molecular mechanisms at the basis of the neuron-glia coupling is of crucial importance. Furthermore, the expected developments in Multi-Modality Microscopy and data visualization will substantially promote the use of microscopy as an investigation tool in neurosciences and biomedical sciences in general.

Large-Scale Connectomics

This project focuses on the visual analysis, exploration, and interactive segmentation of neural structures in mammalian brain tissue from extremely high-resolution electron microscopy (EM) and confocal microscopy scans. Reconstructing the anatomical and functional connectivity within the brain has become one of the most active research areas in neuroscience. Our collaborators at the Harvard Center for Brain Science hope that by ultimately mapping and deciphering a human's entire connectome, i.e., the full "wiring diagram" of the brain comprising billions of neurons and their interconnections, they will be able to gain an understanding of how the brain develops and functions, and how pathologies develop or can be treated. The involved data sets are volumes of extremely high resolution that are created by electron microscopy, resulting in multiple to hundreds of terabytes or even petabytes per volume ("petascale" data). This extreme size and the complexity of the structures contained in these data present significant challenges for segmentation, visualization, and analysis. The Harvard Center for Brain Science has been able to achieve very promising early results, but a major obstacle going forward is handling the enormous size of the acquired data sets. We propose research related to several essential goals for visual computing in connectomics, ranging from interactive proofreading of dense segmentations of neurons in EM data, connectivity analysis of these large-scale and multi-level connected graphs, to efficient out-of-core and streaming-based approaches for a truly visualization-driven acquisition pipeline for connectome datasets.

Research Projects

  • Multi-Modal Biological Understanding and Visualization

  • Large-Scale Connectomics


Related Publications