High Performance Computing Resources for Parallel Simulations and Data Analysis – NSG and HPAC

 

Saturday, 3 November, 2018 | 08:30 - 12:30 
SfN 2018, San Diego, California

 

WORKSHOP ORGANIZERS

Amit Majumdar
San Diego Supercomputer Center, University of California San Diego, La Jolla, CA, USA

Subhashini Sivagnanam
San Diego Supercomputer Center, University of California San Diego, La Jolla, CA, USA

Kenneth Yoshimoto
San Diego Supercomputer Center, University of California San Diego, La Jolla, CA, USA

Ted Carnevale
Department of Neuroscience, Yale University, New Haven, CT, USA

Alexander Peyser
Jülich Supercomputing Center, Forschungszentrum Jülich, Germany

Abstract of the workshop:

The Neuroscience Gateway Portal (NSG https://www.nsgportal.org/) eliminates most administrative and technical barriers, providing free CPU time to users, and easy access to widely used software that currently includes BluePyOpt, Brian, CARLsim, DynaSim, EEGLAB, Freesurfer, GENESIS, Human Neocortical Neuroserver (HNN), Large Scale Neural Modeling Simulator (LSNM), MATLAB, MOOSE, NEST, NetPyNE, NEURON, Octave, PyNN, Python, R, TensorFlow, and the Virtual Personalized Multimodal Connectome Pipeline. NSG's web-based interface simplifies the tasks of uploading models or data, specifying job parameters, monitoring job status, and storing and retrieving output data. The Human Brain Project's High Performance Analytics and Computing Platform (HPAC https://hbp-hpc-platform.fz-juelich.de/) provides extensive HPC support for the HBP community. This includes developing and providing the hardware and software infrastructure required for large scale simulations, data management and analysis, visualization, and managing the complex workflows involved in performing these tasks. The workshop will combine hands-on instruction on how to use NSG with didactic presentations by NSG and HPAC developers and discussions with experienced users of these resources.

Title of Talks and Names of Speakers:

08:30 – 09:00 Introduction to the Neuroscience Gateway and High Performance Analytics and Computing Platform, Amit Majumdar, Subhashini Sivagnanam, Kenneth Yoshimoto, San Diego Supercomputer Center, UCSD; Ted Carnevale, Department of Neuroscience, Yale University, USA; Alexander Peyser, Jülich Supercomputing Center, Forschungszentrum Jülich, Germany

09:00 – 09:30 Towards a Complete Description of the Hippocampal Circuitry Underlying Sharp Wave-Mediated Memory Replay, Ivan Soltesz, Stanford University, USA

09:30 – 10:00 Human Neocortical Neurosolver: A New Modeling Platform for Cellular and Circuit Level Interpretation of EEG/MEG, Samuel A Neymotin1,2, Dylan S Daniels1, Noam Peled3, Robert A McDougal5, Nicholas T Carnevale5, Christopher I Moore1, Michael L Hines5, Matti Hamalainen3, Stephanie R Jones1,4

1Dept. Neuroscience and Carney Institute for Brain Science, Brown University, Providence RI, 2Nathan Kline Institute for Psychiatric Research, Orangeburg, NY, 3Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Boston MA, 4Center for Neurorestoration and Neurotechnology, Providence VAMC, 5Dept. Neuroscience, Yale University, USA

10:00 – 10:30 The Virtual Brain: personalized large-scale brain network modeling and its application, Viktor Jirsa, INS, Aix-Marseille University Inserm, Marseille, France

10:30 – 10:40 COFFEE BREAK

10:40 – 11:10 Large scale dynamics of the thalamocortical networks in awake and sleep states, Maksim Bazhenov, Giri Prashanth, University of California San Diego, USA

11:10 – 11:40 Neuroanatomical models beyond the spatial resolution of MRI require HPC: The Big Brain and fiber tracts. Karl Zilles1,2, Nicola Palomero-Gallagher1,2 , and Katrin Amunts1,3 , 1Institute of Neuroscience and Medicine, Research Center Jülich, Jülich, Germany, 2Department of Psychiatry, Psychotherapy and Psychosomatics, RWTH Aachen University, Aachen, Germany, 3C. and O. Vogt Institute for Brain Research, Heinrich Heine University, Düsseldorf, Germany

11:40 – 12:00 Open discussions

12:00 – 12:30 Lunch

SCHEDULE OF EVENTS

08:30 - 09:00 

Introduction to the Neuroscience Gateway and High Performance Analytics and Computing Platform

Amit Majumdar, Subhashini Sivagnanam, Kenneth Yoshimoto
San Diego Supercomputer Center, University of California San Diego, La Jolla, CA, USA

Ted Carnevale
Department of Neuroscience, Yale University, New Haven, CT, USA

Alexander Peyser
Jülich Supercomputing Center, Forschungszentrum Jülich, Germany

We will provide a brief overview of the Neuroscience Gateway, including all the tools, software and pipelines that are provided via NSG on supercomputing resources. We will describe how users are using the NSG, and how developers interact with the NSG and the NSG team to implement their tools, software and pipelines. We will describe the various research and development activities ongoing at the High Performance Analytics and Computing Platform of the Human Brain Project.

09:00 - 09:30 

Towards a Complete Description of the Hippocampal Circuitry Underlying Sharp Wave-Mediated Memory Replay 

Ivan Soltesz, Professor
Stanford University, USA

Our team is making the first attempt to fully understand a cognitively important event, called memory replay during hippocampal sharp-wave ripples, in terms of the detailed properties of the brain cells involved. We employ large-scale recording technologies to study and manipulate identified cell types in the behaving animal and construct the first data-driven full-scale computational model of the hippocampus in which every cell is explicitly simulated in supercomputers. These powerful new approaches are likely to yield major insights into the principles by which the interactions of neurons give rise to cognitive function, with important implications for memory disorders and cognitive comorbidities in a variety of neuropsychiatric and neurological disorders. Supported by the BRAIN Initiative U19 NS104590. 

09:30 - 10:00 

Human Neocortical Neurosolver: A New Modeling Platform for Cellular and Circuit Level Interpretation of EEG/MEG

Samuel A Neymotin1,2, Dylan S Daniels1, Noam Peled3, Robert A McDougal5, Nicholas T Carnevale5, Christopher I Moore1, Michael L Hines5, Matti Hamalainen3, Stephanie R Jones1,4
1Dept. Neuroscience and Carney Institute for Brain Science, Brown University, Providence RI, 2Nathan Kline Institute for Psychiatric Research, Orangeburg, NY, 3Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Boston MA, 4Center for Neurorestoration and Neurotechnology, Providence VAMC, 5Dept. Neuroscience, Yale University, USA

We developed the Human Neocortical Neurosolver (HNN; https://hnn.brown.edu), an open-source software tool designed to help researchers interpret cellular and circuit origins of EEG/MEG. HNN presents a user-friendly graphical user interface (GUI) to a large-scale biophysically principled model of a neocortical circuit, under thalamic and cortical drive, that simulates the primary electrical currents underlying EEG/MEG recordings. The Neuroscience Gateway Portal team recently deployed HNN on the JetStream server, allowing full GUI access through VNC. Availability of HNN in this high performance compute environment enables researchers to run large-scale thalamocortical models, and bypasses the need for individuals to install HNN on their own computers. In this talk, we will give background information on the use of HNN, describe HNN’s underlying model and GUI structure, and provide examples on how to use HNN to study the origins of commonly measured signals, including event related potentials and low frequency rhythms (alpha/beta/gamma). Participants will see how HNN can be used to compare model results to recorded data and to adjust parameters to develop and test hypotheses on the circuit-level origins of their data. 

10:00 - 10:30

The Virtual Brain: personalized large-scale brain network modeling and its application

Viktor Jirsa

INS, Aix-Marseille University Inserm, Marseille, France

The last decade has shown a large growth of data, models and tools on the system level of the brain. Structural and functional data of « mice and men » are organized on the meso and macro scale, placed into the same reference framework, parcellated and reconstructed in 3D physical space, and modeled using neural mass and neural field techniques with sufficient resolution to predict empirically measured brain signals in various modalities including optical imaging, electro- and magnetoencephalographic and functional Magnetic resonance Imaging (fMRI) signals.  The open source neuroinformatics platform The Virtual Brain (TVB) integrates the population and network level modeling tools – preprocessing pipelines, source level simulators, data management and observer level imaging – with Bayesian inference tools (STAN using Hamiltonian Monte Carlo and variational inference (ADVI)) for personalization of brain network models. We will demonstrate the full workflow from patient brain images to virtual brain models and their usage in clinical environments.

10:30 - 10:40 COFFEE BREAK

 

10:40 - 11:10 

Large scale dynamics of the thalamocortical networks in awake and sleep states  

Maksim Bazhenov, Giri Prashanth
University of California San Diego, USA

Sleep takes up about one-third of our life, yet we are still trying to understand its function. Computational models have shown promising results in understanding how neural activity get organized during sleep in brain networks. We will present a large-scale computational model of the thalamocortical network that implements effects of neuromodulation to simulate transitions between awake and sleep like activity. We will describe the computational challenges and the approaches we used to simulate large-scale network dynamics. We will show that such large-scale network model could explain emergence of oscillatory activity observed in EEG and MEG recordings in humans and also provide insight on how activity during sleep contributes to memory consolidation. 

11:10 - 11:40 

Neuroanatomical models beyond the spatial resolution of MRI require HPC: The Big Brain and fiber tracts 

Karl Zilles1,2, Nicola Palomero-Gallagher1,2 , and Katrin Amunts1,3
1Institute of Neuroscience and Medicine, Research Center Jülich, Jülich, Germany, 2Department of Psychiatry, Psychotherapy and Psychosomatics, RWTH Aachen University, Aachen, Germany, 3C. and O. Vogt Institute for Brain Research, Heinrich Heine University, Düsseldorf, Germany

The Big Brain is a neuroanatomical human reference brain based on contiguous, cell body stained serial sections (n=7,404) of 20µm thickness (Amunts et al. 2013). All sections were digitized at a resolution of 20µm in plane and 3D reconstructed. Thus, it allows extracting data at microscopic resolution for modeling and simulation. BigBrain is a necessary prerequisite to understand the neuroanatomical basis of various brain functions, and to bridge the gap between large-scale neural networks and local circuitry in the cerebral cortex. As planned, to create a volume with a spatial resolution of 1µm for the human brain would result in a data volume of approximately 21,000 TByte. The interactive exploration (not the simple storage) of such data is beyond the capacities of current computing. Among other methodological problems, data processing becomes a major challenge for the future project aiming at the reconstruction of a human brain at cellular resolution. Even larger challenges are caused by our current project of fiber tract visualization in the monkey brain. Up to now, we succeeded in the 3D reconstruction of myelinated nerve fibers and fiber tracts visualized by using Polarized Light Imaging (PLI) in the human hippocampus (Zeineh et al. 2017) and in an entire rat brain (Axer et al. 2011, Schubert et al. 2016) at an in-plane resolution of 1.3µm. We are presently working on an entire monkey hemisphere for identifying fiber tracts at this microscopic resolution.

Acknowledgments: The Big Brain project was supported by the Portfolio project “Supercomputing and Modelling the Human Brain”, funded by the Helmholtz Association, Germany. The monkey PLI project is supported by funding from the European Union’s Horizon 2020 Research and Innovation Programme under Grant Agreement No. 785907 (HBP SGA2)

References

Amunts, K., Lepage, C., Borgeat, L., Mohlberg, H., Dickscheid, T., Rousseau, M.-E., Bludau, S., Lewis, L., Oros-Peusquens, A.-M., Bazin, P.-L., Shah, N.J., Lippert, T., Zilles, K., Evans, A. (2013). The BigBrain – an ultra-high resolution 3D human brain model. Science 340: 1472-1475.

Axer, M., Amunts, K., Gräßel, D., Palm, C., Dammers. J., Axer, H., Pietrzyk, U., Zilles, K. (2011). A novel approach to the human connectome: ultra-high resolution mapping of fiber tracts in the brain. Neuroimage. 54:1091–1101.

Schubert, N., Axer, M., Schober, M., Hy, A.-M., Huysegoms, M., Palomero-Gallagher, N., Bjaalie, J.G., Leergaard, T.G., Amunts, K., Zilles, K. (2016). 3D Reconstructed cyto-, receptor- an fiberarchitecture of the rat brain registered to the Waxholm Space Atlas. Front. Neuroanat., http://dx.doi.org/10.3389/fnana.2016.00051

Zeineh, M.M., Palomero-Gallagher, N., Axer, M., Gräβel, D., Goubran, M., Wree, A., Woods, R., Amunts, K., Zilles, K. (2017). Direct visualization and mapping of the spatial course of fiber tracts at microscopic resolution in the human hippocampus. Cerebral Cortex 27(3):1779-1794.

11:40 - 12:30 Open discussions and lunch