A roadmap to scale connectomics to entire mammalian brains

Share to: X · LinkedIn

Introducing scalable connectomics with PRISM

Today, E11 Bio announces PRISM, our first step towards making brain connectivity mapping and reconstruction (connectomics) radically faster, cheaper and more accessible. This technology will allow brain researchers to affordably chart out millions of connections between individual cells in entire brains. We anticipate PRISM will be essential for scaling up whole-brain connectomics from small brains (e.g. flies[1]), to the large brains of mice and ultimately humans — a critical need for making more human-like AI, treating brain disorders, and even simulating our own brain circuitry.

As a philanthropically-funded non-profit, E11 Bio is uniquely positioned to make these breakthroughs openly available to researchers worldwide, maximizing the impact of funder support.

Here’s a preview of a pilot study using PRISM to map part of the hippocampus region of a mouse brain. Look for colorful structures visualizing individual pieces of brain cell wiring, and the glowing annotations denoting the synapses which connect brain cells together:

This study was a collaborative effort between the non-profit Focused Research Organization E11 Bio and the laboratories of Sam Rodriques at the Crick Institute, Joergen Kornfeld at Max Planck/LMB Cambridge UK, Ed Boyden at MIT and HHMI. Stay tuned — early next year, we will release a preprint with the full details of this study, as well as open methods for a beta version of PRISM that other brain researchers can use in their own laboratories.

What can we learn from the mind of a mammal?

The fly connectome is transforming neuroscience with detailed simulations of fly brain circuitry[2,3,4]. Achieving a similar feat for the mouse brain — which has much more human-like brain structures — would potentially:

• Reveal blueprints for powerful brain-inspired AI systems, unlocking human-like capabilities such as continual learning and energy efficiency[5], as well as enhanced AI safety based on the human brain[6];

• Unlock new treatments for brain disorders by identifying abnormal brain connectivity patterns in diseases[7], screen therapies that restore healthy connectivity[8], and guide advanced brain-computer interfaces[9];

• Make it possible to simulate our own brain circuitry in detail, and create an connectivity-based framework for how individual memories and experiences are stored in brain circuits[10].

We’re also inspired by forward thinking ideas that might sound like science fiction today, such as using connectomics to decipher how some human attributes such as creativity and curiosity might one day be responsibly recreated in artificial intelligence systems for the good of all humanity.

Some of these opportunities may be unlocked by a single connectome. But to understand the vast diversity of individual brains and brain disorders, many comparisons between connectomes are needed — spanning different states of development, health, and disease. These futures are only possible if mouse brain connectomics becomes routine and affordable.

Proofreading must be massively reduced as a cost bottleneck in connectomics

In order to unlock the opportunities above, we need much greater scale: a mouse brain is ~6500x larger than the fly’s. With current electron microscopy technology, scanning and reconstructing a single mouse connectome is projected to be more expensive than the entire human genome project[11]. Surprisingly, even though there are many steps in the connectomics pipeline — spanning sample prep, imaging, image processing, etc. — there is one major cost bottleneck: human proofreading. This step accounts for more than 95% of total project costs, and involves manual human error-correction and annotation of AI image segmentations. To scale connectomics to entire mammalian brains, the human proofreading bottleneck must be massively reduced.

Here are estimates for time and cost to scan and reconstruct one mouse connectome using electron microscopy-based technologies, adapted from a Wellcome Trust report [11]:

E11 Bio’s solution: PRISM, the first self-proofreading connectomics platform

To address this bottleneck, we are helping the brain proofread itself with our PRISM technology (Photo-connectomic Reconstruction by Iterative Staining with Molecular annotations). In PRISM, we instruct each cell to make a different combination of proteins which function as a cellular ID (barcode), and combine these barcodes with expansion microscopy[12] and specialized tissue staining procedures to radically improve the quality and interpretability of brain image data. This barcode-annotated image data then fuels “self-proofreading” AI image segmentation models that utilize these annotations to reduce the need for human proofreading, which could address 95% of current costs.

Beyond the proofreading bottleneck, PRISM also addresses other large cost drivers in the connectomics stack. By using light-sheet microscopes and expansion microscopy, we can achieve similar resolution and throughput as electron microscopy at 1/10th the cost[13]. In addition, by using GPU-efficient segmentation methods such as local shape descriptors[14], PRISM can achieve 10-100x lower costs for segmentation efficiency compared to the Wellcome Trust estimate above. By addressing the primary cost bottleneck of human proofreading, and additional cost drivers of image acquisition and image segmentation efficiency, we anticipate that PRISM can achieve at least a 100-fold reduction in the cost of whole-brain connectomics.

How does self-proofreading with PRISM work?

To create PRISM, we have developed and integrated three advances into a single platform: (1) genetic labeling using combinations of protein tags (protein barcoding), (2) expansion microscopy (including integration of our tools with recent advances, such as LICONN[15]), and (3) AI models for self-proofreading image segmentation. Here’s a preview of how each piece works – detailed methods forthcoming in a 2025 preprint.

Here’s a high-level schematic of the method:

1. Protein Cell Barcoding

We use genetic techniques to label neurons with a unique molecular signature composed of random combinations of protein tags, which can be detected as color combinations in light microscopy. These diverse colors make discriminating between adjacent neurons much easier. Our approach is inspired by Brainbow[16,17], where four distinct colors could be combined into a 100-combination palette. E11 Bio has developed new techniques to expand to more than 100,000 color combinations using protein epitope tags[18], with a roadmap for significant further scaling to millions and even billions of distinguishable combinations.

Here’s a video demonstrating the concept in mouse hippocampus: 18 different protein tags (”barcode bits”) are each shown in a different color channel, from which it is possible to construct 2^18 = 262,144 possible combinations. This is easy to illustrate visually at cell bodies, which we have highlighted. Note that the neurons appear to “flow” into the screen as we scroll from the top plane deeper into the brain.

We anticipate that with additional development this approach can be scaled to millions or even billions of distinguishable combinations to uniquely label every cell in the mouse brain. 

2. Optical connectomics with expansion microscopy

Historically, electron microscopy was the only imaging method with high enough resolution to detect brain connectivity. But advances in expansion microscopy now permit nanoscale brain morphology and connectivity to be imaged on low-cost light microscopes. E11 Bio has developed methods for highly multiplexed imaging in expansion microscopy to detect over 20 separate information channels — including color barcodes for self-proofreading, as well as molecular information such as cell type markers or ion channels.

Here is an example of a cycle of barcode imaging combined with high resolution optical morphology staining in mouse hippocampus:

3. Self-Proofreading Image Segmentation Models

Existing AI models for connectomic image analysis are not designed to leverage rich, multi-channel information, and rely instead on greyscale morphological information. E11 Bio has developed specialist AI image segmentation models that take full advantage of information-rich barcoded brain images, inspired by local shape descriptor approaches[14]. Here’s an example of local shape descriptors applied to light microscopy data, with raw image data on the left, and the automatic AI image segmentation on the right:

Scaling PRISM to entire mammalian brains

Today, we still have much work to do before optical connectomics with PRISM can be scaled up to large brain volumes or an entire mouse brain. But by eliminating human-dependent image analysis and proofreading as cost bottlenecks, an optical mouse connectome should be possible on a significantly shorter timescale than current estimates — we predict it could happen in just five years.

What we plan to do next at E11 Bio:

1. Demonstrate that PRISM can scale to increasingly large brain volumes. In early 2025, our initial preprint will document a beta version of our key methods, followed by the periodic release of increasingly large-scale optical connectomic data drops. We aim to optimize our methods and protocols to support a roughly 10x increase in barcode diversity and image volume each year, putting us on track for an entire mouse connectome within 5 years from today.

2. Demonstrate increasingly powerful self-proofreading AI image segmentation models. Our initial preprint will also document our current models. While they show a significant performance improvement versus greyscale segmentation, there’s still much work to be done if we want to eliminate proofreading as the bottleneck.

We are seeking ambitious and forward-thinking partners for:

1. Applications of connectomics data. We are particularly interested in early engagement with stakeholders who see the potential of mammalian connectomics datasets for developing more human-like AI systems, addressing brain disorders, or simulating brain function and physiology.

2. Increasing microscope and image processing capacity. To image increasingly large optical connectomic volumes each year, more and better microscopes and software are needed. Big steps in the right direction include global collaboration centers and observatories[19], as well as computational tools to process the resulting petascale data[20]. In addition, fundamental hardware advances such as the ExaSPIM pioneered at the Allen Institute[13] will be essential.

3. Better human proofreading tools leveraging barcode information. There will always be edge cases that aren’t caught by image segmentation models, even with automatic self-proofreading. Enhancing proofreading tools like Neuroglancer[21] and CAVE[22] to help human proofreaders leverage color barcode information will also be essential to eliminating proofreading as a bottleneck.

4. Real-time data processing methods. As we scale up to larger volumes, it can become prohibitively expensive to store the raw image data. It would cost more than $1B to store the raw image data for a single mouse connectome in the cloud[11]. This image data will need to be processed on the fly to compress it and reduce it to a more data-efficient representation.

5. Community benchmarks and leaderboards. This will help measure progress and speed up dissemination of the best methods across the community. The Neuron Instance Segmentation Benchmark is a recent big step in this direction[23].

These are just a few examples of what we think needs to happen — connectomics is an exciting and fast-moving field, and many additional opportunities are emerging. But, as we work with the rest of the field to overcome remaining optical and sample engineering challenges we believe it’s possible to achieve a 100-fold reduction in the cost of a mouse connectome in five years. The long-term cost floor may be even lower than that: after accounting for development costs and microscope capital equipment investment, the marginal cost of subsequent connectomes could be just tens of millions of dollars[24], or perhaps as low as $10 million[25].

Molecular connectomics with PRISM: a Rosetta Stone for the brain

An exciting additional opportunity unlocked by PRISM and other approaches to optical connectomics is the ability to capture molecular annotations in multi-cycle imaging. Capturing key molecular information like ion channel placement or molecular markers for cell types will be critical for increasingly realistic simulations of connectomes[2]. Another opportunity is to enable key biological abstractions like transcriptomic cell types to be linked to connectomes captured by optical or electron microscopy. The ability to create Rosetta Stone datasets which connect these different modalities will leverage large existing investments made by the BRAIN Initiative’s BICCN[26], BICAN[27] and CONNECTS[28] programs, as well as other pioneering cell and connectomic atlas efforts. Combining PRISM with in situ RNA profiling[29], or methods for RNA-based projection mapping[30] could be particularly helpful. We’re excited by the vast scientific potential unlocked by this opportunity and will do a deeper dive in future blog posts.

Here’s an example of raw all-optical imaging data of neuron morphology (greyscale), barcodes (blue) and synaptic labeling (green). Animation shows scanning through axial dimension with expansion microscopy on a confocal microscope.

E11 Bio: a fast-paced R&D program for whole-brain connectomics

To achieve our mission of transforming connectomics research, over the last two years we have assembled an all-star interdisciplinary team of scientists and engineers. Our team has previously:

• Scaled-up high-resolution imaging of neuronal wiring across entire mouse brains[31];

• Designed AI neuronal image segmentation models with 100x improved computational efficiency[14], and open-source connectomic analysis tools[32];

• Developed homing CRISPR techniques for barcoding millions of cells[33], and imaging methods to detect millions of barcodes in intact cells and tissues[34];

• Pioneered expansion microscopy methods, including increasing optical resolution by 10x[35], precisely calibrating expansion[36], and resolving nanoscale morphological detail[37].

Our unique structure as one of the first Focused Research Organizations has enabled us to move quickly and focus on the science. As a “non-profit startup”, we are easy to collaborate with, as we are rapidly sharing our results and technology using an Open Science philosophy for the benefit of brain researchers everywhere.

Join us in our mission to map the mouse connectome (and beyond)

E11 Bio is currently a small, focused team open to a wide range of possible collaborations to accelerate progress towards an optical connectome. We seek partners in scaled imaging, compute, and data storage as well as automation and instrumentation. This will drive ever-larger releases of optical connectomics data and analysis. To further accelerate technology development, we are eager to discuss potential applications of our tools and datasets for AI, biomedicine and other key commercial applications. 

Share to: X · LinkedIn

This work was accomplished by the E11 Bio team: Jun Axup, Stephanie Chan, Squirrel Collins, Hugo Damstra, Todd Huffman, Erin Jarvis, Kathleen Leeper, Clarence Magno, Aashir Meeran, Julia Michalska, Sung Yun (Rosa) Park, Arlo Sheridan, Sven Truckenbrodt, Johan Winnubst, Pranathi Vemuri, Michelle Wu; and our collaborators: Sam Rodriques, Ed Boyden, Daniel Leible, Bobae An, Joergen Kornfeld, Jan Funke, Will Patton, Davis Bennett.

Thanks to David Markowitz, Jane Roskams, Adam Marblestone, Sam Rodriques, Joergen Kornfeld, Ed Boyden, George Church, Hemai Parthasarathy, Stephen Smith, Todd Huffman, and the E11 Bio team for reading and providing feedback on drafts of this roadmap. 

References

1. Dorkenwald S, Matsliah A, Sterling AR, Schlegel P, Yu S-C, McKellar CE, et al. Neuronal wiring diagram of an adult brain. Nature. 2024;634: 124–138. doi: 10.10.1038/s41586-024-07558-y

2. Lappalainen JK, Tschopp FD, Prakhya S, McGill M, Nern A, Shinomiya K, et al. Connectome-constrained networks predict neural activity across the fly visual system. Nature. 2024;634: 1132–1140. doi: 10.1038/s41586-024-07939-3

3. Pospisil DA, Aragon MJ, Dorkenwald S, Matsliah A, Sterling AR, Schlegel P, et al. The fly connectome reveals a path to the effectome. Nature. 2024;634: 201–209. doi: 10.1038/s41586-024-07982-0

4. Shiu PK, Sterne GR, Spiller N, Franconville R, Sandoval A, Zhou J, et al. A Drosophila computational brain model reveals sensorimotor processing. Nature. 2024;634: 210–219. doi: 10.1038/s41586-024-07763-9

5. Zador A, Escola S, Richards B, Ölveczky B, Bengio Y, Boahen K, et al. Catalyzing next-generation Artificial Intelligence through NeuroAI. Nature Communications. 2023;14: 1–7. doi: 10.1038/s41467-023-37180-x

6. Mineault P, Zanichelli N, Peng JZ, Arkhipov A, Bingham E, Jara-Ettinger J, et al. NeuroAI for AI Safety. 2024. Available:http://arxiv.org/abs/2411.18526

7. Del Pino I, Rico B, Marín O. Neural circuit dysfunction in mouse models of neurodevelopmental disorders. Curr Opin Neurobiol. 2018;48: 174–182. doi: 10.1016/j.conb.2017.12.013

8. Revah O, Gore F, Kelley KW, Andersen J, Sakai N, Chen X, et al. Maturation and circuit integration of transplanted human cortical organoids. Nature. 2022;610: 319–326. doi: 10.1038/s41586-022-05277-w

9. Brown J, Zappitelli KM, Dawson PM, Yoon E, Schiraga SA, Rochford AE, et al. Optogenetic stimulation of a cortical biohybrid implant guides goal directed behavior. bioRxiv. 2024. p. 2024.11.22.624907. doi: 10.1101/2024.11.22.624907

10. Abbott LF, Bock DD, Callaway EM, Denk W, Dulac C, Fairhall AL, et al. The Mind of a Mouse. Cell. 2020;182. doi: 10.1016/j.cell.2020.08.010

11. Scaling up connectomics. In: Wellcome [Internet]. [cited 1 Dec 2024]. Available: https://wellcome.org/reports/scaling-connectomics

12. Chen F, Tillberg PW, Boyden ES. Optical imaging. Expansion microscopy. Science. 2015;347: 543–548.doi: 10.1126/science.1260088

13. Glaser A, Chandrashekar J, Vasquez S, Arshadi C, Ouellette N, Jiang X, et al. Expansion-assisted selective plane illumination microscopy for nanoscale imaging of centimeter-scale tissues. eLife. 2024;12. doi: 10.7554/eLife.91979.2

14. Sheridan A, Nguyen TM, Deb D, Lee W-CA, Saalfeld S, Turaga SC, et al. Local shape descriptors for neuron segmentation. Nature Methods. 2022;20: 295–303. doi: 10.1038/s41592-022-01711-z

15. Tavakoli MR, Lyudchik J, Januszewski M, Vistunou V, Agudelo N, Vorlaufer J, et al. Light-microscopy based dense connectomic reconstruction of mammalian brain tissue. bioRxiv. 2024. p. 2024.03.01.582884. doi: 10.1101/2024.03.01.582884

16. Livet J, Weissman TA, Kang H, Draft RW, Lu J, Bennis RA, et al. Transgenic strategies for combinatorial expression of fluorescent proteins in the nervous system. Nature. 2007;450: 56–62. doi: [10.1038/nature06293

17. Cai D, Cohen KB, Luo T, Lichtman JW, Sanes JR. Improved tools for the Brainbow toolbox. Nature Methods. 2013;10: 540–547. doi: 10.1038/nmeth.2450

18. Wroblewska A, Dhainaut M, Ben-Zvi B, Rose SA, Park ES, Amir E-AD, et al. Protein Barcodes Enable High-Dimensional Single-Cell CRISPR Screens. Cell. 2018;175: 1141–1155.e16. doi: 10.1016/j.cell.2018.09.022

19. Betzig E. A Cell Observatory to reveal the subcellular foundations of life. Nature methods. 2024 [cited 1 Dec 2024]. doi: 10.1038/s41592-024-02379-3

20. Ruan X, Mueller M, Liu G, Görlitz F, Fu T-M, Milkie DE, et al. Image processing tools for petabyte-scale light sheet microscopy data. Nature Methods. 2024; 1–11. doi: 10.1038/s41592-024-02475-4

21. GitHub - google/neuroglancer: WebGL-based viewer for volumetric data. In: GitHub [Internet]. [cited 1 Dec 2024]. Available: https://github.com/google/neuroglancer

22. Dorkenwald S, Schneider-Mizell CM, Brittain D, Halageri A, Jordan C, Kemnitz N, et al. CAVE: Connectome Annotation Versioning Engine. bioRxiv. 2023. p. 2023.07.26.550598. doi: 10.1101/2023.07.26.550598

23. NISB: Neuron Instance Segmentation Benchmark. [cited 1 Dec 2024]. Available: https://structuralneurobiologylab.github.io/nisb/

24. Marblestone, Adam H. On whole-mammalian-brain connectomics. In: Longitudinal Science [Internet]. 1 Jan 2019 [cited 1 Dec 2024]. Available: https://longitudinal.blog/2019/01/01/on-whole-mammalian-brain-connectomics/

25. Optical Microscopy Provides a Path to a $10M Mouse Brain Connectome (if it eliminates proofreading). In: Sam Rodriques [Internet]. 15 Jul 2023 [cited 1 Dec 2024]. Available: https://www.sam-rodriques.com/post/optical-microscopy-provides-a-path-to-a-10m-mouse-brain-connectome-if-it-eliminates-proofreading

26. BRAIN Initiative Cell Census Network - Brain Cell Data Center (BCDC). [cited 1 Dec 2024]. Available: https://www.biccn.org/

27. Home. In: BICAN [Internet]. [cited 1 Dec 2024]. Available: https://www.portal.brain-bican.org/

28. BRAIN Initiative Connectivity Across Scales. [cited 1 Dec 2024]. Available: https://braininitiative.nih.gov/research/neuroimaging-technologies-across-scales/brain-initiative-connectivity-across-scales

29. Alon S, Goodwin DR, Sinha A, Wassie AT, Chen F, Daugharthy ER, et al. Expansion sequencing: Spatially precise in situ transcriptomics in intact biological systems. Science. 2021;371. doi: 10.1126/science.aax2656

30. Yuan L, Chen X, Zhan H, Henry GL, Zador AM. Massive multiplexing of spatially resolved single neuron projections with axonal BARseq. Nature Communications. 2024;15: 1–17. doi: 10.1038/s41467-024-52756-x

31. Winnubst J, Bas E, Ferreira TA, Wu Z, Economo MN, Edson P, et al. Reconstruction of 1,000 Projection Neurons Reveals New Cell Types and Organization of Long-Range Connectivity in the Mouse Brain. Cell. 2019;179. doi: 10.1016/j.cell.2019.07.042

32. Schubert PJ, Dorkenwald S, Januszewski M, Klimesch J, Svara F, Mancu A, et al. SyConn2: dense synaptic connectivity inference for volume electron microscopy. Nat Methods. 2022;19: 1367–1370. doi: 10.1038/s41592-022-01624-x

33. Leeper K, Kalhor K, Vernet A, Graveline A, Church GM, Mali P, et al. Lineage barcoding in mice with homing CRISPR. Nature Protocols. 2021;16: 2088–2108. doi: 10.1038/s41596-020-00485-y

34. Payne AC, Chiang ZD, Reginato PL, Mangiameli SM, Murray EM, Yao C-C, et al. In situ genome sequencing resolves DNA sequence and structure in intact biological samples. Science. 2021;371. doi: 10.1126/science.aay3446

35. Truckenbrodt S, Sommer C, Rizzoli SO, Danzl JG. A practical guide to optimization in X10 expansion microscopy. Nature Protocols. 2019;14: 832–863. doi: 10.1038/s41596-018-0117-3

36. Damstra HGJ, Passmore JB, Serweta AK, Koutlas I, Burute M, Meye FJ, et al. GelMap: intrinsic calibration and deformation mapping for expansion microscopy. Nature Methods. 2023;20: 1573–1580. doi: 10.1038/s41592-023-02001-y

37. Michalska JM, Lyudchik J, Velicky P, Štefaničková H, Watson JF, Sommer C, et al. Imaging brain tissue architecture across millimeter to nanometer scales. Nature Biotechnology. 2023;42: 1051–1064. doi: 10.1038/s41587-023-01911-8