Researchers at University College London report in an eLife study that activity from the visual cortex of mice can be used to reconstruct short movies the animals had watched. The work turns patterns of neural activity into moving images, giving scientists a new way to study how the brain represents the visual world.
The team focused on mouse visual cortex activity, recorded from individual neurons while mice watched natural video clips. After training a model on the relationship between movies and brain responses, the researchers reconstructed 10-second clips from neural activity alone. The result was a recognizable video that tracked the original scene with surprising accuracy.
The study was led by Dr. Joel Bauer at the Sainsbury Wellcome Centre at UCL, with Troy W. Margrie and Claudia Clopath. Their approach uses detailed recordings from single cells, rather than broad brain activity signals. That gives researchers a more direct window into how groups of neurons carry information about moving scenes.
“We wanted to have a better way of investigating how the brain interprets what we see,” said Dr. Bauer. The method could help scientists compare the outside world with the brain’s internal representation of it, frame by frame.
Brain signals became 10-second videos
The study used brain recordings from mice that had watched short natural movies. These recordings came from the visual cortex, the brain region that handles incoming visual information. The scientists then asked whether those signals contained enough detail to rebuild the moving image that had produced them.
They found that they could reconstruct 10-second video clips from the recorded activity. The clips were reconstructed at 30 frames per second, which meant the model had to recover both spatial detail and the timing of motion. That combination makes the result especially useful for studying real visual processing.
The work relied on two-photon calcium imaging, a technique that lets researchers monitor the activity of many individual brain cells. When neurons become active, calcium signals inside the cells change. Those changes can be detected with microscopy and used as a readout of neural activity.
Dr. Bauer said, “Using this approach, we were able to achieve high-quality reconstructions of 10-second video clips.” In the study, the reconstructed videos were compared with the original clips that had been shown to the mice. The researchers measured that match with pixel-level correlation, which compares each pixel in one video with the corresponding pixel in the other.
How the model rebuilt each frame
At the center of the work was a dynamic neural encoding model. This kind of model learns how visual input relates to brain activity over time. In practical terms, it predicts how neurons should respond when a mouse sees a particular movie.
The UCL team worked with a model developed for the 2023 Sensorium Competition. It took in information about the movie, along with measurements such as pupil diameter and the animal’s movements. Those extra signals matter because the state of the animal can influence how neurons respond to the same visual scene.
The researchers then used the model in reverse. They started with a blank movie and repeatedly adjusted its pixels. Each adjustment was guided by how closely the model’s predicted neural activity matched the real neural activity recorded from the mouse.
That process gradually shaped the blank movie into a reconstruction. The final video was the one whose predicted brain response most closely matched the actual activity pattern. In plain language, the model searched for the movie that best explained what the neurons had done.
This method gave researchers a way to move from neural signals back toward visual content. The approach was tested on videos that had been left out during model training. That step helped show that the system could work on new clips within the study setup.
More neurons made the movies sharper
One clear lesson from the study was that more recorded cells improved the reconstruction. The researchers found that the number of neurons in the dataset was critical for high-quality output. More cells meant more information about the visual scene.
The dataset included recordings from thousands of neurons per mouse. According to the study, the researchers used publicly available Sensorium data that included activity from primary visual cortex neurons, pupil measurements and running speed. Together, these signals helped the model connect the movie to the animal’s brain state.
The team also used model ensembling, a strategy that combines multiple versions of a model. Each model can capture slightly different parts of the relationship between visual input and neural response. Combining them can make the final reconstruction more reliable.
In the eLife paper, the authors reported a pixel-level correlation of 0.57 between the original movies and single-trial reconstructions. That means the reconstruction captured a substantial amount of visual structure. The value also gives scientists a quantitative way to compare future methods with this one.
The finding points to a simple constraint for brain decoding work. Better recordings can make better reconstructions. Wider coverage of the visual cortex and more detailed signals could help future versions recover sharper images and larger parts of the visual scene.
Why perception can differ from reality
The deeper goal of the study goes beyond making a video from brain signals. The researchers want to understand how the brain transforms visual input into perception. The reconstruction method gives them a tool for seeing where that transformation changes the original scene.
Dr. Bauer put the idea directly: “We don’t have a perfect representation of the world in our heads.” The brain filters, emphasizes and modifies sensory information. Some visual features may become stronger in the neural representation, while others may fade.
That difference between the stimulus and the reconstruction could become scientifically valuable. If a movie contains motion, contrast, shapes, or textures, the reconstructed version may reveal which parts the visual system represented most strongly. Those gaps can help researchers infer how the brain organizes information.
The approach may also help scientists study visual perception across species. Mice have a different visual world from humans. By reconstructing what their visual cortex represents, researchers can begin to compare brain processing across animals in a more direct way.
Because the work was done in mice, its findings should be understood within that setting. The study shows a powerful research tool for animal neuroscience. Any direct application to human experience would require separate evidence from human studies.
What researchers want to decode next
The team now plans to improve the method by increasing resolution and visual coverage. Sharper reconstructions would help reveal finer details in the represented scene. Broader coverage could capture more of what the animal sees at once.
The current study focused on activity from the visual cortex. Future work could test how other brain areas shape the representation of a scene. Vision involves more than the first cortical stages, since attention, movement and context can all influence what the brain encodes.
The method also gives researchers a way to test specific questions about perception. They could show mice carefully designed videos and then examine how the reconstructed version differs from the original. That could reveal how the brain handles edges, motion, depth cues and changing light.
In the paper’s abstract, the authors wrote, “This paves the way for movie reconstruction to be used as a tool to investigate a variety of visual processing phenomena.” The phrase captures the main promise of the work. Reconstructed videos can become measurements of brain representation, rather than only demonstrations of decoding power.
For now, the achievement is a striking example of how detailed neural data and modern modeling can work together. A mouse watches a short movie. Thousands of neurons respond. From those signals, scientists can rebuild a moving trace of what the visual brain had just seen.
