Analysis, Visualization & Discovery
Feb. 15, 2017 3:30pm — Johnson 102
Distinguished Young Academic Data Scientists (DYADS) speaker series

Cross-modal semantic representation in the human brain

Data Science Fellow, Berkeley Institute for Data Science, UC Berkeley


An integral part of human language is the capacity to extract the meaning of words through different sensory modalities. For example, humans can easily comprehend the meaning of language presented through auditory speech or through written text. It has been shown that semantic information from spoken language is represented in a broad network of semantically-selective areas distributed across the human cerebral cortex. However, it is unclear whether these representations are specific to the modality of speech or whether they are modality invariant to other presentation modalities of language, such as text. In this talk, I will present our recent findings on how the human brain represents the semantic content of narratives received through two different modalities, listening and reading. Our results suggest that semantic representation of language outside of early sensory areas is independent of the specific modality through which the semantic information is received. I will further discuss a data-drive framework to analyze functional magnetic resonance imaging data and suggest workflows for reproducible neuroimaging experiments.


Fatma Deniz is a Moore-Sloan Data Science Fellow in Berkeley Institute for Data Science and a Postdoctoral-Fellow in Dr. Jack Gallant’s laboratory (Helen Wills Neuroscience Institute and International Computer Science Institute) at UC Berkeley. She is interested in how sensory information is encoded in the brain and uses machine-learning approaches to fit computational models to large-scale brain data. Her current work focuses on cross-modal language representation in the human brain. She did her PhD in Dr. John-Dylan Haynes’s laboratory at Bernstein Center for Computational Neuroscience and Technical University Berlin, where she studied functional connectivity changes during conscious perception in humans. She got a bachelor’s and master’s degrees in Computer Science from the Technical University Munich. During her master’s work Dr. Deniz worked with Dr. Christof Koch at Caltech, where she studied visual saliency and automated text detection. As an advocate of reproducible research practices she is the co-editor of the book titled The Practice of Reproducible Research. In addition, she works on improving Internet security applications using knowledge gained from cognitive neuroscience and Mooney images ( Her work is at the intersection between computer science, human cognition, and neuroscience. She is a passionate coder, baker, and cello player.
The Distinguished Young Academic Data Scientists (DYADS) initiative promotes networking and speaking opportunities for outstanding postdoctoral scholars from our partner institutions. Through speaking engagements in our high profile seminar series and meetings with faculty colleagues, the scholars broaden their academic networks and their visibility on other campuses.