Role of visual and linguistic complexity in language development
This project, funded by the EU Marie Curie action, uses tools of information theory to test the hypothesis that information throughput capacity (ability to extract complex information from a linguistically-meaningful visual signal) increases along the continuum ‘biological motion - gesture – sign language’ in both perception and production. The use of a visual language (sign language) in the analysis allows for a direct comparison between linguistic (sign language) and non-linguistic (gesture) means of communication with non-communicative human motion, while remaining within the same perceptual domain (vision). We develop convergent algorithms for analysis of information transfer based on biological signals across different data types (motion in gesture and sign language; neural activity data from fMRI and EEG).
Relevant publications:
Malaia, E., Borneman, J.D., Wilbur, R.B. (2017). Information transfer capacity of articulators in American Sign Language. Language and Speech. DOI: 10.1177/0023830917708461
Malaia, E., Borneman, J.D., Wilbur, R.B. (2016). Assessment of information content in visual signal: analysis of optical flow fractal complexity. Visual Cognition, 24(3), 246-251
Communication in children on Autism Spectrum
This work, which was funded by the Department of Energy Ralph E. Powe Award, and European Institutes for Advanced Study, investigates neural bases of visual communication in children and adolescents on Autism Spectrum. We use EEG data, and develop new analysis methods for it, including a combination of network and microstate analyses, as well as frequency domain analysis based on statistical physics models.
Relevant publications:
Malaia, E., Cockerham, D., & Rublein, K. (2018). Visual integration of fear and anger emotional cues by children on the autism spectrum and neurotypical peers: An EEG study. Neuropsychologia.
Cockerham, D., Malaia, E. (2017) Neuroscience-supported approaches to teaching students on the autism spectrum. Special Issue on Educational Neuroscience, Zeitschrift fur Psychologie 224 (4), 290-293.
Malaia, E., Bates, E., Seitzman, B., & Coppess, K. (2016). Altered brain network dynamics in youths with autism spectrum disorder. Experimental brain research, 234(12), 3425-3431.
Other publications by topic:
Neurolinguistics
Krebs, J., Malaia, E., Wilbur, R. B., & Roehm, D. (2018). Subject preference emerges as cross-modal strategy for linguistic processing. Brain research, 1691, 105-117.
Malaia, E., Newman, S. (2015) Neural bases of syntax-semantics interface processing. Cognitive Neurodynamics, 9(3), 317-329.
Malaia, E., Tommerdahl, J., Mckee, F.W. (2014). Deductive and heuristic reasoning processing markers in EEG. Journal of Psycholinguistic Research, 44 (5), 533-544.
Malaia, E., Newman, S. (2014) Neural bases of event knowledge and syntax integration in comprehension of complex sentences. Neurocase, 21 (6), 753-766.
Malaia, E., Gonzalez-Castillo, J., Weber-Fox, C., Talavage, T.M., Wilbur, R.B. (2014). Neural processing of verbal event structure: temporal and functional dissociation between telic and atelic verbs. In: Mandouilidou, C., de Ameida, R. (eds.) Cognitive Science Perspectives on Verb Representation and Processing, pp. 131 - 140. Springer: Lausanne.
Malaia, E., Talavage, T., Wilbur, R.B. (2014) Functional connectivity in task-negative network of the Deaf: effects of sign language experience. PeerJ, doi: 10.7717/peerj.446
Newman, S., Malaia, E., & Seo, R. (2014). Does degree of handedness in a group of right-handed individuals affect language comprehension? Brain and Cognition, 86, 98-103.
Newman, S., Malaia, E., Seo, R., Hu, C. (2013) The effect of individual differences in working memory capacity on sentence comprehension: an fMRI study. Brain Topography, 26(3), 458-67.
Malaia, E., Wilbur, R.B., Weber-Fox, C. (2012). Down the garden path in EEG: telicity effects on thematic role re-assignment in relative clauses with transitive verbs. Journal of Psycholinguistic Research 41(5), 323-345.
Malaia, E., Wilbur, R., Weber-Fox, C. (2009). ERP evidence for telicity effects on syntactic processing in garden-path sentences. Brain and Language 108(3), 145-158.
Event segmentation in language - sign and spoken
Malaia, E., Ranaweera, R., Wilbur, R.B., Talavage. T.M. (2013). Event segmentation in a visual language: Neural bases of processing American Sign Language predicates. Neuroimage 59(4), 4094-4101.
Malaia, E. (2014). It Still Isn't Over: Event Boundaries in Language and Perception. Language and Linguistics Compass, 8(3), 89-98.
Malaia, E., Wilbur, R.B. (2012). Motion capture signatures of telic and atelic events in ASL predicates. Language and Speech, 55(3), 407-421.
Malaia, E., Wilbur, R.B., Weber-Fox, C. (2013). Event end-point primes the Undergoer argument: a look at neurobiological bases of event structure. In Gehrke, B., Arsenijevic, B. (eds.) Subatomic semantics of event predicates, pp. 231-248. Springer: Studies in Linguistics and Philosophy.
Malaia, E., Wilbur, R.B., Milković, M. (2013). Kinematic parameters of signed verbs at morpho-phonology interface. Journal of Speech, Language, and Hearing Research, 56, 1-12.
Malaia, E., Wilbur, R.B. (2012). What Sign Languages show: neurobiological bases of visual phonology. Di Sciullo, A.M. (ed.) Towards a biolinguistic understanding of grammar: essays on interfaces, pp. 265-275. John Benjamins.
Malaia, E., Wilbur, R.B. (2012). Telicity expression in visual modality. In McNally, L. & Delmonte, V. (eds.) Telicity, change, and state: A cross-categorial view of event structure, pp. 122-136. Oxford: Oxford University Press.
Computational modeling of language and language processing
Wilbur, R.B., Malaia, E. (2018) A new technique for assessing narrative prosodic effects in sign languages. In A. Hübl & M. Steinbach (eds.), Linguistic Foundations of Narration in Spoken and Sign Languages, Amsterdam: John Benjamins.
Malaia, E., Borneman, J.D., Wilbur, R.B. (2016) Assessment of information content in visual signal: analysis of optical flow fractal complexity. Visual Cognition doi:10.1080/13506285.2016.1225142
Malaia, E. (2017) Methodologies for quantitative analysis of information transfer sign language and gesture data. Behavioral and Brain Sciences.
Barbu, A., Barrett, D., Chen, W., Siddarth, N., Xiong, C., Corso, J., Fellbaum, C., Hanson, C., Hanson, S., Helie, S., Malaia, E., Pearlmutter, B., Siskind, J., Talavage, T., Wilbur, R. (2014). Seeing is Worse than Believing: Reading People’s Minds Better than Computer-Vision Methods Recognize Actions. In D. Fleet et al. (Eds.) European Conference on Computer Vision 2014, Lecture Notes in Computer Science, pp. 612–627. Springer: Lausanne.
Mind, Brain, and Education
Malaia, E., Egorova, E., Hinesley, V. (in press) Developmental Characteristics of Gifted Children: Educational Approaches. In J. Horvath, J. Lodge, and J. Hattie (eds.) From the Laboratory to the Classroom: Translating the Science of Learning for Teachers. Routledge, UK.
Newman, S. D., Malaia, E. (2013). The neural bases of giftedness. In Plucker, J.A., Callahan, C. (eds.) Critical Issues and Practices in Gifted Education: What the Research Says, pp. 451-464. Prufrock Press.
Malaia, E., Wilbur, R.B. (2014). Enhancement of spatial processing in sign language users. In D. R. Montello, K. E. Grossner, and D. G. Janelle (eds.), Space in Mind: Concepts and Ontologies for Spatial Thinking, pp. 159-171, MIT press.
This project, funded by the EU Marie Curie action, uses tools of information theory to test the hypothesis that information throughput capacity (ability to extract complex information from a linguistically-meaningful visual signal) increases along the continuum ‘biological motion - gesture – sign language’ in both perception and production. The use of a visual language (sign language) in the analysis allows for a direct comparison between linguistic (sign language) and non-linguistic (gesture) means of communication with non-communicative human motion, while remaining within the same perceptual domain (vision). We develop convergent algorithms for analysis of information transfer based on biological signals across different data types (motion in gesture and sign language; neural activity data from fMRI and EEG).
Relevant publications:
Malaia, E., Borneman, J.D., Wilbur, R.B. (2017). Information transfer capacity of articulators in American Sign Language. Language and Speech. DOI: 10.1177/0023830917708461
Malaia, E., Borneman, J.D., Wilbur, R.B. (2016). Assessment of information content in visual signal: analysis of optical flow fractal complexity. Visual Cognition, 24(3), 246-251
Communication in children on Autism Spectrum
This work, which was funded by the Department of Energy Ralph E. Powe Award, and European Institutes for Advanced Study, investigates neural bases of visual communication in children and adolescents on Autism Spectrum. We use EEG data, and develop new analysis methods for it, including a combination of network and microstate analyses, as well as frequency domain analysis based on statistical physics models.
Relevant publications:
Malaia, E., Cockerham, D., & Rublein, K. (2018). Visual integration of fear and anger emotional cues by children on the autism spectrum and neurotypical peers: An EEG study. Neuropsychologia.
Cockerham, D., Malaia, E. (2017) Neuroscience-supported approaches to teaching students on the autism spectrum. Special Issue on Educational Neuroscience, Zeitschrift fur Psychologie 224 (4), 290-293.
Malaia, E., Bates, E., Seitzman, B., & Coppess, K. (2016). Altered brain network dynamics in youths with autism spectrum disorder. Experimental brain research, 234(12), 3425-3431.
Other publications by topic:
Neurolinguistics
Krebs, J., Malaia, E., Wilbur, R. B., & Roehm, D. (2018). Subject preference emerges as cross-modal strategy for linguistic processing. Brain research, 1691, 105-117.
Malaia, E., Newman, S. (2015) Neural bases of syntax-semantics interface processing. Cognitive Neurodynamics, 9(3), 317-329.
Malaia, E., Tommerdahl, J., Mckee, F.W. (2014). Deductive and heuristic reasoning processing markers in EEG. Journal of Psycholinguistic Research, 44 (5), 533-544.
Malaia, E., Newman, S. (2014) Neural bases of event knowledge and syntax integration in comprehension of complex sentences. Neurocase, 21 (6), 753-766.
Malaia, E., Gonzalez-Castillo, J., Weber-Fox, C., Talavage, T.M., Wilbur, R.B. (2014). Neural processing of verbal event structure: temporal and functional dissociation between telic and atelic verbs. In: Mandouilidou, C., de Ameida, R. (eds.) Cognitive Science Perspectives on Verb Representation and Processing, pp. 131 - 140. Springer: Lausanne.
Malaia, E., Talavage, T., Wilbur, R.B. (2014) Functional connectivity in task-negative network of the Deaf: effects of sign language experience. PeerJ, doi: 10.7717/peerj.446
Newman, S., Malaia, E., & Seo, R. (2014). Does degree of handedness in a group of right-handed individuals affect language comprehension? Brain and Cognition, 86, 98-103.
Newman, S., Malaia, E., Seo, R., Hu, C. (2013) The effect of individual differences in working memory capacity on sentence comprehension: an fMRI study. Brain Topography, 26(3), 458-67.
Malaia, E., Wilbur, R.B., Weber-Fox, C. (2012). Down the garden path in EEG: telicity effects on thematic role re-assignment in relative clauses with transitive verbs. Journal of Psycholinguistic Research 41(5), 323-345.
Malaia, E., Wilbur, R., Weber-Fox, C. (2009). ERP evidence for telicity effects on syntactic processing in garden-path sentences. Brain and Language 108(3), 145-158.
Event segmentation in language - sign and spoken
Malaia, E., Ranaweera, R., Wilbur, R.B., Talavage. T.M. (2013). Event segmentation in a visual language: Neural bases of processing American Sign Language predicates. Neuroimage 59(4), 4094-4101.
Malaia, E. (2014). It Still Isn't Over: Event Boundaries in Language and Perception. Language and Linguistics Compass, 8(3), 89-98.
Malaia, E., Wilbur, R.B. (2012). Motion capture signatures of telic and atelic events in ASL predicates. Language and Speech, 55(3), 407-421.
Malaia, E., Wilbur, R.B., Weber-Fox, C. (2013). Event end-point primes the Undergoer argument: a look at neurobiological bases of event structure. In Gehrke, B., Arsenijevic, B. (eds.) Subatomic semantics of event predicates, pp. 231-248. Springer: Studies in Linguistics and Philosophy.
Malaia, E., Wilbur, R.B., Milković, M. (2013). Kinematic parameters of signed verbs at morpho-phonology interface. Journal of Speech, Language, and Hearing Research, 56, 1-12.
Malaia, E., Wilbur, R.B. (2012). What Sign Languages show: neurobiological bases of visual phonology. Di Sciullo, A.M. (ed.) Towards a biolinguistic understanding of grammar: essays on interfaces, pp. 265-275. John Benjamins.
Malaia, E., Wilbur, R.B. (2012). Telicity expression in visual modality. In McNally, L. & Delmonte, V. (eds.) Telicity, change, and state: A cross-categorial view of event structure, pp. 122-136. Oxford: Oxford University Press.
Computational modeling of language and language processing
Wilbur, R.B., Malaia, E. (2018) A new technique for assessing narrative prosodic effects in sign languages. In A. Hübl & M. Steinbach (eds.), Linguistic Foundations of Narration in Spoken and Sign Languages, Amsterdam: John Benjamins.
Malaia, E., Borneman, J.D., Wilbur, R.B. (2016) Assessment of information content in visual signal: analysis of optical flow fractal complexity. Visual Cognition doi:10.1080/13506285.2016.1225142
Malaia, E. (2017) Methodologies for quantitative analysis of information transfer sign language and gesture data. Behavioral and Brain Sciences.
Barbu, A., Barrett, D., Chen, W., Siddarth, N., Xiong, C., Corso, J., Fellbaum, C., Hanson, C., Hanson, S., Helie, S., Malaia, E., Pearlmutter, B., Siskind, J., Talavage, T., Wilbur, R. (2014). Seeing is Worse than Believing: Reading People’s Minds Better than Computer-Vision Methods Recognize Actions. In D. Fleet et al. (Eds.) European Conference on Computer Vision 2014, Lecture Notes in Computer Science, pp. 612–627. Springer: Lausanne.
Mind, Brain, and Education
Malaia, E., Egorova, E., Hinesley, V. (in press) Developmental Characteristics of Gifted Children: Educational Approaches. In J. Horvath, J. Lodge, and J. Hattie (eds.) From the Laboratory to the Classroom: Translating the Science of Learning for Teachers. Routledge, UK.
Newman, S. D., Malaia, E. (2013). The neural bases of giftedness. In Plucker, J.A., Callahan, C. (eds.) Critical Issues and Practices in Gifted Education: What the Research Says, pp. 451-464. Prufrock Press.
Malaia, E., Wilbur, R.B. (2014). Enhancement of spatial processing in sign language users. In D. R. Montello, K. E. Grossner, and D. G. Janelle (eds.), Space in Mind: Concepts and Ontologies for Spatial Thinking, pp. 159-171, MIT press.