视听整合神经认知机制对听障者多模态感知研究的意义

周爱保;潘超超;

1:甘肃省行为与心理健康重点实验室西北师范大学心理学院

摘要
<正>社会情景中的多种刺激信息经过不同的感觉通道传递到大脑,大脑不断地接收形色各异的刺激信息,但并未将这个世界感知为支离破碎的各种形状、颜色、声音、气味等,而是整合来自多个感觉通道的线索将其知觉为一个统一、连续、稳定的整体,这个加工过程即为多感觉整合(multisensory integration)[1]。多感觉整合能弥补单通道信息的匮乏及不确定性,有助于人们更迅速而准确的进行认知判断。因为人类所接收的信息中有97%来自视觉和听觉,因此,对
关键词
基金项目(Foundation):
2019年度国家民委民族研究项目(2019-GMD-020)
作者
周爱保;潘超超;
参考文献

1 文小辉,李国强,刘强.视听整合加工及其神经机制[J].心理科学进展,2011,19:976.

2 欧阳丹,欧阳文娟.视听整合的研究进展[J].教育研究,2017,4:176.

3 石涯,王永华,李文靖.唇读对听障儿童语音识别的帮助作用[J].听力学及言语疾病杂志,2016,24:482.

4 罗霄骁,康冠兰,周晓林.McGurk效应的影响因素与神经基础[J].心理科学进展,2018,26:1935.

5 McGurk H,MacDonald J.Hearing lips and seeing voices[J].Nature,1976,263:47.

6 Amedi A,von Kriegstein K,van Atteveldt NM,et al.Functional imaging of human crossmodal identification and object recognition[J].Experimental Brain Research,2005,166:559.

7 Schall S,von Kriegstein K.Functional connectivity between face-movement and speech-intelligibility areas during auditory-only speech perception[J].PLoS One,2014,9:1.

8 Beauchamp MS,Lee KE,Haxby JV,et al.Parallel visual motion processing streams for manipulable objects and human movements[J].Neuron,2002,34:149.

9 Komeilipoor N,Cesari P,Daffertshofer A.Involvement of superior temporal areas in audiovisual and audiomotor speech integration[J].Neuroscience,2017,343:276.

10 Maguinness C,von Kriegstein K.Cross-modal processing of voices and faces in developmental prosopagnosia and developmental phonagnosia[J].Visual Cognition,2017,25:644.

11 Shinozaki J,Hiroe N,Sato M,et al.Impact of language on functional connectivity for audiovisual speech integration[J].Scientific Reports,2016,6:1.

12 Arnal LH,Morillon B,Kell CA,et al.Dual neural routing of visual facilitation in speech processing[J].Journal of Neuroscience,2009,29:13445.

13 王苹,潘治辉,张立洁,等.动态面孔和语音情绪信息的整合加工及神经生理机制[J].心理科学进展,2015,23:1109.

14 de Gelder B,Vroomen J.The perception of emotions by ear and by eye[J].Cognition and Emotion,2000,14:289.

15 Piwek L,Pollick F,Petrini K.Audiovisual integration of emotional signals from others’ social interactions[J].Frontiers in Psychology,2015,6:1.

16 Kreifelts B,Ethofer T,Grodd W,et al.Audiovisual integration of emotional signals in voice and face:an event-related fMRI study[J].NeuroImage,2007,37:1445.

17 Watson R,Latinus M,Noguchi T,et al.Crossmodal adaptation in right posterior superior temporal sulcus during face-voice emotional integration[J].Journal of Neuroscience,2014,34:6813.

18 Thompson JD,Elli G,Rezk M,et al.Hierarchical brain network for face and voice integration of emotion expression[J].Cerebral Cortex,2017,11:1.

19 何良玉,何湛愔.情绪性信息诱发的脑偏侧化认知神经机制研究进展[J].心理科学进展,2016,6:1231.

20 Pourtois G,de Gelder B,Bol A,et al.Perception of facial expressions and voices and of their combination in the human brain[J].Cortex,2005,41:49.

21 Park JY,Gu BM,Kang DH,et al.Integration of cross-modal emotional information in the human brain:an fMRI study[J].Cortex,2010,46:161.

22 Fujisaki W,Goda N,Motoyoshi I,et al.Audiovisual integration in the human perception of materials[J].Journal of Vision,2014,14:1.

23 Beauchamp MS,Lee KE,Argall BD,et al.Integration of auditory and visual information about objects in superior temporal sulcus[J].Neuron,2004,41:809.

24 Kourtzi Z,Kanwisher NG.Cortical regions involved in perceiving object shape[J].Journal of Neuroscience,2000,20:3310.

25 Martin A,Chao LL.Semantic memory and the brain:structure and processes[J].Current Opinion in Neurobiology,2001,11:194.

26 Belin P,Zatorre RJ,Lafaille P,et al.Voice-selective areas in human auditory cortex[J].Nature,2000,403:309.

27 Zatorre RJ,Bouffard M,Belin P.Sensitivity to auditory object features in human temporal neocortex[J].Journal of Neuroscience,2004,24:3637.

28 Molholm S,Ritter W,Javitt DC,et al.Multisensory visual-auditory object recognition in humans:a high-density electrical mapping study[J].Cerebral Cortex,2004,14:452.

29 Giovannelli F,Giganti F,Righi S,et al.Audio–visual integration effect in lateral occipital cortex during an object recognition task:an interference pilot study[J].Brain Stimulation,2016,9:574.

30 Cichy RM,Pantazis D,Oliva A.Similarity-based fusion of meg and fmri reveals spatio-temporal dynamics in human cortex during visual object recognition[J].Cerebral Cortex,2016,26:3563.

31 Peynircioglu ZF,Brent W,Tatz JR,et al.McGurk effect in gender identification:vision trumps audition in voice judgments[J].The Journal of General Psychology,2017,144:59.

32 Axelrod V,Yovel G.Successful decoding of famous faces in the fusiform face area[J].PLoS One,2015,10:1.

33 Bülthoff I,Newell FN.Crossmodal priming of unfamiliar faces supports early interactions between voices and faces in person perception[J].Visual Cognition,2017,25:611.

34 Weibert K,Andrews TG.Activity in the right fusiform face area predicts the behavioural advantage for the perception of familiar faces[J],Neuropsychologia,2015,75:588.

35 Maguinness C,Roswandowitz C,von Kriegstein K.Understanding the mechanisms of familiar voice-identity recognition in the human brain[J].Neuropsychologia,2018,116:179.

36 Maguinness C,von Kriegstein K.Cross-modal processing of voices and faces in developmental prosopagnosia and developmental phonagnosia[J].Visual Cognition,2017,25:644.

37 Joassin F,Maurage P,Campanella S.The neural network sustaining the crossmodal processing of human gender from faces and voices:An fMRI study[J].Neuroimage,2010,54:1654.

38 Joassin F,Maurage P,Bruyer R,et al.When audition alters vision:an event-related potential study of the cross-modal interactions between faces and voices[J].Neuroscience Letter,2004,369:132.

39 Blank H,Anwander A,von Kriegstein K.Direct structural connections between voice-and face-recognition areas[J].The Journal of Neuroscience,2011,31:12906.

40 Hasan BAS,Valdes-Sosa M,Gross J,et al.“Hearing faces and seeing voices”:a modal coding of person identity in the human brain[J].Scientific Reports,2016,6.

41 Schall S,Kiebel SJ,Maess B,et al.Early auditory sensory processing of voices is facilitated by visual mechanisms[J].NeuroImage,2013,77:237.

42 von Kriegstein K,Dogan O,Grüter M,et al.Simulation of talking faces in the human brain improves auditory speech recognition[J].Proceedings of the National Academy of Sciences,2008,105:6747.

43 Senkowski D,Schneider TR,Foxe JJ,et al.Crossmodal binding through neural coherence:implications for multisensory processing[J].Trends in Neurosciences,2008,31:401.

本文信息

PDF(152K)

本文关键词相关文章

本文作者相关文章

周爱保潘超超