Information

How exactly does sensory substitution work?

How exactly does sensory substitution work?


We are searching data for your request:

Forums and discussions:
Manuals and reference books:
Data from registers:
Wait the end of the search in all databases.
Upon completion, a link will appear to access the found materials.

Sensory substitution, when one of sensory modality changes into another sensory modality to help someone restore the ability to perceive defective sensory using a working sensory modality. For example, blind people who have improvements in their other senses like auditory system or olfactory system compared to normal people. Or deaf person who have enhanced vision ability than normal people.

Questions :

  1. Does this work the same way as Alternative Sensation ?
  2. Is there any pattern of which senses will probably improve according to which kind of senses someone's lacking ?
  3. Is there actually any method to train which senses someone would like to improve ?

Tot start of with your definition:

Sensory substitution, when one of sensory modality changes into another sensory modality to help someone restore the ability to perceive defective sensory using a working sensory modality.

I personally like to use a more subtle definition of sensory substitution (Stronks et al., 2015):

[T]he process of obtaining information about the world through an alternative intact sensory channel [] that would normally be obtained by the deficient [sensory] system.

In other words, the information is not changed into another modality; it is simply processed by a sense normally not used for that purpose. Your definition seems to imply that the information stream is swapped. Although this is not strictly true, and hence the reason I like the second more subtle definition better, it is nonetheless true that cross-modal plasticity results in the deafferented cortical regions being taken over by other senses; for example in the blind, it has been shown that tactile and auditory information is processed in the visual regions, even down to the V1. The primary visual cortex is normally strictly associated with low-level visual processing, being the first central brain area where visual information enters the brain. Although this view is contested(V1 is also activated by tactile grids for example), in blind folks there are gross rearrangements where the somatosensory and auditory cortices take over the visual system (e.g., Pascual-Leone et al. (2005)).

To your questions

  • Does this work the same way as Alternative Sensation ?

'Alternative sensation' is unfamiliar to me, it doesn't pop up in a Google search, and its nonexistence is backed up by this answer.

  • Is there any pattern of which senses will probably improve according to which kind of senses someone's lacking

Yes. It depends on which senses are used most extensively to substitute for another sense. Training is an important factor in improving alternative sensory capabilities. In fact, it is the determining factor; the loss of a sensory system in itself does not propagate compensatory mechanisms (Stronks et al., 2015). Compensation being

partial compensation for [the] loss of [a sense] by developing enhanced abilities with [] remaining senses

For example, training with the BrainPort Artificial Vision Device (Fig. 1), which projects electrotactile 'images' onto the tongue using 400 electrodes, gradually improves performance in the blind, all the while the somatosensory information is funneled progressively into the visual system (Stronks et al., 2016).

  • Is there actually any method to train which senses someone would like to improve?

Pick your sensory substitution device of choice and train away; popular ones include, as said, the BrainPort (improves tactile-to-vision substitution), but also visual-to-auditory techniques like the vOICe (Striem-Amit et al., 2012). Also don't forget the guide cane, which funnels tactile information to the brain. especially the visual-to-auditory sensory substitution devices require a lot of training (Stronks et al., 2015). You can try the vOICe out yourself on an Android. However, I have tried the BrainPort myself and it is really difficult!

References
- Pascual-Leone et al. Annu Rev Neurosci (2005); 28: 377-401
- Striem-Amit, et al. PLoSONE (2012); 7(3): e33136
- Stronks et al., Brain Res (2015); 1624: 140-52
- Stronks et al., Exp Rev Med Dev (2016); 13(10): 919-31


Fig. 1. The BrainPort converts camera images into a 'grayscale' 400 pixel electrotactile image on the tongue. It partially restores visual function in the blind. Picture source: Midday Daily


The Stimulating Science of Animal Enrichment

If you have ever been to a zoo, rehabilitation center, or any other place where wild animals are kept, you have possibly seen animal enrichment taking place and not even realized it. For example, perhaps you have seen the animals interacting with interesting items, playing in pools of water on a hot day, or even being trained to do special tricks these are all examples of animal enrichment. Believe it or not, there is a careful and scientific process that goes on behind the scenes to deliver the right type of enrichment to each animal in captivity.


How is the brain organized?

Neurotransmitters are different from ions, because instead of directly affecting the charge of the neurons, neurotransmitters communicate by activating a receptor. In other words, the neurotransmitter is like a key and the receptor is the lock. Once the “key” turns the “lock,” or when the neurotransmitter attaches to the receptor, the message is passed on and the neurotransmitters are recycled. The transmission of information from neuron to neuron, and between networks of neurons, gives rise to everything from thinking to playing sports, solving problems, and even dreaming.

Neurons in the human brain and spinal cord are organized into the central and peripheral nervous systems. The central nervous system is organized into different functional areas:

1) The neocortex, which is organized into lobes seen in the illustration below.
2) The neostriatum or basal ganglia, which can be found deep within the structure.
3) The diencephalon, which contains the thalamus and hypothalamus, and is also found deep within the brain.
4) The brainstem.
5) The spinal cord.

Oftentimes, different lobes and areas work together to accomplish complicated behaviors like talking or learning. Not only are these neurons constantly communicating with each other, but they also interact with neurons in the peripheral nervous system.

The peripheral nervous system is comprised of sensory and motor neurons throughout the rest of your body. The sensory neurons collect information from the outside world through the five senses, while the motor neurons allow you to move and respond to signals from the brain and spinal cord.

When you were born, you had almost all the neurons you will ever have, and many more neuronal connections than you have today. The brain continues to change and grow throughout your lifetime because the connections between neurons are plastic. In other words, your brain can add new connections or subtract unused ones. As you grow up, your experiences and environment help your brain decide which connections are important and useful. In addition to your experiences, genetic information also influences your brain’s development. Although it is very complicated to tease apart what is inherited and what is learned, many behaviors appear to be a combination of both genetic and environmental factors


Sensory Substitution Devices (SSDs) are typically used to restore functionality of a sensory modality that has been lost, like vision for the blind, by recruiting another sensory modality such as touch or audition. Sensory substitution has given rise to many debates in psychology, neuroscience and philosophy regarding the nature of experience when using SSDs. Questions first arose as to whether the experience of sensory substitution is represented by the substituted information, the substituting information, or a multisensory combination of the two. More recently, parallels have been drawn between sensory substitution and synaesthesia, a rare condition in which individuals involuntarily experience a percept in one sensory or cognitive pathway when another one is stimulated. Here, we explore the efficacy of understanding sensory substitution as a form of ‘artificial synaesthesia’. We identify several problems with previous suggestions for a link between these two phenomena. Furthermore, we find that sensory substitution does not fulfil the essential criteria that characterise synaesthesia. We conclude that sensory substitution and synaesthesia are independent of each other and thus, the ‘artificial synaesthesia’ view of sensory substitution should be rejected.

Sections
References

Amedi , A. , Stern , W. M. , Camprodon , J. A. , Bermpohl , F. , Merabet , L. , Rotman , S. , Hemond , C. , Meijer , P. and Pascual-Leone , A. ( 2007 ). Shape conveyed by visual-to-auditory sensory substitution activates the lateral occipital complex , Nat. Neurosci. 10 , 687 – 689 . DOI: 10.1038/nn1912 .

Amedi , A. , Stern , W. M. , Camprodon , J. A. , Bermpohl , F. , Merabet , L. , Rotman , S. , Hemond , C. , Meijer , P. and Pascual-Leone , A.

Arno , P. , De Volder , A. G. , Vanlierde , A. , Wanet-Defalque , M.-C. , Streel , E. , Robert , A. , Sanabria-Bohórquez , S. and Veraart , C. ( 2001 ). Occipital activation by pattern recognition in the early blind using auditory substitution for vision , NeuroImage 13 , 632 – 645 . DOI: 10.1006/nimg.2000.0731 .

Arno , P. , De Volder , A. G. , Vanlierde , A. , Wanet-Defalque , M.-C. , Streel , E. , Robert , A. , Sanabria-Bohórquez , S. and Veraart , C.

Arnold , G. and Auvray , M. ( 2014 ). Perceptual learning: tactile letter recognition transfers across body surfaces , Multisens. Res. 27 , 71 – 90 . DOI: 10.1163/22134808-00002443 .

Arnold , G. and Auvray , M. ( 2018 ). Tactile recognition of visual stimuli: specificity versus generalization of perceptual learning , Vision Res. 152 , 40 – 50 . DOI: 10.1016/j.visres.2017.11.007 .

Arnold , G. , Pesnot-Lerousseau , J. and Auvray , M. ( 2017 ). Individual differences in sensory substitution , Multisens. Res. 30 , 579 – 600 . DOI: 10.1163/22134808-00002561 .

Arnold , G. , Pesnot-Lerousseau , J. and Auvray , M.

Asher , J. E. , Lamb , J. A. , Brocklebank , D. , Cazier , J.-B. , Maestrini , E. , Addis , L. , Sen , M. , Baron-Cohen , S. and Monaco , A. P. ( 2009 ). A whole-genome scan and fine-mapping linkage study of auditory-visual synesthesia reveals evidence of linkage to chromosomes 2q24, 5q33, 6p12, and 12p12 , Am. J. Hum. Genet. 84 , 279 – 285 . DOI: 10.1016/j.ajhg.2009.01.012 .

Asher , J. E. , Lamb , J. A. , Brocklebank , D. , Cazier , J.-B. , Maestrini , E. , Addis , L. , Sen , M. , Baron-Cohen , S. and Monaco , A. P.

Auvray , M. ( 2019 ). Multisensory and spatial processes in sensory substitution , Restor. Neurol. Neurosci. 37 , 606 – 619 . DOI: 10.3233/RNN-190950 .

Auvray , M. and Deroy , O. ( 2015 ). How do synesthetes experience the world? , in: Oxford Handbook of Philosophy of Perception , M. Matthen (Ed.), pp. 640 – 658 . Oxford University Press , Oxford, UK . DOI: 10.1093/oxfordhb/9780199600472.013.027 .

( 2015 ). How do synesthetes experience the world? , in: Oxford Handbook of Philosophy of Perception,

. DOI: 10.1093/oxfordhb/9780199600472.013.027 . )| false

Auvray , M. and Farina , M. ( 2017 ). Patrolling the boundaries of synaesthesia: a critical appraisal of transient and artificially induced forms of synaesthetic experiences , in: Sensory Blending: on Synaesthesia and Related Phenomena , O. Deroy (Ed.), pp. 248 – 274 . Oxford University Press , Oxford, UK .

( 2017 ). Patrolling the boundaries of synaesthesia: a critical appraisal of transient and artificially induced forms of synaesthetic experiences , in: Sensory Blending: on Synaesthesia and Related Phenomena,

Auvray , M. and Myin , E. ( 2009 ). Perception with compensatory devices: from sensory substitution to sensorimotor extension , Cogn. Sci. 33 , 1036 – 1058 . DOI: 10.1111/j.1551-6709.2009.01040.x .

Auvray , M. , Hanneton , S. and O’Regan , J. K. ( 2007 ). Learning to perceive with a visuo–auditory substitution system: localisation and object recognition with ‘The Voice’ , Perception 36 , 416 – 430 . DOI: 10.1068/p5631 .

Auvray , M. , Hanneton , S. and O’Regan , J. K.

Bach-y-Rita , P. and Kercel , S. W. ( 2003 ). Sensory substitution and the human–machine interface , Trends Cogn. Sci. 7 , 541 – 646 . DOI: 10.1016/j.tics.2003.10.013 .

Bach-y-Rita , P. and Kercel , S. W.

Bach-y-Rita , P. , Collins , C. C. , Saunders , F. A. , White , B. and Scadden , L. ( 1969 ). Vision substitution by tactile image projection , Nature 221 , 963 – 964 . DOI: 10.1038/221963a0 .

Bach-y-Rita , P. , Collins , C. C. , Saunders , F. A. , White , B. and Scadden , L.

Barnett , K. J. , Foxe , J. J. , Molholm , S. , Kelly , S. P. , Shalgi , S. , Mitchell , K. J. and Newell , F. N. ( 2008 ). Differences in early sensory-perceptual processing in synesthesia: a visual evoked potential study , NeuroImage 43 , 605 – 613 . DOI: 10.1016/j.neuroimage.2008.07.028 .

Barnett , K. J. , Foxe , J. J. , Molholm , S. , Kelly , S. P. , Shalgi , S. , Mitchell , K. J. and Newell , F. N.

Baron-Cohen , S. , Burt , L. , Smith-Laittan , F. , Harrison , J. and Bolton , P. ( 1996 ). Synaesthesia: prevalence and familiality , Perception 25 , 1073 – 1079 . DOI: 10.1068/p251073 .

Baron-Cohen , S. , Burt , L. , Smith-Laittan , F. , Harrison , J. and Bolton , P.

Bermejo , F. , Di Paolo , E. A. , Hüg , M. X. and Arias , C. ( 2015 ). Sensorimotor strategies for recognizing geometrical shapes: a comparative study with different sensory substitution devices , Front. Psychol. 6 , 679 . DOI: 10.3389/fpsyg.2015.00679 .

Bermejo , F. , Di Paolo , E. A. , Hüg , M. X. and Arias , C.

Block , N. ( 2003 ). Tactile sensation via spatial perception , Trends Cogn. Sci. 7 , 285 – 286 . DOI: 10.1016/S1364-6613(03)00132-3 .

Bologna , G. , Deville , B. and Pun , T. ( 2009 ). Blind navigation along a sinuous path by means of the See ColOr interface , in: International Work-Conference on the Interplay Between Natural and Artificial Computation , J. Mira , J. M. Ferrández , J. R. Álvarez , F. de la Paz and F. J. Toledo (Eds), pp. 235 – 243 . Springer , Berlin, Germany . DOI: 10.1007/978-3-642-02267-8_26 .

Bologna , G. , Deville , B. and Pun , T.

( 2009 ). Blind navigation along a sinuous path by means of the See ColOr interface , in: International Work-Conference on the Interplay Between Natural and Artificial Computation,

J. Mira , J. M. Ferrández , J. R. Álvarez , F. de la Paz and F. J. Toledo

. DOI: 10.1007/978-3-642-02267-8_26 . )| false

Bor , D. , Rothen , N. , Schwartzman , D. J. , Clayton , S. and Seth , A. K. ( 2015 ). Adults can be trained to acquire synesthetic experiences , Sci. Rep. 4 , 7089 . DOI: 10.1038/srep07089 .

Bor , D. , Rothen , N. , Schwartzman , D. J. , Clayton , S. and Seth , A. K.

Brang , D. and Ramachandran , V. S. ( 2011 ). Survival of the synesthesia gene: why do people hear colors and taste words? , PLoS Biol. 9 , e1001205 . DOI: 10.1371/journal.pbio.1001205 .

Brang , D. and Ramachandran , V. S.

Brown , D. J. , Macpherson , T. and Ward , J. ( 2011 ). Seeing with sound? Exploring different characteristics of a visual-to-auditory sensory substitution device , Perception 40 , 1120 – 1135 . DOI: 10.1068/p6952 .

Brown , D. J. , Macpherson , T. and Ward , J.

Carton , A. and Dunne , L. E. ( 2013 ). Tactile distance feedback for firefighters: design and preliminary evaluation of a sensory augmentation glove , in: Proceedings of the 4th Augmented Human International Conference , pp. 58 – 64 . DOI: 10.1145/2459236.2459247 .

( 2013 ). Tactile distance feedback for firefighters: design and preliminary evaluation of a sensory augmentation glove , in: Proceedings of the 4th Augmented Human International Conference, pp. 58 – 64 . DOI: 10.1145/2459236.2459247 . )| false

Cecchetti , L. , Kupers , R. , Ptito , M. , Pietrini , P. and Ricciardi , E. ( 2016 ). Are supramodality and cross-modal plasticity the yin and yang of brain development? From blindness to rehabilitation , Front. Syst. Neurosci. 10 , 89 . DOI: 10.3389/fnsys.2016.00089 .

Cecchetti , L. , Kupers , R. , Ptito , M. , Pietrini , P. and Ricciardi , E.

Chan , K. C. , Murphy , M. C. , Bang , J. W. , Sims , J. , Kashkoush , J. and Nau , A. C. ( 2018 ). Functional MRI of sensory substitution in the blind , in: 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) , pp. 5519 – 5522 . Honolulu, HI, USA . DOI: 10.1109/EMBC.2018.8513622 .

Chan , K. C. , Murphy , M. C. , Bang , J. W. , Sims , J. , Kashkoush , J. and Nau , A. C.

( 2018 ). Functional MRI of sensory substitution in the blind , in: 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 5519 – 5522 .

. DOI: 10.1109/EMBC.2018.8513622 . )| false

Chebat , D.-R. , Schneider , F. C. , Kupers , R. and Ptito , M. ( 2011 ). Navigation with a sensory substitution device in congenitally blind individuals , Neuroreport 22 , 342 – 347 . DOI: 10.1097/WNR.0b013e3283462def .

Chebat , D.-R. , Schneider , F. C. , Kupers , R. and Ptito , M.

Chebat , D.-R. , Maidenbaum , S. and Amedi , A. ( 2015 ). Navigation using sensory substitution in real and virtual mazes , PloS ONE 10 , e0126307 . DOI: 10.1371/journal.pone.0126307 .

Chebat , D.-R. , Maidenbaum , S. and Amedi , A.

Clark , A. ( 2003 ). Natural-Born Cyborgs: Minds, Technologies, and the Future of Human Intelligence . Oxford University Press , Oxford, UK .

( 2003 ). Natural-Born Cyborgs: Minds, Technologies, and the Future of Human Intelligence.

Cohen Kadosh , R. , Henik , A. , Catena , A. , Walsh , V. and Fuentes , L. J. ( 2009 ). Induced cross-modal synaesthetic experience without abnormal neuronal connections , Psychol. Sci. 20 , 258 – 265 . DOI: 10.1111/j.1467-9280.2009.02286.x .

Cohen Kadosh , R. , Henik , A. , Catena , A. , Walsh , V. and Fuentes , L. J.

Cytowic , R. E. ( 1989 ). Synesthesia and mapping of subjective sensory dimensions , Neurology 39 , 849 – 850 . DOI: 10.1212/WNL.39.6.849 .

De Volder , A. G. , Catalan-Ahumada , M. , Robert , A. , Bol , A. , Labar , D. , Coppens , A. , Michel , C. and Veraart , C. ( 1999 ). Changes in occipital cortex activity in early blind humans using a sensory substitution device , Brain Res. 826 , 128 – 134 . DOI: 10.1016/S0006-8993(99)01275-5 .

De Volder , A. G. , Catalan-Ahumada , M. , Robert , A. , Bol , A. , Labar , D. , Coppens , A. , Michel , C. and Veraart , C.

Deen , B. , Saxe , R. and Bedny , M. ( 2015 ). Occipital cortex of blind individuals is functionally coupled with executive control areas of frontal cortex , J. Cogn. Neurosci. 27 , 1633 – 1647 . DOI: 10.1162/jocn_a_00807 .

Deen , B. , Saxe , R. and Bedny , M.

Deroy , O. and Auvray , M. ( 2012 ). Reading the world through the skin and ears: a new perspective on sensory substitution , Front. Psychol. 3 , 457 . DOI: 10.3389/fpsyg.2012.00457 .

Deroy , O. and Auvray , M. ( 2014 ). A crossmodal perspective on sensory substitution , in: Perception and Its Modalities , D. Stokes , M. Matthen and S. Biggs (Eds), pp. 327 – 349 . Oxford University Press , Oxford, UK .

( 2014 ). A crossmodal perspective on sensory substitution , in: Perception and Its Modalities,

D. Stokes , M. Matthen and S. Biggs

Deroy , O. and Spence , C. ( 2013 ). Why we are not all synesthetes (not even weakly so) , Psychon. Bull. Rev. 20 , 643 – 664 . DOI: 10.3758/s13423-013-0387-2 .

Deroy , O. , Fasiello , I. , Hayward , V. and Auvray , M. ( 2016 ). Differentiated audio-tactile correspondences in sighted and blind individuals , J. Exp. Psychol. Hum. Percept. Perform. 42 , 1204 – 1214 . DOI: 10.1037/xhp0000152 .

Deroy , O. , Fasiello , I. , Hayward , V. and Auvray , M.

Dixon , M. J. , Smilek , D. , Cudahy , C. and Merikle , P. M. ( 2000 ). Five plus two equals yellow , Nature 406 , 365 . DOI: 10.1038/35019148 .

Dixon , M. J. , Smilek , D. , Cudahy , C. and Merikle , P. M.

Dormal , G. and Collignon , O. ( 2011 ). Functional selectivity in sensory-deprived cortices , J. Neurophysiol. 105 , 2627 – 2630 . DOI: 10.1152/jn.00109.2011 .

Dormal , G. and Collignon , O.

Eagleman , D. M. , Kagan , A. D. , Nelson , S. S. , Sagaram , D. and Sarma , A. K. ( 2007 ). A standardized test battery for the study of synesthesia , J. Neurosci. Methods 159 , 139 – 145 . DOI: 10.1016/j.jneumeth.2006.07.012 .

Eagleman , D. M. , Kagan , A. D. , Nelson , S. S. , Sagaram , D. and Sarma , A. K.

Farina , M. ( 2013 ). Neither touch nor vision: sensory substitution as artificial synaesthesia? , Biol. Philos. 28 , 639 – 655 . DOI: 10.1007/s10539-013-9377-z .

Grossenbacher , P. G. and Lovelace , C. T. ( 2001 ). Mechanisms of synesthesia: cognitive and physiological constraints , Trends Cogn. Sci. 5 , 36 – 41 . DOI: 10.1016/S1364-6613(00)01571-0 .

Grossenbacher , P. G. and Lovelace , C. T.

Haigh , A. , Brown , D. J. , Meijer , P. and Proulx , M. J. ( 2013 ). How well do you see what you hear? The acuity of visual-to-auditory sensory substitution , Front. Psychol. 4 , 330 . DOI: 10.3389/fpsyg.2013.00330 .

Haigh , A. , Brown , D. J. , Meijer , P. and Proulx , M. J.

Hamilton-Fletcher , G. , Mengucci , M. and Medeiros , F. ( 2016 ). Synaestheatre: sonification of coloured objects in space , in: Proceedings of the 2016 International Conference on Live Interfaces , pp. 252 – 256 . Brighton, UK .

Hamilton-Fletcher , G. , Mengucci , M. and Medeiros , F.

( 2016 ). Synaestheatre: sonification of coloured objects in space , in: Proceedings of the 2016 International Conference on Live Interfaces, pp. 252 – 256 .

Hanneton , S. , Auvray , M. and Durette , B. ( 2010 ). The Vibe: a versatile vision-to-audition sensory substitution device , Appl. Bionics Biomech. 7 , 269 – 276 . DOI: 10.1080/11762322.2010.512734 .

Hanneton , S. , Auvray , M. and Durette , B.

Hanneton , S. , Hoellinger , T. , Forma , V. , Roby-Brami , A. and Auvray , M. ( 2020 ). Ears on the hand: reaching three-dimensional targets with an audio-motor device , Multisens. Res. 33 , 433 – 455 . DOI: 10.1163/22134808-20191436 .

Hanneton , S. , Hoellinger , T. , Forma , V. , Roby-Brami , A. and Auvray , M.

Heimler , B. and Amedi , A. ( 2020 ). Task-selectivity in the sensory deprived brain and sensory substitution approaches for clinical practice: evidence from blindness , in: Multisensory Perception , K. Sathian and V. S. Ramachandran (Eds), pp. 321 – 342 . Academic Press , London, UK . DOI: 10.1016/B978-0-12-812492-5.00015-2 .

( 2020 ). Task-selectivity in the sensory deprived brain and sensory substitution approaches for clinical practice: evidence from blindness , in: Multisensory Perception,

K. Sathian and V. S. Ramachandran

. DOI: 10.1016/B978-0-12-812492-5.00015-2 . )| false

Heimler , B. , Weisz , N. and Collignon , O. ( 2014 ). Revisiting the adaptive and maladaptive effects of crossmodal plasticity , Neuroscience 283 , 44 – 63 . DOI: 10.1016/j.neuroscience.2014.08.003 .

Heimler , B. , Weisz , N. and Collignon , O.

Heimler , B. , Striem-Amit , E. and Amedi , A. ( 2015 ). Origins of task-specific sensory-independent organization in the visual and auditory brain: neuroscience evidence, open questions and clinical implications , Curr. Opin. Neurobiol. 35 , 169 – 177 . DOI: 10.1016/j.conb.2015.09.001 .

Heimler , B. , Striem-Amit , E. and Amedi , A.

Humphrey , N. ( 2006 ). Seeing Red: a Study in Consciousness . Harvard University Press , Cambridge, MA, USA .

( 2006 ). Seeing Red: a Study in Consciousness.

Hurley , S. and Noë , A. ( 2003 ). Neural plasticity and consciousness , Biol. Philos. 18 , 131 – 168 . DOI: 10.1023/A:1023308401356 .

Jacobson , R. ( 2014 ). App helps the blind “see” with their ears, Nat. Geogr. Available at https://www.nationalgeographic.com/news/2014/4/140403-eyemusic-ssd-visual-impairment-software-science/ .

Jäncke , L. , Beeli , G. , Eulig , C. and Hänggi , J. ( 2009 ). The neuroanatomy of grapheme–color synesthesia , Eur. J. Neurosci. 29 , 1287 – 1293 . DOI: 10.1111/j.1460-9568.2009.06673.x .

Jäncke , L. , Beeli , G. , Eulig , C. and Hänggi , J.

Kaczmarek , K. A. ( 2011 ). The tongue display unit (TDU) for electrotactile spatiotemporal pattern presentation , Sci. Iran. 18 , 1476 – 1485 . DOI: 10.1016/j.scient.2011.08.020 .

Keeley , B. L. ( 2002 ). Making sense of the senses: individuating modalities in humans and other animals , J. Philos. 99 , 5 – 28 . DOI: 10.5840/jphil20029915 .

Kim , J.-K. and Zatorre , R. J. ( 2008 ). Generalized learning of visual-to-auditory substitution in sighted individuals , Brain Res. 1242 , 263 – 275 . DOI: 10.1016/j.brainres.2008.06.038 .

Kiverstein , J. , Farina , M. and Clark , A. ( 2014 ). Substituting the senses , in: The Oxford Handbook of the Philosophy of Perception , M. Matthen (Ed.), pp. 659 – 675 . Oxford University Press , Oxford, UK .

Kiverstein , J. , Farina , M. and Clark , A.

( 2014 ). Substituting the senses , in: The Oxford Handbook of the Philosophy of Perception,

Kujala , T. , Alho , K. , Paavilainen , P. , Summala , H. and Näätänen , R. ( 1992 ). Neural plasticity in processing of sound location by the early blind: an event-related potential study , Electroencephalogr. Clin. Neurophysiol. 84 , 469 – 472 . DOI: 10.1016/0168-5597(92)90034-9 .

Kujala , T. , Alho , K. , Paavilainen , P. , Summala , H. and Näätänen , R.

Kupers , R. and Ptito , M. ( 2011 ). Insights from darkness: what the study of blindness has taught us about brain structure and function , in: Progress in Brain Research, Vol. 192 , A. Green , C. E. Chapman , J. F. Kalaska and F. Lepore (Eds), pp. 17 – 31 . Elsevier , Amsterdam, The Netherlands . DOI: 10.1016/B978-0-444-53355-5.00002-6 .

( 2011 ). Insights from darkness: what the study of blindness has taught us about brain structure and function , in: Progress in Brain Research, Vol. 192,

A. Green , C. E. Chapman , J. F. Kalaska and F. Lepore

Amsterdam, The Netherlands

. DOI: 10.1016/B978-0-444-53355-5.00002-6 . )| false

Kupers , R. , Fumal , A. , Maertens de Noordhout , A. , Gjedde , A. , Schoenen , J. and Ptito , M. ( 2006 ). Transcranial magnetic stimulation of the visual cortex induces somatotopically organized qualia in blind subjects , Proc. Natl Acad. Sci. U.S.A. 103 , 13256 – 13260 . DOI: 10.1073/pnas.0602925103 .

Kupers , R. , Fumal , A. , Maertens de Noordhout , A. , Gjedde , A. , Schoenen , J. and Ptito , M.

Kupers , R. , Chebat , D. R. , Madsen , K. H. , Paulson , O. B. and Ptito , M. ( 2010 ). Neural correlates of virtual route recognition in congenital blindness , Proc. Natl Acad. Sci. U.S.A. 107 , 12716 – 12721 .

Kupers , R. , Chebat , D. R. , Madsen , K. H. , Paulson , O. B. and Ptito , M.

Levy-Tzedek , S. , Hanassy , S. , Abboud , S. , Maidenbaum , S. and Amedi , A. ( 2012 ). Fast, accurate reaching movements with a visual-to-auditory sensory substitution device , Restor. Neurol. Neurosci. 30 , 313 – 323 . DOI: 10.3233/RNN-2012-110219 .

Levy-Tzedek , S. , Hanassy , S. , Abboud , S. , Maidenbaum , S. and Amedi , A.

Levy-Tzedek , S. , Riemer , D. and Amedi , A. ( 2014 ). Color improves “visual” acuity via sound , Front. Neurosci. 8 , 358 . DOI: 10.3389/fnins.2014.00358 .

Levy-Tzedek , S. , Riemer , D. and Amedi , A.

Loomis , J. M. , Klatzky , R. L. and Giudice , N. A. ( 2013 ). Sensory substitution of vision: importance of perceptual and cognitive processing , in: Assistive Technology for Blindness and Low Vision , R. Manduchi and S. Kurniawan (Eds), pp. 161 – 192 . CRC Press , Boca Raton, FL, USA . DOI: 10.1201/9781315216935 .

Loomis , J. M. , Klatzky , R. L. and Giudice , N. A.

( 2013 ). Sensory substitution of vision: importance of perceptual and cognitive processing , in: Assistive Technology for Blindness and Low Vision,

R. Manduchi and S. Kurniawan

. DOI: 10.1201/9781315216935 . )| false

Luke , D. P. and Terhune , D. B. ( 2013 ). The induction of synaesthesia with chemical agents: a systematic review , Front. Psychol. 4 , 753 . DOI: 10.3389/fpsyg.2013.00753 .

Luke , D. P. and Terhune , D. B.

Maidenbaum , S. , Abboud , S. and Amedi , A. ( 2014 ). Sensory substitution: closing the gap between basic research and widespread practical visual rehabilitation , Neurosci. Biobehav. Rev. 41 , 3 – 15 . DOI: 10.1016/j.neubiorev.2013.11.007 .

Maidenbaum , S. , Abboud , S. and Amedi , A.

Martin , J.-R. and Le Corre , F. ( 2015 ). Sensory substitution is substitution , Mind Lang. 30 , 209 – 233 . DOI: 10.1111/mila.12078 .

Martin , J.-R. and Le Corre , F.

Mattingley , J. B. ( 2009 ). Attention, automaticity, and awareness in synesthesia , Ann. N.Y. Acad. Sci. 1156 , 141 – 167 . DOI: 10.1111/j.1749-6632.2009.04422.x .

Meijer , P. B. L. ( 1992 ). An experimental system for auditory image representations , IEEE Trans. Biomed. Eng. 39 , 112 – 121 . DOI: 10.1109/10.121642 .

Merabet , L. B. , Battelli , L. , Obretenova , S. , Maguire , S. , Meijer , P. and Pascual-Leone , A. ( 2009 ). Functional recruitment of visual cortex for sound encoded object identification in the blind , Neuroreport 20 , 132 – 138 . DOI: 10.1097/WNR.0b013e32832104dc .

Merabet , L. B. , Battelli , L. , Obretenova , S. , Maguire , S. , Meijer , P. and Pascual-Leone , A.

Mills , C. B. ( 1999 ). Digit synaesthesia: a case study using a Stroop-type test , Cogn. Neuropsychol. 16 , 181 – 191 . DOI: 10.1080/026432999380951 .

Murphy , M. C. , Nau , A. C. , Fisher , C. , Kim , S.-G. , Schuman , J. S. and Chan , K. C. ( 2016 ). Top-down influence on the visual cortex of the blind during sensory substitution , NeuroImage 125 , 932 – 940 . DOI: 10.1016/j.neuroimage.2015.11.021 .

Murphy , M. C. , Nau , A. C. , Fisher , C. , Kim , S.-G. , Schuman , J. S. and Chan , K. C.

Noë , A. ( 2004 ). Action in Perception . MIT Press , Boston, MA, USA .

( 2004 ). Action in Perception.

O’Regan , J. K. ( 2011 ). Why Red Doesn’t Sound Like a Bell: Understanding the Feel of Consciousness . Oxford University Press , Oxford, UK .

( 2011 ). Why Red Doesn’t Sound Like a Bell: Understanding the Feel of Consciousness.

O’Regan , J. K. and Noë , A. ( 2001 ). A sensorimotor account of vision and visual consciousness , Behav. Brain Sci. 24 , 939 – 973 . DOI: 10.1017/S0140525X01000115 .

Ortiz , T. , Poch , J. , Santos , J. M. , Requena , C. , Martinez , A. M. , Ortiz-Teran , L. , Turrero , A. , Barcia , J. , Nogales , R. , Calvo , A. , Martinez , J. M. , Cordoba , J. L. and Pascual-Leone , A. ( 2011 ). Recruitment of occipital cortex during sensory substitution training linked to subjective experience of seeing in people with blindness , PLoS ONE 6 , e23264 . DOI: 10.1371/journal.pone.0023264 .

Ortiz , T. , Poch , J. , Santos , J. M. , Requena , C. , Martinez , A. M. , Ortiz-Teran , L. , Turrero , A. , Barcia , J. , Nogales , R. , Calvo , A. , Martinez , J. M. , Cordoba , J. L. and Pascual-Leone , A.

Pacherie , E. ( 1997 ). Du problème de Molyneux au problème de Bach-y-Rita , in: Perception et Intermodalité, Approches Actuelles du Problème de Molyneux , J. Proust (Ed.), pp. 255 – 293 . Presses Universitaires de France , Paris, France .

( 1997 ). Du problème de Molyneux au problème de Bach-y-Rita , in: Perception et Intermodalité, Approches Actuelles du Problème de Molyneux,

Presses Universitaires de France

Pascual-Leone , A. and Hamilton , R. ( 2001 ). The metamodal organization of the brain , in: Vision: from Neurons to Cognition , Progress in Brain Research, Vol. 134 , C. Casanova and M. Ptito (Eds), pp. 427 – 446 . Elsevier , Amsterdam, The Netherlands . DOI: 10.1016/S0079-6123(01)34028-1 .

Pascual-Leone , A. and Hamilton , R.

( 2001 ). The metamodal organization of the brain , in: Vision: from Neurons to Cognition, Progress in Brain Research, Vol. 134 ,

Amsterdam, The Netherlands

. DOI: 10.1016/S0079-6123(01)34028-1 . )| false

Peltier , S. , Stilla , R. , Mariola , E. , LaConte , S. , Hu , X. and Sathian , K. ( 2007 ). Activity and effective connectivity of parietal and occipital cortical regions during haptic shape perception , Neuropsychologia 45 , 476 – 483 . DOI: 10.1016/j.neuropsychologia.2006.03.003 .

Peltier , S. , Stilla , R. , Mariola , E. , LaConte , S. , Hu , X. and Sathian , K.

Pietrini , P. , Furey , M. L. , Ricciardi , E. , Gobbini , M. I. , Wu , W.-H. C. , Cohen , L. , Guazzelli , M. and Haxby , J. V. ( 2004 ). Beyond sensory images: object-based representation in the human ventral pathway , Proc. Natl Acad. Sci. U.S.A. 101 , 5658 – 5663 . DOI: 10.1073/pnas.0400707101 .

Pietrini , P. , Furey , M. L. , Ricciardi , E. , Gobbini , M. I. , Wu , W.-H. C. , Cohen , L. , Guazzelli , M. and Haxby , J. V.

Poirier , C. , De Volder , A. G. and Scheiber , C. ( 2007 ). What neuroimaging tells us about sensory substitution , Neurosci. Biobehav. Rev. 31 , 1064 – 1070 . DOI: 10.1016/j.neubiorev.2007.05.010 .

Poirier , C. , De Volder , A. G. and Scheiber , C.

Prinz , J. ( 2006 ). Putting the brakes on enactive perception , Psyche 12 , 1 – 19 .

Proulx , M. and Stoerig , P. ( 2006 ). Seeing sounds and tingling tongues: qualia in synaesthesia and sensory substitution , Anthropol. Philos. 7 , 135 – 150 .

Proulx , M. J. ( 2010 ). Synthetic synaesthesia and sensory substitution , Consc. Cogn. 19 , 501 – 503 . DOI: 10.1016/j.concog.2009.12.005 .

Proulx , M. J. , Stoerig , P. , Ludowig , E. and Knoll , I. ( 2008 ). Seeing ‘where’ through the ears: effects of learning-by-doing and long-term sensory deprivation on localization based on image-to-sound substitution , PloS ONE 3 , e1840 . DOI: 10.1371/journal.pone.0001840 .

Proulx , M. J. , Stoerig , P. , Ludowig , E. and Knoll , I.

Proulx , M. J. , Brown , D. J. , Pasqualotto , A. and Meijer , P. ( 2014 ). Multisensory perceptual learning and sensory substitution , Neurosci. Biobehav. Rev. 41 , 16 – 25 . DOI: 10.1016/j.neubiorev.2012.11.017 .

Proulx , M. J. , Brown , D. J. , Pasqualotto , A. and Meijer , P.

Proulx , M. J. , Gwinnutt , J. , Dell’Erba , S. , Levy-Tzedek , S. , de Sousa , A. A. and Brown , D. J. ( 2016 ). Other ways of seeing: from behavior to neural mechanisms in the online “visual” control of action with sensory substitution , Restor. Neurol. Neurosci. 34 , 29 – 44 . DOI: 10.3233/rnn-150541 .

Proulx , M. J. , Gwinnutt , J. , Dell’Erba , S. , Levy-Tzedek , S. , de Sousa , A. A. and Brown , D. J.

Ptito , M. , Moesgaard , S. M. , Gjedde , A. and Kupers , R. ( 2005 ). Cross-modal plasticity revealed by electrotactile stimulation of the tongue in the congenitally blind , Brain 128 , 606 – 614 . DOI: 10.1093/brain/awh380 .

Ptito , M. , Moesgaard , S. M. , Gjedde , A. and Kupers , R.

Ptito , M. , Fumal , A. , Martens De Noordhout , A. , Schoenen , J. , Gjedde , A. and Kupers , R. ( 2008 ). TMS of the occipital cortex induces tactile sensations in the fingers of blind Braille readers , Exp. Brain Res. 184 , 193 – 200 . DOI: 10.1007/s00221-007-1091-0 .

Ptito , M. , Fumal , A. , Martens De Noordhout , A. , Schoenen , J. , Gjedde , A. and Kupers , R.

Ptito , M. , Iversen , K. , Auvray , M. , Deroy , O. and Kupers , R. ( 2018 ). Limits of the classical functionalist perspective on sensory substitution , in: Sensory Substitution and Augmentation , F. Macpherson (Ed.), pp. 130 – 149 . Oxford University Press , Oxford, UK .

Ptito , M. , Iversen , K. , Auvray , M. , Deroy , O. and Kupers , R.

( 2018 ). Limits of the classical functionalist perspective on sensory substitution , in: Sensory Substitution and Augmentation,

Reich , L. , Szwed , M. , Cohen , L. and Amedi , A. ( 2011 ). A ventral visual stream reading center independent of visual experience , Curr. Biol. 21 , 363 – 368 . DOI: 10.1016/j.cub.2011.01.040 .

Reich , L. , Szwed , M. , Cohen , L. and Amedi , A.

Renier , L. and De Volder , A. ( 2013 ). Sensory substitution devices: creating “artificial synesthesias” , in: The Oxford Handbook of Synaesthesia , J. Simner and E. Hubbard (Eds), pp. 853 – 868 . Oxford University Press , Oxford, UK .

Renier , L. and De Volder , A.

( 2013 ). Sensory substitution devices: creating “artificial synesthesias” , in: The Oxford Handbook of Synaesthesia,

Renier , L. , Laloyaux , C. , Collignon , O. , Tranduy , D. , Vanlierde , A. , Bruyer , R. and De Volder , A. G. ( 2005 ). The Ponzo illusion with auditory substitution of vision in sighted and early-blind subjects , Perception 34 , 857 – 867 . DOI: 10.1068/p5219 .

Renier , L. , Laloyaux , C. , Collignon , O. , Tranduy , D. , Vanlierde , A. , Bruyer , R. and De Volder , A. G.

Renier , L. , Bruyer , R. and De Volder , A. G. ( 2006 ). Vertical-horizontal illusion present for sighted but not early blind humans using auditory substitution of vision , Percept. Psychophys. 68 , 535 – 542 . DOI: 10.3758/BF03208756 .

Renier , L. , Bruyer , R. and De Volder , A. G.

Ricciardi , E. and Pietrini , P. ( 2011 ). New light from the dark: what blindness can teach us about brain function , Curr. Opin. Neurol. 24 , 357 – 363 . DOI: 10.1097/WCO.0b013e328348bdbf .

Ricciardi , E. and Pietrini , P.

Rich , A. N. and Mattingley , J. B. ( 2002 ). Anomalous perception in synaesthesia: a cognitive neuroscience perspective , Nat. Rev. Neurosci. 3 , 43 – 52 . DOI: 10.1038/nrn702 .

Rich , A. N. and Mattingley , J. B.

Sadato , N. , Pascual-Leone , A. , Grafman , J. , Ibañez , V. , Deiber , M.-P. , Dold , G. and Hallett , M. ( 1996 ). Activation of the primary visual cortex by Braille reading in blind subjects , Nature 380 , 526 – 528 . DOI: 10.1038/380526a0 .

Sadato , N. , Pascual-Leone , A. , Grafman , J. , Ibañez , V. , Deiber , M.-P. , Dold , G. and Hallett , M.

Safran , A. B. and Sanda , N. ( 2015 ). Color synesthesia. Insight into perception, emotion, and consciousness , Curr. Opin. Neurol. 28 , 36 – 44 . DOI: 10.1097/WCO.0000000000000169 .

Sagiv , N. , Heer , J. and Robertson , L. ( 2006 ). Does binding of synesthetic color to the evoking grapheme require attention? , Cortex 42 , 232 – 242 . DOI: 10.1016/S0010-9452(08)70348-4 .

Sagiv , N. , Heer , J. and Robertson , L.

Siegle , J. H. and Warren , W. H. ( 2010 ). Distal attribution and distance perception in sensory substitution , Perception 39 , 208 – 223 . DOI: 10.1068/p6366 .

Siegle , J. H. and Warren , W. H.

Simner , J. , Harrold , J. , Creed , H. , Monro , L. and Foulkes , L. ( 2009 ). Early detection of markers for synaesthesia in childhood populations , Brain 132 , 57 – 64 . DOI: 10.1093/brain/awn292 .

Simner , J. , Harrold , J. , Creed , H. , Monro , L. and Foulkes , L.

Stevenson , R. J. and Boakes , R. A. ( 2004 ). Sweet and sour smells: learned synesthesia between the senses of taste and smell , in: The Handbook of Multisensory Processing , G. A. Calvert , C. Spence and B. E. Stein (Eds), pp. 69 – 83 . MIT Press , Cambridge, MA, USA .

Stevenson , R. J. and Boakes , R. A.

( 2004 ). Sweet and sour smells: learned synesthesia between the senses of taste and smell , in: The Handbook of Multisensory Processing,

G. A. Calvert , C. Spence and B. E. Stein

Stevenson , R. J. and Tomiczek , C. ( 2007 ). Olfactory-induced synesthesias: a review and model , Psychol. Bull. 133 , 294 – 309 . DOI: 10.1037/0033-2909.133.2.294 .

Stevenson , R. J. and Tomiczek , C.

Stewart , J. and Khatchatourov , A. ( 2007 ). Transparency_1 , in: Enaction and Enactive Interfaces: a Handbook of Terms , A. Luciani and C. Cadoz (Eds), pp. 2990 – 2991 . Enactive Systems Books .

Stewart , J. and Khatchatourov , A.

( 2007 ). Transparency_1 , in: Enaction and Enactive Interfaces: a Handbook of Terms,

Stiles , N. R. B. and Shimojo , S. ( 2015 ). Auditory sensory substitution is intuitive and automatic with texture stimuli , Sci. Rep. 5 , 15628 . DOI: 10.1038/srep15628 .

Stiles , N. R. B. and Shimojo , S.

Striem-Amit , E. and Amedi , A. ( 2014 ). Visual cortex extrastriate body-selective area activation in congenitally blind people “seeing” by using sounds , Curr. Biol. 24 , 687 – 692 . DOI: 10.1016/j.cub.2014.02.010 .

Striem-Amit , E. and Amedi , A.

Striem-Amit , E. , Dakwar , O. , Hertz , U. , Meijer , P. , Stern , W. , Pascual-Leone , A. and Amedi , A. ( 2011 ). The neural network of sensory-substitution object shape recognition , Funct. Neurol. Rehabil. Ergon. 1 , 271 – 278 .

Striem-Amit , E. , Dakwar , O. , Hertz , U. , Meijer , P. , Stern , W. , Pascual-Leone , A. and Amedi , A.

Striem-Amit , E. , Cohen , L. , Dehaene , S. and Amedi , A. ( 2012 ). Reading with sounds: sensory substitution selectively activates the visual word form area in the blind , Neuron 76 , 640 – 652 . DOI: 10.1016/j.neuron.2012.08.026 .

Striem-Amit , E. , Cohen , L. , Dehaene , S. and Amedi , A.

Stroop , J. R. ( 1935 ). Studies of interference in serial verbal reactions , J. Exp. Psychol. 18 , 643 – 662 .

Terhune , D. B. , Luke , D. P. , Kaelen , M. , Bolstridge , M. , Feilding , A. , Nutt , D. , Carhart-Harris , R. and Ward , J. ( 2016 ). A placebo-controlled investigation of synaesthesia-like experiences under LSD , Neuropsychologia 88 , 28 – 34 . DOI: 10.1016/j.neuropsychologia.2016.04.005 .

Terhune , D. B. , Luke , D. P. , Kaelen , M. , Bolstridge , M. , Feilding , A. , Nutt , D. , Carhart-Harris , R. and Ward , J.

Terhune , D. B. , Luke , D. P. and Kadosh , R. C. ( 2017 ). The induction of synaesthesia in non-synaesthetes , in: Sensory Blending: on Synaesthesia and Related Phenomena , O. Deroy (Ed.), pp. 215 – 247 . Oxford University Press , Oxford, UK .

Terhune , D. B. , Luke , D. P. and Kadosh , R. C.

( 2017 ). The induction of synaesthesia in non-synaesthetes , in: Sensory Blending: on Synaesthesia and Related Phenomena,

Trivedi , B. ( 2010 ). Sensory hijack: rewiring brains to see with sound , New Sci. Health 2773 , 42 – 45 . Available at https://www.newscientist.com/article/mg20727731-500-sensory-hijack-rewiring-brains-to-see-with-sound/ .

Ward , J. ( 2004 ). Emotionally mediated synaesthesia , Cogn. Neuropsychol. 21 , 761 – 772 . DOI: 10.1080/02643290342000393 .

Ward , J. ( 2013 ). Synesthesia , Annu. Rev. Psychol. 64 , 49 – 75 . DOI: 10.1146/annurev-psych-113011-143840 .

Ward , J. and Mattingley , J. B. ( 2006 ). Synaesthesia: an overview of contemporary findings and controversies , Cortex 42 , 129 – 136 . DOI: 10.1016/S0010-9452(08)70336-8 .

Ward , J. and Mattingley , J. B.

Ward , J. and Meijer , P. ( 2010 ). Visual experiences in the blind induced by an auditory sensory substitution device , Consc. Cogn. 19 , 492 – 500 . DOI: 10.1016/j.concog.2009.10.006 .

Ward , J. and Wright , T. ( 2014 ). Sensory substitution as an artificially acquired synaesthesia , Neurosci. Biobehav. Rev. 41 , 26 – 35 . DOI: 10.1016/j.neubiorev.2012.07.007 .

Ward , J. , Huckstep , B. and Tsakanikos , E. ( 2006 ). Sound–colour synaesthesia: to what extent does it use cross-modal mechanisms common to us all? , Cortex 42 , 264 – 280 . DOI: 10.1016/S0010-9452(08)70352-6 .

Ward , J. , Huckstep , B. and Tsakanikos , E.

White , B. W. , Saunders , F. A. , Scadden , L. , Bach-Y-Rita , P. and Collins , C. C. ( 1970 ). Seeing with the skin , Percept. Psychophys. 7 , 23 – 27 . DOI: 10.3758/BF03210126 .

White , B. W. , Saunders , F. A. , Scadden , L. , Bach-Y-Rita , P. and Collins , C. C.


Results

Pointing Results

We measured the precision of the subject’s direction estimation in a pointing exercise to determine the impact of tactile signals and the familiarity of the environment on spatial learning. The absolute angular deviation in each session was collapsed over all six pointing locations to determine the median error (Figure 3B). Median pointing errors for all pointing sessions are smaller for the conditions with belt (park, camping site) than for the control condition (lake). Pointing estimates of beeline direction in the familiar environment (camping site, gray) yielded the smallest deviation between true direction and estimated direction. We applied the sign rank test for repeated measurements to the distributions of errors calculated in the different pointing sessions. In case of session 2 and session 3, the errors of the estimated airline directions from the park (with belt) and the lake (no belt) are significantly different (for session 2: p < 0.05 for session 3: p < 0.001). Collapsing the error of beeline direction estimation over all four sessions leads to a significant difference between the conditions park and lake (p < 0.001). There are no significant differences between the four sessions of either environment, i.e., the performance of the subject was stable over time. In summary, the tactile signal improved performance in an unknown environment from the first pointing session on as would be predicted by the hypothesis of weak integration.

Pointing performance suggests that the utility of the signal was partly location dependent. Two locations of the park were compared to obtain further insights. During the pointing sessions at the park, the subject indicated several times that he had more trouble with some locations than with others. In the locations perceived as �sy” by the subject, performance was high from the first session on, although the subject had only been introduced once to the six objects’ locations prior to the first airline direction estimation assessment. Performance was stable over sessions at this location, with the average pointing error staying approximately at constant level. The locations perceived as 𠇍ifficult” posed a bigger problem for the subject. Performance varied over sessions and pointing estimations appeared to be more arbitrary. In Figure 3C, the estimated airline direction and the true airline direction for two object locations (one easy, one difficult) are shown before and after training. In the first pointing session before the training with the belt, the median error of the easy location was 41°, and 163° for the difficult location (in several cases the subject pointed almost in the opposite direction). In post-training session, the easy location’s median pointing error was reduced to 23° and the difficult location’s median pointing error to 84°. This leads to the conclusion that variability of performance is not on a single trial basis, but that some locations are consistently more difficult than others for the subject.

Beeline direction estimation performance is not symmetrical, e.g., pointing from location A to location B is not equal to the performance of pointing from location B to location A. The pointing error from the easy to the difficult location in the first session was 16°, in the fourth session 22°. The error in the first session from the difficult to the easy location was 172° and in the fourth session, it still came to 84°. This asymmetry indicates that the subject’s internal estimates are based on an egocentric (self-based) rather than allocentric (bird’s eye perspective) representation of the environment.

Finally, we tested the subject’s ability to instantly integrate the belt signal after the training period of 6 weeks at the lake. In all previous sessions, the subject explored the lake entirely without any information from the belt. After four sessions without the belt, the subject’s belt was switched on in a fifth session. The subject stated several times that the task of giving beeline direction estimates was subjectively facilitated through the belt. This statement was indicative of his actual performance (Figure 3D). The median pointing error for each location decreased in the fifth session, except for the location depicted in pink, where the error was always low, and the location in black. The latter location was the first on the route where the belt’s information was of no use for the subject, since knowing magnetic north did not by itself supply any new information about the other five locations. However, while walking the distance to the other locations, the subject was obviously able to adjust his internal map with the aid of the new information. His beeline direction estimations for session five show an improved understanding of the environment’s layout. After the training, the belt was instantly useful for the beeline direction estimation in an environment previously explored without the belt.

The subject’s ability to give beeline direction estimation was tested in familiar and unfamiliar environments with and without directional information supplied by the belt. The results give experimental evidence for the hypothesis of weak integration. The belt’s signal was integrated and gave the subject useful additional information about the environment: the median pointing error made at the park, where the belt was worn, was reduced by nearly 50% compared to the median pointing error made at the lake. Furthermore, the belt’s signal was instantly useful for an improvement of airline direction estimation in a semi-familiar environment, satisfying the requirements for the weak integration hypothesis: the device can instantly be used if attended to.

The subject stated in several interviews that he perceived the belt as especially useful for the correction of his internal maps of familiar environments (see Subjective Methods Results for the results of the interviews). In accordance with this statement, the median pointing error of the familiar environment �mping site” is reduced by almost 50% compared to the unfamiliar environment explored with the belt (the park). Thus, it seems that the belt’s information could be especially well integrated in a familiar environment. Since there was no control condition for the familiar environment, it is not clear whether the pointing performance was dependent on the belt or the familiarity of the environment. However, we did not want to impose yet another task on the subject, since he perceived experimental sessions as a test of his navigational abilities rather than an experiment relating the usability of the tactile belt. Furthermore, in a familiar environment it would have been difficult to ensure that he never – accidentally or not – enters the track during his daily routines while wearing the belt. Due to the lack of a suitable control condition, it cannot be finally resolved how much the belt contributes to pointing performance in a familiar environment.

There are no significant differences between the four sessions of either condition. The performance of the subject was stable and the median error of the pointing sessions remained at the same level from the first pointing session on. It is likely that the subject’s cognitive grounding in regard to cardinal directions allowed him to instantly make use of the signal. Another explanation of the results would be a difference in difficulty at the park and the lake however, since the median pointing error at the park did not decrease even after training with the belt and the first pointing with belt at the lake yielded a reduced median pointing error compared to the four sessions without the belt, we have no indication that the difficulties of the environment were fundamentally different. The stability of the median values in all three pointing environments indicates that the subject was instantly able to integrate the belt’s signal into his behavior by concentrating on it, as is predicted by the weak integration hypothesis.

Despite the directional signal, not all locations were understood and placed into an internal map equally well. From the first pointing sessions on, the subject had a preference for certain locations that he perceived as easy, while others were perceived as difficult. His performance varied according to his own difficulty rating. It is remarkable that even after training with the belt some locations posed more difficulties than others. Since there were only six pointing locations on each track, it must remain speculative what distinguished the difficult from the easy ones. Further tests with more subjects would be necessary to determine whether the location in itself posed the difficulty or whether difficulty ratings would be depended on each subject’s personal preference. Another pointing test at the locations perceived as difficult to a wide range of objects, arranged in a circle around the location, could shed light on the contortion of the subject’s internal map. It was not possible to do further tests in the frame of this study since all testing situations imposed stress on the subject. Further tests are necessary to draw an informed conclusion.

In the fifth session, the subject received information from the belt at the lake, which in all previous sessions had been explored without directional information. The median error dropped to the level of pointing performance at the park, indicating that the subject instantly integrated the belt’s signal. The finding that the signal was useful for the re-exploration of the lake provides further evidence. All in all, the results of the beeline estimation task provide evidence for the weak integration hypothesis and can be taken as an indicator that the signal is instantly useful if attended to.

Straight-Line-Walking Task Results

We tested whether the belt enabled the subject to keep a straight direction for a longer distance and whether performance in this regard was depended on attentional mechanisms. Of the four trials, the subject had the largest deviation (15.6°) from the straight path in the dual-task condition without belt. The angular deviation in all other conditions were between 9.9° and 11.3°. The subject’s ability to walk a straight line with the aid of the belt was the same regardless of an additional task that drew on attentional capacity. In the trials without the belt, the one with an additional subtracting exercise has a larger angular error.

The task of walking a straight line is of high relevance for the blind subject. It was found that keeping a straight direction was easier for the subject if he received direction information from the tactile belt. Due to the small number of trials, no final conclusion can be drawn, but the results show a trend, indicating that the tactile belt could be a helpful device in everyday situations such as crossing a large street. Especially relevant is the finding that the integration of the belt’s signal was not disturbed by the additional subtraction exercise, suggesting that straight-line-walking could be facilitated for the blind traveler by directional information while at the same time other environmental inputs, e.g., auditory cues from traffic, could be processed in parallel. Usually, the blind are taught different methods to walk straight however, without a directional signal they are unable to examine their own performance without further environmental cues. In terms of our hypotheses, the results of the straight-line-walking task satisfy both, the weak integration hypothesis, because the belt was helpful when attended to, and the sub-cognitive hypothesis, because the belt’s information could be integrated just as well when an additional subtraction task impedes the conscious, attentional mechanisms of signal processing. Further tests with more trials would be necessary to draw stronger conclusions. The resulting deviations from the straight direction indicate that the belt can be a helpful tool in everyday situations for blind individuals. Furthermore, the results can also be seen as further evidence for the weak integration hypothesis, i.e., that the belt’s signal can be meaningfully integrated into behavior, and the sub-cognitive hypothesis, that attention is not a necessary prerequisite for a successful integration of the signal.

Homing Results

In this part of the experiment, the subject’s ability to complete a complex path with and without information supplied by the tactile device was compared. An additional memory task restricted attentional capacity. We observed a significant difference between the distributions of error values with/without belt in the experimental session directly after training (p < 0.05, Figure 5B). Compared to the no belt condition the angular error made by the subject was significantly smaller for the trials where the subject received information from the tactile belt. In the session before training and 4 weeks after the study, no significant difference between the trials with and without belt could be determined. Since attentional capacities were partly blocked by the additional memory task, these results can be taken as evidence for a sub-cognitive integration of the signal.

The subject completed a homing task before training with the belt, directly after 6 weeks of training and 4 weeks later. In the session directly after training, the angular error was significantly decreased only in those trials where the belt was switched on. In all other sessions, no significant difference between conditions was found.

In the frame of our hypotheses, the results support the hypothesis of sub-cognitive integration of the signal. The inability of the subject to use to belt signal in the first session does not contradict the hypothesis of weak integration, since attention to the signal is one of its presumptions. Weak integration relies on conscious and attentional mechanisms, which were blocked at least partly by a memory task. Through training with the belt, sub-cognitive mechanisms in response to the signal developed and integration of the tactile information became possible for the subject in parallel to the attention demanding memory task.

Four weeks after the end of the study, the belt did not affect performance. Thus it can be presumed that the sub-cognitive mechanisms that were at work in the trial directly after training diminish after a prolonged period without training. It is possible that a longer time span of training would result in a longer storage of the newly learned sensorimotor contingencies and thus the homing results could stay stable over a longer period of time. In the present case, the signal has clearly lost its effects after 4 weeks. It seems that the hypothesis of sub-cognitive integration must be extended by a temporal component, since the sub-cognitive mechanisms have to be trained constantly to be effective.

Subjective Methods Results

The daily questionnaires assessed daily activities with the belt as well as health and mood of the subject. During working hours as a computer operator he did few trunk movements and therefore took off the belt. This resulted in 8 h per day net training time. He actively explored the belt’s function during long tandem tours with his tandem partner and by walks in familiar territory. Throughout the whole study he felt healthy at an average rating of 4.93 (scale 1𠄵) and slept well with an average rating of 4.00. During the study he rated his cheerfulness (average 4.65 of 5), alertness (average 4.50 of 5), quietness (average 3.15 of 5), and listlessness (average 1.23 of 5) on a daily basis. Unfortunately, the day of the final measurements of the study was an outlier in all aspects. Due to private difficulties from family matters, the subject rated his health with 4 (which is a strong statement, since he chose 5 in all other cases except one other outlier in the second week of the study), did not sleep well (2 of 5), felt neither cheerful nor alert (both rated 3 of 5), and felt more quiet (4 of 5) and listless (3 of 5) than usual. With this exception, the subject’s mood and health were stable throughout the study and the belt had no negative impact whatsoever.

Face-to-face interviews with a standardized catalog of questions were conducted weekly. Additional questions were asked in a spontaneous, situation-dependent manner. The questions of the face-to-face interviews fall into four categories: the hedonic quality of the belt the kind of information perception utility of the belt, and the subjective feeling of security with the belt.

Three questions targeted how the subject and those in his environment reacted to the belt. The subject rated his motivation to wear the belt as “very high” (5 of 5) throughout the study, and only in the fifth week a dip (to 4 of 5) occurred. The belt did not constrain him in his daily routine (rated 1 or 2 of 5). The only problematic issue he reported was the sweating that the belt, worn tightly around the waist, promoted during hot weather. Strangers he encountered did either not react to the belt at all, assuming it was simply a gadget for the blind, or were curious and responded positively. Overall the subject gave a strong positive evaluation of the hedonic quality of the belt.

A second focus of the questionnaire was the quality of the signal as perceived by the subject. During the first 2 weeks of training, the vibrations were prominent. The subject perceived phantom vibrations regularly after the belt was taken off in the evening. After that period, the salience of the vibration decreased. “By now, I really have to concentrate otherwise I don’t perceive the belt, because the prickling is already being internalized.” However, the subject did not report the emergence of a qualitatively new sensation. Over the course of training the subject got accustomed more and more to the belt’s signal. In the first 3 weeks, he consciously attended to the signal to use the additional information. In the second half of the training period no conscious efforts was necessary to use the information provided by the belt.

With respect to the utility of the belt, the subject found the directional signal helpful in some situations, but less useful in others. Furthermore, his evaluation changed repeatedly during the study. Initially, he believed the belt would be of great value for navigation in unfamiliar environments. In the third week, he stated that the belt would be very useful in case he was placed in an unknown environment by himself and had to find his way back: “If I was now abandoned somewhere, I would absolutely with the belt, else I had to with the sun” (incomplete statement). However, in the following weeks the subject noticed that solely the directional information of magnetic north would not suffice for successful navigation in unknown environments. In the fifth week he contradicts his earlier statement: “If I was abandoned somewhere somehow, the belt is not really useful for me, because I don’t know the environment.”

His rating of the belt’s utility in unfamiliar environments reflects this change of mind: in the first 3 weeks, he fully agrees with the statement “I find it much easier with belt than without to orient myself in an unfamiliar environment” (rating of 5 out of 5). In the fourth week, his enthusiasm declines, and his rating drops to 2 in the fifth and sixth week. In other regards, the subject found the information received from the belt extremely useful. He fully agreed with the statement “Since I’ve started wearing the belt, I perceive the cardinal directions more consciously,” giving it a rating of 5 of 5 during the whole study. The statement “With the belt I can estimate the streets’ arrangement in respect to each other better as without belt,” initially rated with 3 of 5 in the first week, was rated with 4 or 5 of 5 from the second week on which gives an indication of the learning by the subject. The statement “With the belt, I always know where I am in relation to my home” received low ratings (in the second week 1 or 2, with one maximum of 3 of 5). However, the subject’s explanation for the low ratings is not related to the tactile device: he said repeatedly that knowledge about the direction of his home was not relevant for him as a blind person as he needs to be attentive to his immediate environment at all times. Conclusively, the subjective ratings concerning the utility of the belt showed a characteristic time course.

The subject found the belt most helpful to verify his internal representations of familiar environments. He used the expressions “mental map” or “internal map” to explain how he memorizes environments. In an unfamiliar environment, he uses an initial point for orientation and starts exploring the environment step by step. Even though he owns a speaking compass, he usually uses the warmth of the sun for directional information. In several occasions during the study, the subject expressed surprise concerning the difference of the actual direction supplied by the belt and his subjective estimation of direction, a sense he had previously believed to be very accurate. Talking about his mental map and its compatibility with the belt at the end of week three: “I have to correct it by 20°. Because I always had an estimation of direction, and now I have accurate information.” He describes how his internal map of familiar environments reluctantly changes to integrate the information supplied by the belt: “In familiar environments, the belt is useful for validation if I was right (=his mental map) []. If the belt did not support the mental map, at first I did not want to accept it, but eventually, the belt has had me convinced.” In the last interview, the subject explained that the belt was especially helpful for an accurate understanding of familiar environments. “If a long road is slightly curved, I realized with the belt. Without the belt, I wouldn’t have realized, but the vibration moved slightly. On the camping site, we also have this long straight way that is slightly curved – I always had to count steps there, but now I don’t have to anymore. When the other vibrator vibrates, then I have to turn right after passing the forest.” In summary, the subject perceived the directional signal of limited practical use in unfamiliar environments, but experienced the tactile device as especially valuable for the correction of already existing mental maps and easing navigation.

An important part of the study investigated whether the subject would feel more secure with the belt, and whether the directional signal would introduce new behavioral possibilities for the subject. The statement “With the belt, I feel more secure in an unfamiliar environment” was rated with high agreement (4 or 5) in the first 4 weeks of the study, dropped to the lowest value in week five and climbed up to a medium rating in the last week. The statement “my ability to orient myself has subjectively improved since I’ve been wearing the belt” was given medium to high assent in the first 3 weeks, lower ratings in week four and five, and went up to high in the last week. The subject emphasized that his orientation had been already excellent prior to the study. While in the first 3 weeks he gave the statement “my ability of orientation declines after taking off the belt” a medium rating, he later on rated this statement even lower, indicating that he did not feel any subjective loss in orientation ability without the belt. Please note that this pertains to taking off the belt in the evening, not to the time after the training period. While the standardized questions regarding his subjective experience of security where rather poorly approved by the subject, he noted on several occasions that the belt had inspired him to try new things like, e.g., extending the radius of the tandem tours. In the interviews prior to the study, the subject explained that shortcuts are a risky undertaking for a blind person, since it is easy to loose the initial direction, and re-orientating is difficult once the familiar path has been left: “I can’t just walk diagonally over a meadow, I’ve tried before, if you are lucky, you make it, if notwell…[…] but for a blind person it is simply much too dangerous and unsafe. One always goes in right angles, looking for fix points… the life of a blind person is oriented in right angles.” Here, he had positive experiences with the belt: “One dares to go a little diagonal, because I can keep the direction with the aid of the vibration. I don’t digress in a curve and get lost… I would never do this otherwise (without the belt), if you get really lost as a blind person, then you don’t know where you are.” Furthermore, he explained that he feels insecure if the sun does not shine and can thus not be used for orientation during the tandem tours he regularly makes with a sighted partner. In this regard, he also perceived the belt as being useful: “It helps in the countryside, when I drive the tandem with H. To combine a familiar environment with an unfamiliar one, by simply entering the unknown environment, that’s where the belt is helpful.” In the last interview, he summarized his experiences: “I summarize: In a familiar environment, the belt provides verification, and in an unfamiliar environment, one is more courageous.” In summary, the interviews reflected an initial enthusiasm, followed by disenchantment and lastly a differentiated valuation of the belt with an emphasis on ease of navigation in familiar environments and a more courageous activity in unfamiliar environments.

Through daily and weekly interviews, we assessed different aspects of the subject’s experience during the study. The subject was highly motivated to wear the belt, especially since his mental maps of the environment were strong and well arranged before onset of the study and continued to improve with the aid of the device. He perceived the belt as helpful in situations where he extended his knowledge about familiar environments. In later weeks, he had a differentiated understanding of the belt’s utility and how the additional spatial information can be meaningfully integrated into everyday behavior.

The rating scheme through the course of the study suggests that the subject had high expectations at the beginning of the study and enthusiastically embraced the possible utilities of the directional signal. In the fourth and fifth week, his answers reflect disillusionment as he started to critically evaluate the usability of the device in his everyday life. In the last week, his ratings reflect a differentiated understanding of the belt’s potential. Notably, the subject never received quantitative feedback after taking part in the behavioral experiments and thus had no objective knowledge of his performance with the belt. The discrepancy between the subject’s experience and his real performance was particularly evident in the final interview: when asked about the homing test conducted directly after the study, he answered: “I wanted to use the belt a little, but you shouldn’t find a big difference between the trials with belt and the ones without, I believe.” This self-assessment is discordant with the behavioral results, where a significant difference was found between the trials with belt and the ones without. Thus, the subject came to a cautious rational and a positive emotional evaluation of the belt, and the latter better reflects his true performance with the belt.


SENSORY ORGANS FACTS

While the five senses are considered the basic starting block for the sensory system, each sense has additional senses that work within its framework, creating a multitude of senses. The sense of touch has pain, cold, heat, and so forth within its framework of additional senses. The senses are considered either general if their pathways of communication are basic and simple and are considered special if the pathway of communication are complex or require distortion by the sensory system.


Cerebral Cortex

The outer portion of your cerebrum is covered by a thin layer of gray tissue called the cerebral cortex. This layer is 1.5 to 5 millimeters in thickness. Your cerebral cortex is in turn divided into four lobes: frontal lobes, parietal lobes, temporal lobes, and occipital lobes. Your cerebrum, along with the diencephalon, which includes the thalamus, hypothalamus, and the pineal gland, comprises the two major divisions of the prosencephalon (forebrain).

Your cerebral cortex handles a number of the most important brain functions. Among these functions is the processing of sensory information by the cortex lobes. Limbic system brain structures located beneath the cerebrum also assist in sensory information processing. These structures include the amygdala, thalamus, and hippocampus. Limbic system structures use sensory information to process emotions and connect your emotions with memories.

Your frontal lobes are responsible for complex cognitive planning and behaviors, language comprehension, speech production, and the planning and control of voluntary muscle movement. Nerve connections with the spinal cord and brainstem allow the cerebrum to receive sensory information from your peripheral nervous system. Your cerebrum processes this information and relays signals that produce the appropriate response.


How Gate Control Works

Following an injury, pain signals are transmitted to the spinal cord and then up to the brain. Melzack and Wall suggest that before the information is transmitted to the brain, the pain messages encounter "nerve gates" that control whether these signals are allowed to pass through to the brain.

In some cases, the signals are passed along more readily and pain is experienced more intensely. In other instances, pain messages are minimized or even prevented from reaching the brain at all.

This gating mechanism takes place in the dorsal horn of the body's spinal cord. Both small nerve fibers (pain fibers) and large nerve fibers (normal fibers for touch, pressure, and other skin senses) both carry information to two areas of the dorsal horn.

These two areas are either the transmission cells that carry information up to the spinal cord to the brain or the inhibitory interneurons which halt or impede the transmission of sensory information.

  • Large fiber activity excites the inhibitory neurons, which diminishes the transmission of pain information. When there is more large fiber activity in comparison to small fiber activity, people tend to experience less pain. This means that the pain gates are closed.
  • Small fibers impede the inhibitory interneurons, allowing pain information to travel up to the brain. When there is more small fiber activity, it inactivates the inhibitory neurons so that pain signals can be sent to the brain in order for pain perception (also known as nociception) to take place. In other words, the pain gates are now open.

While it is perhaps the most influential theory of pain perception, gate control is not without problems. Many of the ideas suggested by Melzack and Wall have not been substantiated by research, including the very existence of an actual gating system in the spinal cord.


Sensory neurons

Sensory neurons are the nerve cells that are activated by sensory input from the environment - for example, when you touch a hot surface with your fingertips, the sensory neurons will be the ones firing and sending off signals to the rest of the nervous system about the information they have received.

The inputs that activate sensory neurons can be physical or chemical, corresponding to all five of our senses. Thus, a physical input can be things like sound, touch, heat, or light. A chemical input comes from taste or smell, which neurons then send to the brain.

Most sensory neurons are pseudounipolar, which means they only have one axon which is split into two branches.


Types of Sensory Memory

Experts also believe that different senses have different types of sensory memory. The different types of sensory memory have also been shown to have slightly different durations.

    : Also known as visual sensory memory, iconic memory involves a very brief image. This type of sensory memory typically lasts for about one-quarter to one-half of a second.  
  • Echoic memory: Also known as auditory sensory memory, echoic memeory involves a very brief memory of sound a bit like an echo. This type of sensory memory can last for up to three to four seconds.
  • Haptic memory: Also known as tactile memory, haptic involves the very brief memory of a touch. This type of sensory memory lasts for approximately two seconds.