A Novel Form of Stereo Vision in the Praying Mantis by Nityananda V, Tarawneh G, Henriksen S, Umeton D, Simmons A, Read JCA , NityanandaTarawnehHenriksenUmetonSimmonsRead2018.pdf (2.5 MiB) - Stereopsis is the ability to estimate distance based on the different views seen in the two eyes. It is an important model perceptual system in neuroscience and a major area of machine vision. Mammalian, avian, and almost all machine stereo algorithms look for similarities between the luminance-defined images in the two eyes, using a series of computations to produce a map showing how depth varies across the scene. Stereopsis has also evolved in at least one invertebrate, the praying mantis. Mantis stereopsis is presumed to be simpler than vertebrates’, but little is currently known about the underlying computations. Here, we show that mantis stereopsis uses a fundamentally different computational algorithm from vertebrate stereopsis --
rather than comparing luminance in the two eyes’ images directly, mantis stereopsis looks for regions of the images where luminance is changing. Thus, while there is no evidence that mantis stereopsis works at all with static images, it successfully reveals the distance to a moving target even in complex visual scenes with targets that are perfectly camouflaged against the background in terms of texture. Strikingly, these insects outperform human observers at judging stereoscopic distance when the pattern of luminance in the two eyes does not match. Insect stereopsis has thus evolved to be computationally efficient while being robust to poor image resolution and to discrepancies in the pattern of luminance between the two eyes.
Blog Archives
Invisible noise obscures visible signal in insect motion detection
Invisible noise obscures visible signal in insect motion detection by Tarawneh G, Nityananda V, Rosner R, Errington S, Herbert W, Cumming BG, Read JCA, Serrano-Pedraza I, TarawnehNityanandaRosnerErringtonHerbertCummingReadSerranoPedraza2017.pdf (2.6 MiB) - The motion energy model is the standard account of motion detection in animals from beetles to
humans. Despite this common basis, we show here that a difference in the early stages of visual
processing between mammals and insects leads this model to make radically different behavioural
predictions. In insects, early filtering is spatially lowpass, which makes the surprising prediction that
motion detection can be impaired by “invisible” noise, i.e. noise at a spatial frequency that elicits
no response when presented on its own as a signal. We confirm this prediction using the optomotor
response of praying mantis Sphodromantis lineola. This does not occur in mammals, where spatially
bandpass early filtering means that linear systems techniques, such as deriving channel sensitivity from
masking functions, remain approximately valid. Counter-intuitive effects such as masking by invisible
noise may occur in neural circuits wherever a nonlinearity is followed by a difference operation.
humans. Despite this common basis, we show here that a difference in the early stages of visual
processing between mammals and insects leads this model to make radically different behavioural
predictions. In insects, early filtering is spatially lowpass, which makes the surprising prediction that
motion detection can be impaired by “invisible” noise, i.e. noise at a spatial frequency that elicits
no response when presented on its own as a signal. We confirm this prediction using the optomotor
response of praying mantis Sphodromantis lineola. This does not occur in mammals, where spatially
bandpass early filtering means that linear systems techniques, such as deriving channel sensitivity from
masking functions, remain approximately valid. Counter-intuitive effects such as masking by invisible
noise may occur in neural circuits wherever a nonlinearity is followed by a difference operation.
Stereopsis in animals: evolution, function and mechanisms
Stereopsis in animals: evolution, function and mechanisms by Nityananda V, Read JCA, NityanandaRead2017.pdf (0.8 MiB) - Stereopsis is the computation of depth information from views
acquired simultaneously from different points in space. For many
years, stereopsis was thought to be confined to primates and other
mammals with front-facing eyes. However, stereopsis has now been
demonstrated in many other animals, including lateral-eyed prey
mammals, birds, amphibians and invertebrates. The diversity of
animals known to have stereo vision allows us to begin to investigate
ideas about its evolution and the underlying selective pressures in
different animals. It also further prompts the question of whether all
animals have evolved essentially the same algorithms to implement
stereopsis. If so, this must be the best way to do stereo vision, and
should be implemented by engineers in machine stereopsis.
Conversely, if animals have evolved a range of stereo algorithms in
response to different pressures, that could inspire novel forms of
machine stereopsis appropriate for distinct environments, tasks or
constraints. As a first step towards addressing these ideas, we here
review our current knowledge of stereo vision in animals, with a view
towards outlining common principles about the evolution, function
and mechanisms of stereo vision across the animal kingdom. We
conclude by outlining avenues for future work, including research into
possible new mechanisms of stereo vision, with implications for
machine vision and the role of stereopsis in the evolution of
camouflage.
acquired simultaneously from different points in space. For many
years, stereopsis was thought to be confined to primates and other
mammals with front-facing eyes. However, stereopsis has now been
demonstrated in many other animals, including lateral-eyed prey
mammals, birds, amphibians and invertebrates. The diversity of
animals known to have stereo vision allows us to begin to investigate
ideas about its evolution and the underlying selective pressures in
different animals. It also further prompts the question of whether all
animals have evolved essentially the same algorithms to implement
stereopsis. If so, this must be the best way to do stereo vision, and
should be implemented by engineers in machine stereopsis.
Conversely, if animals have evolved a range of stereo algorithms in
response to different pressures, that could inspire novel forms of
machine stereopsis appropriate for distinct environments, tasks or
constraints. As a first step towards addressing these ideas, we here
review our current knowledge of stereo vision in animals, with a view
towards outlining common principles about the evolution, function
and mechanisms of stereo vision across the animal kingdom. We
conclude by outlining avenues for future work, including research into
possible new mechanisms of stereo vision, with implications for
machine vision and the role of stereopsis in the evolution of
camouflage.
The optomotor response of the praying mantis is driven predominantly by the central visual field
The optomotor response of the praying mantis is driven predominantly by the central visual field by Nityananda V, Tarawneh G, Errington S, Serrano-Pedraza I, Read JCA, NityanandaTarawnehErringtonSerranoPedrazaRead.pdf (1.9 MiB) - The optomotor response has been widely used to
investigate insect sensitivity to contrast and motion. Several
studies have revealed the sensitivity of this response
to frequency and contrast, but we know less about the
spatial integration underlying this response. Specifically,
few studies have investigated how the horizontal angular
extent of stimuli influences the optomotor response. We
presented mantises with moving gratings of varying horizontal
extents at three different contrasts in the central or
peripheral regions of their visual fields. We assessed the
relative effectivity of different regions to elicit the optomotor
response and modelled the dependency of the response
on the angular extent subtended by stimuli at these different
regions. Our results show that the optomotor response
is governed by stimuli in the central visual field and not
in the periphery. The model also shows that in the central
region, the probability of response increases linearly with
increase in horizontal extent up to a saturation point. Furthermore,
the dependency of the optomotor response on the
angular extent of the stimulus is modulated by contrast. We
discuss the implications of our results for different modes
of stimulus presentation and for models of the underlying
mechanisms of motion detection in the mantis.
investigate insect sensitivity to contrast and motion. Several
studies have revealed the sensitivity of this response
to frequency and contrast, but we know less about the
spatial integration underlying this response. Specifically,
few studies have investigated how the horizontal angular
extent of stimuli influences the optomotor response. We
presented mantises with moving gratings of varying horizontal
extents at three different contrasts in the central or
peripheral regions of their visual fields. We assessed the
relative effectivity of different regions to elicit the optomotor
response and modelled the dependency of the response
on the angular extent subtended by stimuli at these different
regions. Our results show that the optomotor response
is governed by stimuli in the central visual field and not
in the periphery. The model also shows that in the central
region, the probability of response increases linearly with
increase in horizontal extent up to a saturation point. Furthermore,
the dependency of the optomotor response on the
angular extent of the stimulus is modulated by contrast. We
discuss the implications of our results for different modes
of stimulus presentation and for models of the underlying
mechanisms of motion detection in the mantis.
Unravelling the illusion of flicker fusion
Unravelling the illusion of flicker fusion by Umeton D, Read JCA, Rowe C, UmetonReadRowe2017.pdf (0.7 MiB) - For over 150 years, researchers have investigated the anti-predator function of animal patterns. However, this work has mainly focused on when prey remain still, and has only recently started to incorporate motion into the study of defensive coloration. As motion breaks camouflage, a new challenge is to understand how prey avoid predators while moving around their environment, and if a moving prey can ever be camouflaged. We propose that there is a solution to this, in that a ‘flicker fusion effect’ can change the appearance of the prey in the eyes of their predators to reduce the chances of initial detection. This effect occurs when a high contrast pattern blurs at speed, changing the appearance of the prey, which may help them better match their background. Despite being widely discussed in the literature, the flicker fusion effect is poorly described, there is no clear theoretical framework for testing how it might reduce predation, and the terminology describing it is, at best, rather confusing. Our review addresses these three key issues to enable researchers to formulate precise predictions about when the flicker fusion effect occurs, and to test how it can reduce predation.
Small or far away? Size and distance perception in the praying mantis.
Small or far away? Size and distance perception in the praying mantis. by Nityananda V, Bissianna G, Tarawneh G, Read JCA, Nityananda_et_al2015_PhilTrans_PostReview.pdf (1.5 MiB) - Stereo or '3D' vision is an important but costly process seen in several evolutionarily distinct lineages including primates, birds and insects. Many selective advantages could have led to the evolution of stereo vision, including range finding, camouflage breaking and estimation of object size. In this paper, we investigate the possibility that stereo vision enables praying mantises to estimate the size of prey by using a combination of disparity cues and angular size cues. We used a recently developed insect 3D cinema paradigm to present mantises with virtual prey having differing disparity and angular size cues. We predicted that if they were able to use these cues to gauge the absolute size of objects, we should see evidence for size constancy where they would strike preferentially at prey of a particular physical size, across a range of simulated distances. We found that mantises struck most often when disparity cues implied a prey distance of 2.5 cm; increasing the implied distance caused a significant reduction in the number of strikes. We, however, found no evidence for size constancy. There was a significant interaction effect of the simulated distance and angular size on the number of strikes made by the mantis but this was not in the direction predicted by size constancy. This indicates that mantises do not use their stereo vision to estimate object size. We conclude that other selective advantages, not size constancy, have driven the evolution of stereo vision in the praying mantis.This article is part of the themed issue 'Vision in our three-dimensional world'.
Insect stereopsis demonstrated using a 3D insect cinema
Insect stereopsis demonstrated using a 3D insect cinema by Nityananda V, Tarawneh G, Rosner R, Nicolas J, Crichton S, Read JCA, NityanandaTarawnehRosnerNicolasCrichtonRead2016.pdf (1.2 MiB) - Stereopsis - 3D vision – has become widely used as a model of perception. However, all our knowledge of possible underlying mechanisms comes almost exclusively from vertebrates. While stereopsis has been demonstrated for one invertebrate, the praying mantis, a lack of techniques to probe invertebrate stereopsis has prevented any further progress for three decades. We therefore developed a stereoscopic display system for insects, using miniature 3D glasses to present separate images to each eye, and tested our ability to deliver stereoscopic illusions to praying mantises. We find that while filtering by circular polarization failed due to excessive crosstalk, “anaglyph” filtering by spectral content clearly succeeded in giving the mantis the illusion of 3D depth. We thus definitively demonstrate stereopsis in mantises and also demonstrate that the anaglyph technique can be effectively used to deliver virtual 3D stimuli to insects. This method opens up broad avenues of research into the parallel evolution of stereoscopic computations and possible new algorithms for depth perception.
The contrast sensitivity function of the praying mantis Sphodromantis lineola
The contrast sensitivity function of the praying mantis Sphodromantis lineola by Nityananda V, Tarawneh G, Jones L, Busby N, Herbert W, Davies R, Read JCA , NityanandaTarawnehJonesEARead2015.pdf (2.6 MiB)