Visual perception and visual mental imagery are cognitive functions and both are sub-served by common mechanisms. Visual mental imagery (VMI) is known as images in the mind or a visual representation without the environmental inputs. It’s seeing with the mind’s eye.
It is well known that the behavior of visual information involves the eye movements which serve as a window into the operation of the attentional system. In fact, there is an important relationship between eye movements and visual attention. However, the functional role of the eye movements has long been discussed in visual mental imagery.
One of several devices to measure the eye movements is the electro-oculography (EOG). This technique will be used in this study to record the eye movements during observation and imagery tasks.
Record the eye movements using EOG in order to evaluate the difference between visual mental imagery and observation.
Twenty participants (13 males and 7 females) are familiarized on a video of grasping hand for 20 sec. Then, they are asked to imagine and to observe (for 10 sec each one) this motion video. Eye movements are recorded during 3 trials of imagery and observation phases.
No significant difference was observed on horizontal and vertical EOG recorded signals during both observation and imagery tasks.
According to the EOG recorded signals, the imagery and observation of a grasping hand video have approximately the same effect on the eye movements.
Keywords: Electro-Oculography; Visual mental imagery; Action Observation.
Both action observation and motor imagery have been presented to play a role in learning or re-learning motor tasks . For example, the motor performance in sport training improved by applying motor imagery. In addition, the action observation presents positive effects on the rehabilitation of both chronic stroke patients and Parkinson’s disease patients. But it has been shown that motor imagery is less powerful than the action observation in the foundation of early learning task. For this reason, we try to find which one of both imagery and observation is more effective on the eye movements .
Vision is a principal sense of human, it is used for two types of purposes: to determine objects, elements, and features like colors and structures, and to follow objects in motion, to navigate and to attain properly . The relationship between vision and eye movements continues to give many possibilities for probing the interaction between perception and action . The perception of the visual scene is a basic cognitive ability that allows us to recognize where we are and how to act upon our environment .
Mental imagery relates to the imitation and reconstitution of perceptual experience . The visual imagery is an essential form of cognition that is the principal of various mental activities . In fact, VMI has long been studied with various methods such as the questionnaire [8, 9], the functional magnetic resonance imaging (fMRI)[10, 11] and the electroencephalography (EEG) [12, 13]. Moreover, the eye movements are recorded for studying the visual mental imagery (VMI) and the visual perception (VP) whether by electro-oculography (EOG) or by another techniques. Meanwhile, the practice of using the eye movements is considered well to coordinate components of a mental design with components of the visual field .
Most of the previous researches used the eye movements by video-based eye trackers in different tasks to analyze VMI and VP [15–19]. For example, in order to find the relationship between VMI and VP, Brandt and Stark requested the participants to view and to imagine irregularly-checkered diagrams while their eye movements were registered. For any given picture, sequences of fixations and saccades are closely correlated with those who are recorded while viewing the same items .
A further study showed that preventing eye movements in a visuospatial task interfered with the performance, thus suggesting that eye movements can play a causal role in imagery .
In addition, Bourlon et al. requested the participants to visualize a map of France and to estimate whether towns are situated in left or in right of Paris. They found that participants moved their eyes in the corresponding directions as if they actually saw the map .
A recent study suggested that eye movements can serve for larger level of the spatial than the visual component of mental imagery .
In an earlier study, horizontal eye movements were monitored using EOG, during three treatment conditions: free, check and fix condition. Three equivalent experimental lists of 24 nouns were presented to participants who are instructed to form a visual image to each separate noun of these lists. This study presented a reliable work, but minor effects of treatment conditions on the recall scores were obtained .
In their research, Dermarais and Cohen placed EOG electrodes for participants and they presented an auditory series of terms in relation with left/right and below/above. They found that the words “below” and “above” provoked more vertical eye movements and words “left” and “right” sparked more horizontal eye movements .
The purpose of this study is to record eye movements using EOG in order to evaluate the difference between visual mental imagery and visual perception.
The EOG is one of the few measurement techniques used for recording eye movements. The corresponding electrical signal is called electro-oculogram. In this study, according to the EOG, we obtained the same effect of imagery and observation on the eye movements.
Materials and Methods
Twenty participants or subjects (13 males and 7 females) between 20 and 35 years old are participated in this study. They are in good health and have neither personal nor family history of ocular or systemic disorders. In addition, they have a correct visual acuity. Subjects having ocular problem or wearing contact lenses or glasses were excluded from this study. Moreover, participants are well informed concerning this study and before executing the experimental protocol. They are also informed that their pupil size will be measured during the visualization task. Finally, the Ethics Committee at public health faculty of the Lebanese University has been approved the protocol and all participants provided their written informed consensus.
The data of eye movements, in the horizontal and vertical directions, are recorded using Biopac EOG data acquisition system .
During the experiment, participants have been placed in a darkened and silent room. They are seating just in front of a computer screen at the same level of their eyes. The computer screen changed the color to white when the imaging procedure started.
The participant skin is cleaned just before placing the 5 electrodes as shown in figure 1. The metal part of electrodes is not directly in contact with the skin. The small cavity between the skin and the metal part is filled with a conductive gel to provide good contact.
Figure 1. Placement of the 5 EOG electrodes.
First, each participant is familiarized with a motion video covering for 20 sec, a hand grasping without any object. Secondly, participant is asked to leave the eyes opened, and to imagine for 10 sec what he/she saw in the video. Just after, we asked participant to view once again the same video for 10 sec. EOG is recorded during the 10 seconds of imagery and viewing periods in 3 separated trials in a cadence of 1 min as shown in figure 2.
Figure 2. Protocol of imagery/viewing experiment.
Each subject is instructed for 3 phases:
· Familiarization phase: subject is asked to watch the video of hand grasping, and focus on the movement of hand and fingers when opening and closing.
· Imagery phase: In front of a white screen, participant is instructed to consider that the video remains playing on this screen, and to concentrate watching the video in front of his/her eyes without gaze dispersion and always with opening eyes.
· Observation phase: Participant is instructed to watch again the video and still focusing on the motion of hand and fingers.
Statistical analysis is done using Matlab software. Signals are extracted according to the following parameters: mean and standard deviation of the EOG signals of all subjects, the evolution of the average of EOG horizontal and vertical signals of each subject, and the correlation between imagery and observation signals.
The statistical difference between imagery and observation for both horizontal and vertical signals is analyzed by a paired student’s t-test and by considering that the difference is significant for p <0.05.
The average evolution of EOG horizontal signals depending on subjects during imagery and observation is presented in figure 3. This figure shows that horizontal signals change in parallel and have behavior similarity during both imagery and observation tasks except the 2nd and 17th subjects. This is demonstrated by the box plot of figure 4. It shows the average distribution of 20 horizontal signals of imagery and observation are in the same margin except 3 subjects marked in red crosses [2, 3, and 17].
Figure 3. Average evolution of EOG horizontal signals depending on subjects.
Figure 4. Box plot of the average distribution of horizontal signals in imagery case (left side) and in observation case (right side).
The mean (M) and standard deviation (SD) of horizontal signals during imagery (M=-3.4729, SD=0.1017) and observation (M=- 3.4592, SD=0.0655) indicate that the mean of imagery and observation are too close to each other with a low standard deviation.
The student t-test revealed no significant difference between the average of EOG horizontal signals in imagery and observation when p > 0.05. This means that imagery has approximately the same effect of observation on the horizontal eye movement.
Likewise, the average of EOG vertical signals changed in parallel during both imagery and observation tasks except for the 11th and 17th which having aberrant values as shown in figure 5.
Figure 5. Average evolution of vertical signals depending on subjects.
The mean and standard deviation of vertical signals during both of imagery (M=-0.0153, SD=0.1434) and observation (M=-0.0462, SD=0.1238) are too close together.
As in horizontal, there is no significant difference between the average of EOG vertical signals in imagery and observation when p > 0.05, which means that observation and imagery have nearly the same effect on the eye movement concerning the vertical signals.
Table 1 shows the mean and standard deviation differences between the average of horizontal and vertical signals during imagery and observation tasks. It demonstrate that the potential of horizontal signals (-3.4729<M<-3.4592) is higher than the vertical ones (-0.0462<M<-0.0153). So, we note well the scale difference of order 100 between horizontal and vertical signals.
Table 1. Mean and standard deviation of EOG signals.
Correlation between imagery and observation
Table 2 presents the correlation between imagery and observation of horizontal and vertical signals. It shows moderate correlations for subject 13 (R=0.56), but there are also very low correlations (R≈0) as well for the fourth subject. If R>0.4 is considered as a significant correlation between imagery and observation is when, then we can note that just the seventeenth subject presents a significance correlation between both imagery and observation of horizontal signals (R=-0.4631). In addition, 3 subjects showed significance correlation between imagery and observation of vertical signals are the participants 10, 13 and 18 when R>0.4.
Table 2. Correlation between imagery and observation of horizontal and vertical signals for each subject.
Figure 6 clarifies the low correlation value even if the imagery (Figure 6 (a)) and observation (Figure 6 (b)) signals have approximately the same behavior. In fact, since we don’t have time synchronization between imagery and observation (the subject do not imagine the grasping hand video at the same time when observed it), there are a slight time lag between both signals as shown in figure 6 which cause a decreasing in the correlation.
Figure 6. Horizontal signals during imagery (a) and observation (b).
In this study, we compare for 20 participants the horizontal and vertical EOG signals during both imagery and observation of a grasping hand video. We found that no significant difference between the average of EOG signals during imagery and observation. This means that imagery has approximately the same effect of observation on the eye movement. These results have been justified recently by Spivey and Geng . They indicated that analyses of eye movements modalities in tasks such as remembering object location or resolving geometrical problems, and demonstrated that subjects use saccadic eye movements to repeatedly return back to the physical scene .
Another research realized by Brandt and Stark in order to study the relationship between VMI and perception proved our results.
A further work with a similar kind of findings demonstrated that the sequence of fixations during imagery was very analogous to the sequence that happened when the subjects studied the pattern at the beginning .
By comparing the mean and standard deviation differences between the average of horizontal and vertical signals during imagery and observation tasks, we found that the potential of horizontal signals was higher than the vertical ones with a scale difference of order 100. This might be due to the movement of grasping hand directed over in the horizontal level.
Despite approximately the same behavior of the correlation of imagery and observation during horizontal and vertical signals, the low correlation may be due to the time asynchronization between imagery and observation. In fact, the subject do not imagine the grasp at the same time that he/she observed it, there are between both signals a slight time lag which cause a decreasing in the correlation.
Future work will avoid this time lag by using software tool which provides auditory signs indicating the beginning of imagery and observation involving the high synchronization between them.
Finally, the best choice between imagery and action observation is to use the action observation because all its parameters can be controlled more than imagery. It can avoid the time asynchronization and get more accurate results by using a feedback either in the neuro-rehabilitation of patients or the improvement of athletic performance. This proves the previous study of Gatti R. et al. showing that action observation attains better performance than imagery .
In this study, twenty participants (13 males and 7 females) visualized a video of grasping hand for 20 seconds are involved in this study to find the effect on eye movements. The recorded EOG signals of the imagery and observation showed that they have nearly the same effect on the eye movements. Eye movements are recorded during 3 trials of imagery and observation phases. This technique can be repeated with different motion videos and studying the comparison between the imagery of one action and the perception of the same action. The perception of another action is a future challenge and the open the door for new experiments.
This study could not have been realised without the kind support of the staff of AZM center for biotechnology research and its applications where this study has been accomplished.