Sergio C. Torres (1) and Susana Ruiz Fernández (2)
(1) Multimodal Interaction Lab. Leibniz Institut für Wissensmedien (IWM), Tübingen, Germany.
(2) Brandenburg University of Technology Cottbus-Senftenberg (BTU), Senftenberg, Germany.
(cc) Sergio C. Torres.
Interaction gestures with our electronic devices, such as swiping an image on the screen, can influence our emotional experience. In this article we introduce the Spatial Affective Interaction framework, which aims to explain how our mind integrates (1) associations between space and emotional valence, (2) our tendencies to move towards or away from digital content, and (3) the impact of our hands on the processing of digital content. This research opens doors to closer technology design, seeking to create interactive experiences that are not only responsive to our gestures, but also emotionally meaningful.
As touch devices, such as smartphones and digital tablets, become ubiquitous in our everyday lives, it is increasingly evident that the actions of touching and moving digital content are also becoming as natural as our physical interaction with real-world objects. In this sense, the influence of such interactions on our perception of the content we manipulate and on our affective reactions has become a very relevant aspect for device design. This issue has stimulated the development of a research model that we will refer to as Spatial Affective Interaction (SAI; cf. Torres et al., 2024). Its main goal is to explain the fundamental role of physical interactions with digital content in our emotional experience.
The core of the SAI is based on three main elements: (1) mental associations between space and emotional valence; (2) approach-avoidance tendencies; and (3) perceptual biases related to hand proximity to objects (see Figure 1). We describe these elements in more detail below.
Figure 1. Ilustration of the Spatial Affective Interaction model (SAI).
The first pillar within the SAI refers to associations between space (especially the vertical and horizontal axes) and emotional valence. Bodily experiences that are recurrently associated with emotional states, such as jumping for joy or adopting a slumped posture, may give rise to mental associations such as up-positive and down-negative (Meier & Robinson, 2004). Recent studies suggest that these associations may also play an important role in digital interaction environments. For example, interacting with pleasant images at the top of a touchscreen and unpleasant images at the bottom may result in a better user experience (Cervera-Torres et al., 2018).
Horizontally, it has also been proposed that actions performed with the dominant hand are often perceived as more pleasant, probably because they are easier to execute than those performed with the non-dominant hand. Consequently, right-handers tend to form right-positive and left-negative mental associations and left-handers tend to form inverse associations (Casasanto, 2009; Santiago, 2009). Such associations have been observed in different countries and cultures (Yamada et al., 2024). Following this line of reasoning, it has been shown that interacting with pleasant emotional images on the right side of a screen results in a more pleasurable experience when these actions are performed with the right dominant hand. In contrast, interacting with unpleasant emotional images on the left side of the same screen may result in a less pleasant experience when these actions are performed with the left non-dominant hand (Cervera Torres et al., 2020).
These findings not only highlight the power of manual interactions to shape emotional sensations, but also reveal how user experience in digital environments can be influenced by the relationship between space and emotional valence.
A second pillar of the SAI framework refers to approach and avoidance mechanisms that reflect our natural tendencies to move towards positive stimuli and away from negative stimuli. These tendencies have also been clearly observed in digital interaction environments. For example, Cervera-Torres et al. (2021) contrasted the sensations produced by emotionally charged images when these images were observed without any interaction involved or when participants used their hand on a touchscreen to move toward or away the pictures, in relation to their body. It was observed that, when moving the images with pleasant content toward the body, participants experienced more positive feelings. Furthermore, moving unpleasant images away attenuated the negative feelings produced by the images. These results reinforce the idea that physical interactions with digital content can causally modify our emotional feelings. In the context of digital technologies, where these gestures are an integral part of the user experience, these effects have important implications for the design of more intuitive and emotionally resonant interactive environments.
The findings reported up to this point may be explained by a third pillar of the SAI. Visuospatial biases arising from the proximity of the hand to visual objects have been reported, which is particularly relevant in a world increasingly dominated by tactile devices. For example, research suggests that, compared to simple observation, the act of moving or manipulating digital images directly with the hand intensifies our attention to those stimuli, which in turn may influence how we evaluate their content (Brucker et al., 2021).
In short, the SAI framework provides a novel perspective for investigating how interactive digital environments can be designed to influence users’ emotions more effectively. Moreover, this approach may be key in technologies such as virtual or augmented reality, where manual interactions lead to more immersive experiences. As we move towards a more digital and physical interaction-based future with the virtual, it is critical to continue to explore how these interactions can be designed to create more meaningful and emotionally resonant
References
Brucker, B., Brömme, R., Ehrmann, A., Edelmann, J., & Gerjets, P. (2021). Touching digital objects directly on multi-touch devices fosters learning about visual contents. Computers in Human Behavior, 119, 106708.
Casasanto, D. (2009). Embodiment of abstract concepts: Good and bad in right- and left-handers. Journal of Experimental Psychology: General, 138, 351–367.
Cervera Torres, S., Ruiz Fernández, S., Lachmair, M., & Gerjets, P. (2020). Coding valence in touchscreen interactions: Hand dominance and lateral movement influence valence appraisals of emotional pictures. Psychological Research, 84, 23–31.
Cervera-Torres, S., Fernández, S. R., Lachmair, M., & Gerjets, P. (2018). Valence-space associations in touchscreen interactions: Valence match between emotional pictures and their vertical touch location leads to pictures’ positive evaluation. PLOS ONE, 13(7), e0199972.
Cervera-Torres, S., Ruiz Fernández, S., Lachmair, M., Riekert, M., & Gerjets, P. (2021). Altering emotions near the hand: Approach–avoidance swipe interactions modulate the perceived valence of emotional pictures. Emotion, 21, 220–225.
Meier, B. P., & Robinson, M. D. (2004). Why the sunny side is up: Associations between affect and vertical position. Psychological Science, 15, 243–247.
Torres, S. C., Ruiz Fernández, S., & Gerjets, P. (2024). Positive-right and negative-left: Affective spatialization by digital “grab” interactions. En C. Stephanidis, M. Antona, S. Ntoa, & G. Salvendy (Eds.), HCI International 2024 Posters (pp. 215–223). Springer Nature Switzerland.
Yamada, Y., et al. (2024). Where the ‘bad’ and the ‘good’ go: A multi-lab direct replication report of Casasanto (2009, Experiment 1). Memory & Cognition. https://doi.org/10.3758/s13421-024-01637-1
Manuscript received on October 26th, 2024.
Accepted on November 28th, 2024.
This is the English version of
Torres, S. C., y Ruiz Fernández, S. (2025). Interacciones digitales al alcance de la mano: ¿influyen en nuestras emociones? Ciencia Cognitiva, 19:2, 30-33.
