The Cross-Sensory Experience: How AI and the Human Brain Merge Senses in Unexpected Ways

• Humans naturally associate sensory stimuli like colours and shapes with tastes, shaping our perceptions of food and drink.
• Research shows that AI systems, including ChatGPT, mimic human sensory connections, offering potential for enhanced product design and marketing strategies.
• While AI has great potential, human creativity is essential to refine sensory experiences, ensuring they go beyond just automated suggestions.

The human brain often links sensory experiences in surprising ways, a technique that marketers have long leveraged in food packaging design. But what if artificial intelligence (AI) could replicate this sensory fusion? Can AI “taste” a colour or “hear” a flavour? Recent research by Carlos Velasco from BI Norwegian Business School suggests that, like humans, AI may blur the boundaries between sensory experiences. This capability offers valuable insights for industries such as marketing and design.

Sensory Blending: Humans and AIs Share a Common Thread

The idea that humans associate colours, shapes, and sounds with specific tastes may seem unusual at first, but it’s grounded in science. Our brains naturally create “cross-modal correspondences”—connections between different sensory inputs. For instance, the colour red is often linked to sweetness, while green may evoke a sense of sourness. Similarly, circular shapes are commonly associated with sweet flavours, while sharper shapes tend to be linked with bitterness or sourness. These associations are so ingrained that they shape our perceptions, even without our conscious awareness.

For years, studies have shown that colours like pink and red often trigger feelings of sweetness, while darker colours like brown and black are typically associated with bitterness. These patterns have been observed across cultures, with researchers in China, Malaysia, and India documenting similar sensory connections. A fascinating example comes from a study in Japan, where participants reported that bitter chocolate tasted sweeter when packaged in pink rather than black.

How AI Perceives and “Tastes” the World

Velasco and his colleagues were intrigued by the human brain’s ability to merge sensory inputs and wondered if artificial intelligence could mimic this phenomenon. Their research used generative AI technologies, including ChatGPT, to explore sensory associations. They prompted the AI to reflect on the relationships between colours and tastes, as well as shapes and flavours, similar to how they had done with human participants. The results were striking.

AI models, especially ChatGPT-4, showed remarkable alignment with the sensory connections humans commonly make. For instance, the AI linked colours like pink with sweetness and green with sourness, reflecting patterns often seen in human sensory experiences. This reinforced the idea that AI, much like the human brain, develops these associations based on patterns from its training data.

However, AI does not have sensory experiences in the same way humans do. Its responses are shaped by the data it’s trained on, which often reflects well-established scientific findings. For example, when asked about sensory correlations, the AI referred to studies on colour-taste correspondences, directly drawing from existing research.

The Science of Sensory Blending

Why do we associate certain colours or shapes with specific tastes? Charles Spence, a leading researcher in cross-modal perception at the University of Oxford, suggests that these correlations are likely a result of our evolutionary experiences. For instance, fruits often change colour from green (when sour) to red (when sweet), and humans may have internalised this pattern over time to help select the ripest fruit. Similarly, spherical shapes might evoke sweetness because they are often linked to safe, enjoyable experiences, while sharp, spiky shapes could trigger associations with danger or bitterness.

While these associations make evolutionary sense, their exact origins remain a subject of debate. Some researchers argue that these links might also be tied to the emotional responses these cues evoke, influencing our overall perception of taste.

How Artificial Intelligence Can Transform Marketing and Product Design

AI’s ability to mimic human sensory connections unlocks exciting possibilities for industries like marketing and product design. By leveraging AI’s skill in recognising sensory patterns, businesses can create more effective packaging, colours, and shapes that align with consumers’ subconscious preferences. For example, a company could design packaging that subtly suggests sweetness through its choice of colour and shape or use AI to develop innovative combinations that resonate with a broader audience.

However, as Velasco highlights, while AI can generate valuable insights, human creativity remains essential. AI-generated ideas might lack the depth and spontaneity of human intuition. The key is to combine AI’s data-driven insights with the inventive skills of human designers, leading to products that are both engaging and appealing.

The Future of Cross-Sensory AI

Looking ahead, researchers aim to explore whether AI can uncover new, untested sensory correlations that could revolutionise product development. By analysing vast amounts of data, companies could discover innovative ways to engage the senses, from novel flavour pairings to the perfect blend of music and beverages.

In parallel, AI’s ability to blend sensory experiences holds great promise for advertisers. By testing various combinations of colour, shape, and sound, AI could help create more precise and impactful marketing strategies. This could result in enhanced consumer engagement, more effective campaigns, and ultimately, higher revenue.

AI and Sensory Perception: A Journey Toward Synergy

While AI holds significant potential, it is not without its limitations. Occasionally, it may “hallucinate” or produce inaccurate responses that lack a basis in reality. Moreover, the sensory connections AI identifies may not carry the same depth or emotional resonance that human perception naturally provides.

Nevertheless, AI’s ability to recognise patterns and generate creative ideas makes it an invaluable asset for industries that depend on sensory experiences, such as food and beverage and entertainment. As AI continues to evolve, it could offer even more profound insights into how we perceive and interact with the world. Until then, both human intuition and AI capabilities will continue to guide us through the fascinating world of cross-sensory experiences.