Chroma Flow

Affective Interactive Installation
2024.04 - 2024.06

Flowing Conversations in Color allows individuals to seamlessly join ongoing discussions, picking up where others left off, while visually perceiving the emotional context of the conversation. By employing emotion detection algorithms, we aim to create an empathetic environment that fosters deeper connections and understanding among participants.

The model we used in our implementation is roberta-base-go_emotions on huggingface, which is a text classification model and has 28 different possibilities, we assigned each emotion different colors by referencing color symbolisms and color meanings.

The color mapping is initially implemented by changing the background color based on the emotion with the highest score of the user input, that is the color associated with the emotion with the highest score will be displayed. However, this approach does not showcase the complex emotions well. Thus, we implemented color blending, which maps the score of each emotion to the opacity of its associated color and blends all the colors altogether, and then the background color will change based on the combination of the emotion of the user input.

The color mapping is then integrated with the WebSocket so that it can be used by more people on different browsers or devices. The user input will be sent to the server when it’s submitted, and the query function will then be called to get the emotional analysis result from the model. After that, the message and result will be sent to other available clients. When the client receives the data from the server, it will do the color mapping and display the color associated with the message

Code available on GitHub.

Tools: JavaScript, API, WebSocket

In collaboration with: Riva Wang, Seba Alabdullatif