UNRAVELLING
Critical Journal Seminar
We had our first seminar lesson of the semester. For me, I picked Critical Journal as one of my frameworks for my dissertation due to the intense nature of my experiments.
I learnt that a journal focuses on iterative and reflective making, supported by references to external examples/precedents and relevant literature concepts/theories. The outcomes from the making can shed valuable research insights surrounding my research topic.
Shu Min shared that there are 4 main components within a critical journal.
looks at what we are making, the tools and the processes.
Referencing to relevant concepts/theories from exisiting literature/precedents/research to frame our making.
Analysing and reflecting on the results/process of making to uncover findings.
Documenting outcomes, processes and research through sketches, videos, images or any other appropriate medium.
Research plan for Critical Journal
Following the lesson by Shu Min, I did a rough mindmap which details the process of my experimentation. I plan to introduce different sets of Multisensory Interfaces that consists of varying sets of sensory modalities to test the effectiveness of the implementation of Sensorial Principles within the interface design (I.E Varying stimuli congruency). The making stage will be supported by literatures on Sensorial Principles and Multisensory Interface Design to help guide my making. The outcomes will be reflected on and consolidated to make sense of the findings.
Overall, I found the seminar very informative and really like the framework of Critical Journaling. Outside of it being a robust framework for disseration writing, I find that it is also very informative for my own practice as a designer as well. The emphasis on referencing literature and precedents within the making process reminds me to ground my making in strong theoretical concepts and stay inspired by creative projects done by precedents.
Mindmapping Exercise (Pardon the handwriting)
In Thursday's class, we did a quick mindmapping exercise where we detailed our concepts into it’s respective pillars. We had the chance to walk around to glance at each other’s mindmap, and use stickers to highlight what was interesting. For me, many people placed their stickers on the concept of sensorial dominance and also the considerations of cultural and societal context in sensorial experiences. It was a interesting observation that many people were curious about the sensorial principle side of things, but not one which was inspiring to me. Personally, I wanted to focus more on the design aspects of Multisensory Interfaces and have sensorial principles as a supporting factor.
I feel that what worked for me here was how I got a clearer look at my research topic. By detailing out the different pillars and have my peers share their input, I obtain important feedback and perspectives. For example, Letitia shared with me that there were some technical terms (I.E Sensorial Princples) that was hard to understand. This made me realize that I should put in more effort to frame my research in a more approachable manner for easier understanding.
Andreas gave us a couple of helpful tools to help us narrow down our research direction. Out of all the task I had planned, this was the highest on my priority list as I really wanted to narrow down on my currently broad research topic.
At this pivotal point of amending my research direction, it becomes important for me to reflect on my own interests as a designer to really see how I can blend my them within my research to make the whole process more meaningful and enjoyable for me.
Street Photographs I took
-
-
Isidro's creative photography module kickstarted my journey into the world of photography. I fell in love with the deliberate and creative process of street photography and have ever since treat it as a fun and productive hobby of mine.
While photographs can be great storytelling visual mediums, It primarily depends on the singular sense of sight. This presents a interesting opportunity: Can Multisensory Interfaces augment Visual Photographs by introducing more interactive sensory experiences?
Be a Hawker - Children Museum of Singapore
-
Rain Room - Random International
I always enjoyed Multisensory Storytelling and how they tell stories. In most cases, It allows me to be really be in the shoes of the protagonist and establish a deeper level of understanding and empathy. Some of the previous case studies I discusssed are already great examples of this (Be a Hawker, Rain Room). For me, It is a personal goal to be able to recreate these immersive and magical storytelling experiences for people with creative coding!
It is already related to my topic because Multisensory Interfaces are one of the means to create Multisensory Storyelling experiences. However, it is noteworthy to look into how Multisensory Experiences help to enrich/facilitate storytelling (I.E how engaging different senses help tell stories better), on top of studying the design of the interface.
Photography are great visual storytelling mediums, but are typically reliant on visuals for its storytelling. This presents a opportunity for exploration to see how Multisensory Storytelling can fit in to augment the storytelling of visual photographs. This is a interesting exploration for me personally because I want to see how I can further people's enjoyment of visual-dependent storytelling mediums.
After that quick reflection on my interests, I head back to my current topic and see how It can work together. First off, I tried out the Funnel Technique, which seeks to narrow down broad ideas to a more focused topic.
Multisensory interfaces and its impact on human emotions and behaviors.
Narrowing down on type of Multisensory Interface
How Multisensory interfaces (Audio, Visual, Touch) affect human emotions and behavior.
Narrowing down on what type of Human Emotion/Behavior and Context
Augmenting Storytelling in Visual Arts: Multisensory Interfaces for Audience Engagement and Meaning Making
How can multisensory interfaces, specifically in the Audio-Visual-Touch domain, augment storytelling of conventional uni-sensory visual arts through facilitating the process of Audience Engagement and Meaning Making
From the General Idea, I started to first narrow down on the type of Multisensory Interface (MI). I chose Touch-Visual-Audio because these were interfaces which I felt had plenty of room for exploration for storytelling. To further focus my topic, I decided to contextualize the use of my MI for visual arts storytelling (inspired by my own reflection above) and niche down the two pillars of Human Emotions and Behavior to Meaning-Making and Engagement respectively as I feel that they important factors to look into that impacts how audiences understand the narratives in our multisensory experiences.
what's Meaning Making and Audience Engagement?
For the reader's understanding, I think it is important to help establish what are these two concepts that I just introduced in my new research direction.
Keyword - Audience Engagement
In the context of my research, Audience Engagement refers to how the design of the interface can enhance the capture of attention of audiences within the interface experience. There are two portions of attention I am looking at: (1) the initial capture of attention and (2) sustaining attention while using the interface.
The initial capture of attention is important to draw in participants to interact with the Multisensory Interface. One of the concepts that helps in this is Facilitatory Attentional Capture (FAC). FAC is a cognitive framework which proposes that cohesively integrated multisensory stimuli capture the attention of participants more effectively then single-sensory experiences. For instance, a painting of a boat in a thunderstorm, accompanied with sounds of thunder and rain, can draw the attention of audiences better.
Concept of FAC from Multisensory Enhancement of Attention by Lunn et al.
Concept of Emotional
Contextualizing derived from Emotional Responses to Multisensory... by Schreuder et al.
Other than capturing their attention, it is also important to sustain it. One of the concepts which serves as a effective tool is "Congruency of Stimuli" (a concept we discussed back in week 0). To recall, it simply looks at how well the different sensory stimuli in a multisensory experience work together. Congruent stimuli in a multisensory experience tend to generate a more positive emotional reponse in the participant, thus encouraging their continued usage. To paraphrase in really simple words, it just means that when the multisensory experiences feels cohesive, it makes it more enjoyable for us to sustain our attention on the interface.
Concept of Congruency of Stimuli derived from Emotional Responses to Multisensory... by Schreuder et al.
Keyword - Meaning Making
Meaning Making is a pillar that discusses how audiences perceive and makes sense of their experiences with the multisensory experience. Below are some relevant concepts in this pillar.
Within our day to day lives, we often rely on sensory experiences to help us perceive the things that are happening around us. This nature of information processing runs parallel with our experiences with multisensory interfaces as they often engage our different senses. Therefore, multisensory interfaces becomes a efficient way for its participants to understand the narratives of their experiences.
Concept of Nature of Information Processing from Multisensory Visual-Tactile by Yamauchi et al.
While Multisensory Interfaces makes it more effective for us to understand our experiences, it is only achievable if we are deliberate with the sensorial interactions we design it with. Sensory Modality Mapping is a concept by Product Designer Christine Park, where it proposes that we should consider the relevance of the included sensorial interaction within our design multisensory experiences. This is important as irrelevant sensorial interactions (such as touching a cube but the on screen visual is a sphere) can create a conflicting experience for the users and hinder their understanding and meaning making.
Concept of Sensory Modality Mapping derived from Designing Across Senses by Park and Alderman.