Getting to Know Neuromarketing: Facial Coding

Happiness and joy or apprehension and disgust: imagine knowing exactly how a customer feels about a moment to moment commercial, website or smartphone device, when they perceive it. It's not facial encryption, it's not science fiction.

Happiness and joy or apprehension and disgust: imagine knowing exactly how a customer feels about a moment to moment commercial, website or smartphone device, when they perceive it. It's not facial encryption, it's not science fiction.

How does it work?

Analysis of facial expression as the Facial Action Coding System was originally developed by Carl-Herman Hjortsjö in the late 60's and early 70's, and pioneered by Paul Ekman and Wallace Friesen in the 70's, where facial expressions are studied, primarily by changes in different categories of facial muscles, to determine the emotional response of an person.

Within this system, facial muscle contractions or relaxations are broken down, defined by a number, into "action units" (AU). Each unit reflects one or more facial muscular activation. For example, AU 0 is a neutral face and AU 1 is the "increase of the inner part of the eyebrows." Hence, an expression can correspond to several units of action. The expression intensity is noted on a scale from A to E, and the highest intensity is E. For example, in the inner part of the eyebrows 1C shows a marked or pronounced increase, but not very extreme.

The emotional theory of facial feedback is the theory of how facial expressions are related to emotional experiences. Two influential supporters, Charles Darwin and William James, both recognized that physiological responses frequently had a strong effect on emotional elicitation, rather than merely emotional outcome. Supporters of this theory have indicated that emotions are directly related to changes in facial muscles; for example , people who are compelled to smile positively at a social activity are likely to have a better experience at the event than they would if they had smiled or had a neutral expression of the face.

In Facial Expression Analysis over the past four decades, a wide body of work has been conducted, contributing to various fields such as psychology, sociology, business , marketing, and crime.

How does this look?

By using a high quality picture, facial expressions are easily recorded with today. For example, a participant will perform a set task on a website, while a webcam will monitor their facial expressions. Using advanced software, the video will then be analyzed frame by frame, usually in tandem with documenting what the person witnessed on-screen.

What feelings the participant experienced at some point during the task can then be calculated. For example, if your participant struggled when navigating your website at key moments of a user trip, you may want to reconsider the particular elements of your user interface because users might have trouble or find certain aspects rather confusing.

Some Pros and Cons

A clear, front-on recording of the participant 's face is required for accurate facial coding, with good and even lighting (a change in lighting can have a dramatic effect on the readings). This can make applying in real-world settings where the individual moves freely, such as during a shopper trip evaluation, difficult. Some participants just don't get as upset as others. They obviously have a dead pan-facial expression that gives nothing away, just like a professional poker player covering his winning hand, thereby not providing any valuable data to researchers. The software that is being used at this stage is often prone to error and may lack the meaning or more subtle emotional signals in the data that it sets out. For these reasons (and more) facial coding has been shown to be the least effective in predicting future customer behavior, out of all the neuromarketing techniques available.

There are also good aspects to it though. For example, facial coding experiments are non-invasive, and because a good camera is all that is required, it is possible to collect data from more than one participant at a time. While an specialist is ideal for studies of Complex Facial Expression Analysis , it is possible to conduct less rigorous data collection by just about anyone with limited preparation. Given that data collection is also largely automated, processing time is relatively high, while the cost is relatively low compared to other methods of neuromarketing.

Even facial expressions are similar-a smile means the same thing in humans almost everywhere-and cultural distinctions do not need to be taken into account. Overall, facial coding has moderate reliability, and is a reliable indicator of emotional expression; providing conveyed emotional content and form in particular.

A Fun Example

Disney also used AI and facial recognition to capture data from 400 people of audience sizes. They recorded the facial expressions of the audience-and thus their emotional reactions-through nine of their films. The study included 150 showings and ultimately yielded a remarkable 16 million facial landmarks from 3179 members of the audience.

With this information fed into an AI, Disney can now predict in the first ten minutes what the response of an audience member will be to the key points in a film based on their facial expressions. It would be a powerful resource for assessing the success of their latest films in reaching the right emotional beats. This enhances their ability to tell stories, and consequently the quality of their feature films.

Link for the Disney Example - https://mashable.com/2017/07/27/disney-facial-recognition-prediction-movies/?europe=true