Blog

Emotion is a universal language, difficult to decipher, and very particular. What if brands today were able to know how a consumer feels when he/she sees an advertisement or enters a store? How would this change marketing? Daxue Consulting has dived into the field of emotional data in China and gives you all the keys to leverage it in your strategy by collaborating with the best-specialized players in China.

Emotional data in China: What does that mean?

Over the past decade, the world of research has seriously examined emotional data. According to the Tractica research institute, the market for emotional data is expected to grow from $123 million in 2018 to $3.8 billion in 2025.

What about emotional data in China?

Putting humans at the center of everything

Emotional data is data that measures the emotions and feelings of someone. It is also known as “Feel Data,” a term popularized by the agency Datakalab, a French start-up specializing in brain tech that aggregates feel data to stand in the distance and obtain valuable insights. It is estimated that they are now the future of research. The aim is to capture consumers’ emotions about a product, atmosphere, or advertisement and use it as marketing data.

Datakalab, was interviewed by Daxue Consulting to get insight for this article; they give the following definition of feel data:

“Feel data is different from the data brands are used to be working with, which are transactional data such as click rate, average basket, number of visits. With Feel data, we are able to give the attention rate, the emotional engagement and the positive emotions. These are crucial metrics because capturing attention today is the ultimate goal of any brand.”

Thus it is possible to detect emotional connections through data when an audience is in front of a content to modify or personalize it later.

Emotional data comes directly from the world of neuroscience and more specifically from neuromarketing. Neuromarketing takes into account advances in neuroscience that aim to observe and measure brain and nervous system reactions and apply them to the consumer. It is in this sense that feel data puts the human being back at the heart of research.

Neuromarketing, for example, has shown that the context of an advertising message strongly influences consumer reactions.

Measuring emotional data is becoming more and more critical for AI in China

In the advertising field, there can be no memorization without engagement, just as there can be no engagement without emotion. This is the biggest challenge all brands are facing: to reach the hearts of consumers.

We now know that humans react first and think second. Indeed, according to the famous American researcher Milton Erikson, 90% of the stimuli that impact us every day are treated by the unconscious. This means that declarative data are no longer sufficient. So there is a huge gap between what people say, what they think they are doing, and what they are actually doing.

That’s why using feel data in China can be the perfect way to really understand the consumer in 2019.

In China, digital advertising is particularly confronted with this problem due to intrusive ads that interfere with Internet users’ navigation. This has resulted in the growing number of users using Adblock and the decision of China’s State Administration for Industry and Commerce to ban them.

In addition, with the democratization of the smartphone, companies working in emotional data in China have excellent support to develop these new emotional recognition technologies.

According to data from Statista, smartphone penetration in China had already reached 50% by 2018 and will reach around 55% by 2020. Since phones are carried everywhere and hardly left alone these days, it is an excellent way to increase the accuracy of the emotional measurement.

Neuromarketing in China, not-so-new marketing concept

Although all this may seem futuristic, it is on the agenda. Many companies have already been using emotional recognition technologies for years. Moreover, we can consider that emotional data and neuromarketing in China are not that recent.

A few years ago, Weibo took a step forward in neuromarketing and emotional data in China with the reaction buttons: joy, sadness, surprise, laugh, and anger. Even if this data only remained declaratory, they still promised to measure the feelings of Internet users directly. Information that the social media giant can use to personalize each user’s feed according to their preferences.

Everything about experiential retail that tries to make the customer feel good in a purchasing context is also a form of emotional data collection. The purpose of sensory marketing and the use of New retail in China is to adapt to the emotions and feelings of customers to promote shopping.

How does emotional data in China work?

There are many ways to measure emotional data in China. Thanks to different technologies and applications, which can be used independently or at the same time to have more accurate detection.

Different types of information can be analyzed:

• Voice modulations

This consists in deciphering the emotions transmitted by the customers’ voices through vibrations. Some software would be able to detect up to 5 emotions in the vibrations of the human voice such as anxiety, joy, or sadness.

In China, Huawei launched a voice assistant in 2013 and is now researching to add emotion-based features to it. This entry into the world of artificial intelligence in China would allow Huawei to refine the relevance of its voice assistant, used by more than 110 million users on a daily basis.

The idea is that a specific program in the virtual personal assistant would be able to detect a user’s mood and respond accordingly.

• Pupil movements

This method is part of eye tracking. It measures eye movements, pupil dilation, fixation points, and fixation times of the consumer’s gaze.

Some 30,000 data per second can be processed with eye tracking.

“Many companies in China’s market, both domestic and international, have leveraged the eye-tracking technology to do shelf testing, ads design and many other commercial research, which all benefit from it.” said a salesperson in Tobii China, a branch of an international eye-tracking firm focused on the Chinese market.

Daxue Consulting leverages the eye tracking methodology to study Chinese consumers. It has especially been a key method for international brands looking at testing and validating their packaging adaptation or their online China-focused user interface design. If you’re interested in this research method, you can directly contact us.

• Facial microexpressions

Facial coding or facial decoding is probably the most widely used method when it comes to emotional data in China. Theoretically developed by the researcher Paul Ekman as the Facial Action Coding System (FACS), it measures emotions by observing the reactions expressed by a consumer’s face.

The combination of what Ekman called Action Units (AUs) defines which feeling or emotion a person feels. These microexpressions are often difficult to perceive to the naked eye and required the development of specific technologies and algorithms.

Recently, the Hangzhou No. 11 High School has invested in facial coding technology to measure students’ emotions during class. Several cameras have been installed in the classrooms that will film the students at each stage of the lesson, reading, writing, answering questions.

School officials explained that the aim was to measure the level of stress and satisfaction experienced during school hours. This will optimize the way teachers teach and the learning capacity of students. A first very good example of feel data used in China, outside the field of marketing.

• Heartbeats, body temperature, and blood pressure

Finally, “health” data can be measured to get an idea of the consumer’s condition at a given time. This method is particularly effective in measuring the level of stress or calmness of consumers in a store. Many companies specialized in emotional data in China have developed connected bracelets that measure heartbeats, body temperature, or adrenaline levels.

All this is often combined with sociological information to refine the user’s profile and optimize advertising targeting.

Thus, by combining data from facial expressions with data from the brain, body, and sociology, human emotions can be measured and analyzed in a specific context. In China, thanks to these data, deep learning algorithms now interpret the emotional state of consumers and immediately adjust responses according to perceived feelings.

How can businesses leverage emotional data in China?

According to Tractica’s analysis, the fields where emotional data is most commonly used are customer experience, market research, and customer service.

Improve and customize the user and customer experience thanks to emotional data in China

The customer experience field in which emotional data is most commonly used is the shopping experience, online or offline.

By putting temperature sensors or facial coding cameras in stores, retailers can know how customers feel when entering a store or looking at a product. It also allows you to guess which customers wants the help of a salesperson and which customers would rather be left alone

In China, the chain of stores Suning collects data as part of its New Retail approach. Shelves record the frequency with which clients look at or touch a product and their emotions at that time. This allows Suning to know its customers well because they also collect demographic data such as age and gender.

For online experience, it works the same way. Emotional data can be collected with eye-tracking software that also measures pupil movements and facial tightenings. We can, therefore, improve navigation on a website by analyzing the most complicated moments of navigation or by discovering the untapped areas on a website.

It also allows you to customize the user experience according to everyone’s browsing preferences.

Enhance customer relationship

For Customer Relationship Management, emotional data is a formidable weapon. Knowing if a customer is satisfied, enthusiastic, disappointed, or even angry after an experience is the best way to get customer feedback.

So you can use feel data in China to improve service in your call centers and sales departments.

Moreover, combining sentiment analysis with existing CRM data can provide your team with a holistic view of customers to provide personalized retargeting ad campaigns.

How feel data in China can help you with management and human resources

Emotional data in China are also used in the fields of management or HR. As for human resources, emotional data is used by recruiters to analyze the expressions of potential job candidates. Are you claiming to be highly qualified for the position? Algorithms can detect some signs of self-confidence or lie. Companies in China begin to record interviews, and then the videos are analyzed by the facial coding software.

The problem with this is the discrimination and possible biases that the software can induce. If you want to use feel data in China for human resources, use blind scores, which means that the software will not recognize the sex, gender, race or age of the candidates to avoid any bias.

In 2014, State Grid Zhejiang Electric Power in the southeast city of Hangzhou began to leverage emotional data in China. The company has developed an emotional barometer that measures the emotional level of employees and internal managers. This aimed at improving happiness at work, reducing stress, and thus increase the level of employee involvement.

In these factories, employees are filmed. This also aims at preventing any incidents if an employee is tired or distracted.

The managers told the newspaper South China Morning Post that the aim was also to see if management was effective. If half of the employees feel anxious when they go to work, it is because of poor management which needs to be changed.

On the same principle, the caps of train drivers between Beijing and Shanghai have now sensors that measure the tension and attention of drivers to avoid an accident.

Leverage artificial intelligence in China for your research: test your content!

The use of emotional data in China and around the world is, therefore, completely disrupting the field of research. This is the end of simple satisfaction questionnaires that involve several problems when used alone: data is not collected directly, declarative data is always subject to the will of the respondents, and questions can be biased.

Datakalab used the following example: they measured the emotions of travelers in a French train station. When analyzing the data, they realized that travelers felt a high-stress level when they entered the toilets. When they asked people why they felt that way, they answered: “because the bathrooms were not free”. In a declarative questionnaire, if the question asked was “do the bathrooms are clean”, everyone would have been answering ‘yes’ and they could not have discovered this insight.

This proves two fundamental things:

• If emotion is not transformed at the moment it is experienced, we lose relevancy
• To measure customer experience, it is difficult to use a simple questionnaire because questions can miss the real problem

Claire Latourre, project manager at Datakalab, explained:

“Emotional data can be combined with all kinds of transactional data. Our data can be used alone, but it can become the ultimate data if it is smartly combined with declarative data, for example.”

For this reason, you can even use feel data in China during focus groups in addition to collecting simple declarative data.

Finally, to test your advertising content, feel data is the ideal solution. Not only will you be able to measure the audience’s attention and satisfaction level, but you will also be able to identify the exact moment when the client is laughing, sad, or angry. In an advertisement, we will, therefore, be able to see which precise punchline makes the audience laugh, or which image touches them.

For the creative industry, it’s a real revolution!

How to find the right neuromarketing partner in China?

Daxue Consulting has compiled a list of the best neuromarketing partners in China. For more information and advice on how to choose the right partner for your project, you can contact our experts.

Facethink – 德麟科技

FaceThink offers one of the best facial expression analysis technique in the field of brain tech in China. It enables the understanding of customers’ emotion and leads to better business decision making.

They offer three main services: face detection, eye tracking, and emotional data analysis. They claim to have one of the best algorithms in the world and are specialized in audience analysis. They analyze user’s emotional feedback when watching your content in direct life.

Seeta Tech – 中科视拓

Seesatech is a neuromarketing company in China based in Beijing, which provides technology, talent, and knowledge services in the area of face recognition, computer vision, and machine learning. It is specialized in research and innovation in artificial intelligence in China. Best known for his SEO, Prof. SHAN Shiguang, who has more than 20 years of experience in face recognition and brain tech in China. He even received the Annual Young Scientist Award of CCF 2015.

The company has developed a facial recognition technology, called Seeta Face Detection. It can accurately locate the human face in the image and analyze its features, including emotional data.

They promise to be able to pick out all kinds of faces in complicated environments, including side faces, emotions, and even partially-occluded faces. The accuracy is supposed to have reached a high standard in the world.  

Seeta Tech won the 1st Place Award in ACM IMCI 2014 Emotion Recognition in The Wild Challenge in 2014.

Vionvision – 技术中心

Vionvision or Viontech is a company founded in 2006, specializing in the research and development of core algorithms and products of artificial intelligence in China. Located in Beijing, Zhongguancun Science Park, the teams are composed of doctors and masters from institutions including Tsinghua University and Chinese Academy of Sciences.

VionTech has developed a series of core algorithms of image recognition and high-performance AI hardware platforms.

In terms of emotional data, they can monitor the passenger flow and analyze face and behavior.

Their technology can be used for passenger flow detection and analysis in various announcement places like shopping malls, stores, transportation, streets, etc.  

It allows real-time face capture, face image retrieval, face clustering, age/gender analysis, and of course, emotional analysis.  

Authenmetric – 中科奥森

Authenmetric is a Chinese company with the focus of face recognition. They accumulate decades of face image processing research and development experience and are able to quickly detect emotional connections through data thanks to their continuous iterative algorithm.

Translation:

Core Technology

Specialized in machine vision research and artificial intelligence technology with face recognition as the core

Arcsoft

Arcsoft is the leading algorithm in the computer vision industry. Their solution is well known around the world in all types of industries such as smart cars, smart homes, Smart retail, Internet video, etc.

Since the beginning, ArcSoft has been focusing on the technology and software applications related to imaging and computer vision. Arcsoft is a potential good neuromarketing partner in China for your business because, even if it is not their core business, teams have also developed race, age, gender, and facial expression recognition capabilities, in other words, emotional data collection.

Their technology can be leveraged in all kinds of situations and can make the difference between the change of expressions due to age, movement, or light, which is quite tricky to apply.

Thus, research possibilities are strongly increased thanks to emotional data in China. However, this raises two questions:

Is emotional data the end of personal privacy? In China, where the laws for personal data are a bit less strict than in Europe, consumers may feel annoyed to be analyzed in this way. According to a study carried out by CCTV and Tencent Research (AI Lab) in 2018, 76.3% of respondents believe that artificial intelligence in China can be a violation of privacy.

Finally, critics believe that it is impossible to decipher human emotions even if they are universal because each human reacts differently, and the signs are too complex.

According to Claire Latourre, despite the revolution initiated by Feel data, human beings will always remain extremely difficult to decipher:

« Today, algorithms are trained on million of data, which allows us to reach a certain truth, but human beings remain unique. It should be kept in mind that we are only working on global trends, but they are reliable and precise enough to draw strategic conclusions. »

Author: Steffi Noël

Our report on AI in China: