How deep is your love for deepfakes? We ask marketers

Hrithik Roshan is a special forces agent in a bulletproof vest who is seated in a black van, holding a gun. Along with him are several other agents. Suddenly, they hear a knock on the door. As soon as they open it, the group point their guns at the intruder - a Zomato delivery man who is as shocked as they.

As one of them asks, "Yeh kisne mangaaya?" Roshan replies, “Maine! Burger ka mann kiya. Sirsa main hain, toh Grillmasters se mangaliya.” But the name of the dish, the restaurant and the city and locality changes every time depending on the ad’s viewer - and their phone’s GPS.

This is Zomato’s hyper-localised campaign that claims to use deepfake technology to deliver personalized messages to audiences.

At the 2022 Cannes Lions International Festival of Creativity, another hyper-localised campaign - ‘Not Just a Cadbury ad’, by Ogilvy India and Wavemaker, which featured actor Shah Rukh Khan, brought home a Titanium Lion. The brand made use of machine learning to recreate the actor’s face and voice to highlight the names of various local stores in the ad, extending help to hundreds of small business owners who were deeply affected by the lockdown.

Pepsi and Ageas Life Insurance have also used the technology to create a much younger Salman Khan and Sachin Tendulkar in their commercials.

Brands are playing and experimenting with new technologies to give audiences something new and to break through the clutter.

But what is the technology and how does it work?

Rephrase.ai’s founder Ashray Malhotra refers to the technology driving these campaigns as ‘synthetic media’. “Every person thinks synthetic media is deepfake. If someone even generates Shah Rukh Khan’s voice, that gets labelled as deepfake voice. But, that is not how it began,” says Malhotra, whose company uses artificial intelligence technology to create hyper-personalized videos for brands and businesses.

In fact, deepfake tech turned into a menace and continues to be so.

Deepfake technology, which is a combination of two concepts ‘deep learning’ and ‘fake’, was initially used for creating pornographic representations of women. Today, this technology is also used to fool recruiters and voters.

But it has other useful applications. Marketers have begun using the technology to help brands communicate and reach their audiences in what they hope are more engaging ways.

How it works

Speaking about the Zomato campaign featuring Hrithik Roshan, the company's spokesperson tells us, “Business owners—big and small—dream of having Bollywood celebrities endorse them. We wanted to give a platform to the restaurant partners that have helped Zomato reach over 1000 cities and have helped serve millions of consumers.”

Four months in the making, the company shares, “AI technology married with some smart targeting on Google and Facebook helped us land a relevant message for the viewers at a mass scale.”

Zomato partnered with Gan Studio, a startup with focus on state of art algorithms.

This campaign was also advertising agency Enormous’ first experience with experimenting with AI-driven technology in crafting a narrative around the brand and its proposition. The agency’s managing partner Ashish Khazanchi mentions that this technology is of great use when the distribution of advertisements is through digital.

“When it comes to distribution, deepfake does make for a lot of interactivity, and a lot of personalisation of assets. It works well when you are trying to create personalized content for people. Therefore, you can have any number of output which would work very well for a hyperlocal market, or a pin pointed kind of a distribution strategy,” explains Khazanchi.

Saumya Rathor, Category Lead, Pepsi Cola, PepsiCo India, agrees with Khazanchi. She says, "With the acceleration of a digital forward communications approach, audiences are turning towards a technologically advanced way of consuming information."

Malhotra looks at it as an evolution of marketing. “The end goal of a marketer is to talk to their audience in a personalized manner, and in a video format which is the most effective. Hence, personalization plus videos is what synthetic media now enables, and one can see more and more marketers make use of this technology,” he says.

Bengaluru-based Rephrase.ai builds generative AI tools to ease video creation. Its clients include Cadbury, Castrol, Raymond, online teaching platform Classplus, edtech startup Sunstone, and learning platform White Hat Jr as clients.

Rephrase.ai has also worked with CXOs like Titan Company’s Ajoy Chawla, CoinDCX’s Minal Thukral, DBS Singapore’s Royce Teo, Becton Dickinson’s Pavan Mocherla, Bagic’s Tapan Singhel, Johnson & Johnson Vision’s Sherri Ferstler, and Meesho’s Vidit Aatrey.

Inclusion of deepfake in advertising: Critical stages

Khazanchi explains that the critical stage is always finding the right kind of idea where technology would be of use.

In the case of Zomato, it made sense to the agency due to specific restaurants, and specific dishes in specific parts of the country. He says, “You need to shoot in a static block where you do not need too many changes in the backdrop. You will want the person to be stationed in that particular place. There is enough data captured using a celebrity so that all we have is the information on the way he or she would speak or intonate any particular part of him or her.”

Speaking of the Pepsi campaign, which recreated a scene from the 1994 Bollywood super hit Hum Aapke Hain Kaun, Rathor says, "To start with ensuring that we got the right script to do justice to an iconic character of Indian cinema was a task. We had to ensure that while we make the film quirky, witty, fun, it had to be respectful to the character. At the same time, there is so much love for Prem (Salman Khan’s character in Hum Aapke Hain Kaun) that we knew there would always be high expectations from the audiences while recreating this avatar."

That's the tale. Now, what about tech?

Rephrase.ai’s Malhotra recalls the three steps involved while rolling out a campaign involving deepfake technology. Step one: creating a clone. A video is recorded for a few minutes. Then the video is handed over to a machine learning algorithm, which trains for a few days, and creates a clone of the person’s face and voice.

Step two is integration and testing. Malhotra explains, “This process is different for every company. If you want to integrate with your customised relationship management (CRM), which means, as soon as a new customer signs up, they get a personalised video from, say the CEO of the company, we need to integrate the AI generated clone in the middle of the pipeline.”

Step three is deployment, where it is decided that this is a live process. Anybody can type in a name and a video gets generated which can be sent via any medium. “The timeline for the creation of a campaign with the inclusion of this technology on an average takes two weeks. For a concerned campaign, we have done it in as short as five to six days,” adds Malhotra.

For ‘Not Just a Cadbury ad’, it took two weeks and Rephrase.ai created five different avatars of Khan. This was done parallelly because he was in five different locations. Plus, Google Maps API was used to highlight the names of the shops. Malhotra mentions, “The initial intention was to create 20,000 videos. But, we went as high as 1,40,000 videos in a span of 10 days.”

Given Salman Khan's popularity, Pepsi's ad was also well received by audiences and "got social media brimming with conversations through media and memes on the return of Salman's iconic character 'Prem'," adds Rathor.

Putting in checks and balances

Since Rephrase.ai works with well-known personalities, it recommends companies add the ‘Assured by Rephrase.ai’ logo. Malhotra says, “The logo assures brands that even if something goes wrong, they can say that the video was AI generated, and not of the actual celebrity or executive.”

In the case of celebrities, every video is created with written consent. Malhotra says, ‘Even though brands have the ability to create new videos, they don’t have the authority until and unless they obtain consent. When the final video—besides the written scripts—is created, they are also sent to the celebrity for approval to ensure that they are comfortable with the final video going out. This means that in every part of the process, they are in control of what goes out.”

In the case of customers, ‘Assured by Rephrase.ai’ logo is added to highlight that the video was created with permission and consent of the person in the video.

Can’t fake creativity

For agencies and creators, there’s also the additional challenge of how to keep refreshing the use of the technology in campaigns.

Khazanchi says the challenge will always be creativity, and not technology. “It is not that challenging to shoot because the basics are simple. Now what to do, and who to do it for, that is where the challenge lies,” he adds.

The managing partner says, “The bigger challenge would also be crafting a new narrative so that it always stays fresh. Since three to four brands have already experimented with this technology, you have to find a way of making use of it which makes a lot of sense and is still new.”

The point is to find a way to marry creativity and technology to create clutter-breaking work, and not let technology consume creativity.