Indian Elections; deep fakes, false memes, and misinformation galore

Indian Elections; deep fakes, false memes, and misinformation galore
Resurrecting dead politicians in election campaigns online may sound unbelievable, but in the Indian elections it seems that almost anything is possible.  From deep fakes and AI meme wars to bogus audio messages, Bangalore based investigative journalist Nilesh Christopher has been reporting on them all.  He’s on the show to tell us how voters are being manipulated online.

A quarter of the world has experienced election internet censorship this year
Elections are happening in 60 countries around the world this year. Internet outages and restrictions were unfortunately to be expected in some countries. New analysis shows over 90% of voters in elections that have taken place so far this year live in countries where internet censorship has increased.  These include India, Indonesia, Pakistan, Bangladesh, Russia, Taiwan, Senegal, Azerbaijan, Comoros and The Maldives. Top10VPN’s Head of Research Simon Migliano tells us more.

The programme is presented by Gareth Mitchell and the studio expert is Angelica Mari.

Indian elections and deep fakes
A quarter of the world has faced election internet censorship in 2024

Chapters

• 2:48 Indian Elections; deep fakes, false memes, and misinformation galore
• 16:20 A quarter of the world has experienced election internet censorship this year

Transcript

00:00:00 Gareth Mitchell 

Hello, I'm Gareth. Welcome along to Somewhere on Earth. We're right here in our studio in London or should I say studios in London. I'll leave it at that. 

00:00:17 Gareth Mitchell 

And with us today for some experting is Angelica Mari, journalist and person extraordinaire Angelica Mari joining us from Sao Paulo. How are you, Angelica? I gather climate change is a little bit on the agenda with you this week. 

00:00:32 Angelica Mari 

Yeah, that's right. Lovely to be with you again, Gareth. It's always a pleasure. Yes. Unfortunately, the the news agenda here and all the conversations have been about climate change because of the climate disaster in the south of the country. 

00:00:52 Angelica Mari 

There are lots of tech stories still unfolding. The main one about connectivity or the lack of connectivity. There are lots of initiatives around platforms being created to connect the people that have been affected by the floods with people who can donate. 

00:01:17 Angelica Mari 

Also, all sorts of digital initiatives to support crisis management. And I've been quite focused on that, but very sad situation, but the in things in in situations like this, the resourcefulness of the techies and the the speed in which they can put things together gives us a little bit of hope. 

00:01:44 Gareth Mitchell 

Alright, Angelica, thank you for that. And coming up today. 

00:01:54 Gareth Mitchell 

Now, is this a voice clone or could this be the real Gareth Mitchell? Hey, who cares? I'm only a podcaster, but in this year of elections leaders and their opponents are being ritually faked, adding memes and fun and satire to the often protracted seemingly endless election campaigns, but we're focusing on the massive democracy of India today and assessing the 

00:02:18 Gareth Mitchell 

role and potential harms from deep fakes and voice clones. Other elections, of course, are available. Many more of them around the world. In fact, new figures say that 1/4 of the world has experienced election censorship so far in 2024. So we'll explore that figure as well and hear about the 10 countries that are behind that figure. It's all right here on the Somewhere on Earth podcast. 

00:02:49 Gareth Mitchell 

So AI powered absurdity continues to break out online amid this year's Indian elections. Chatbots are phone canvassing voters in the voice of whichever candidate the voter thinks they're talking to. There are AI generated campaign songs and even videos. In an Instagram reel, Prime Minister Narendra Modi is sitting cross legged strumming a guitar, crooning a popular Bollywood hit. 

00:03:14 Gareth Mitchell 

This is video, but the image itself of Prime Minister Modi is static, but there's nothing static about that voice. Very tuneful, indeed, and all AI generated. Now, this particular video seems to be doing well for the Modi cause, and anyway it's a bit of fun and it's getting the campaigning voice out there. 

00:03:34 Gareth Mitchell 

But voice cloning and related tech isn't just being used to satirise the candidates. The candidates themselves, as we hear there, are kind of embracing it themselves to partially automate their campaigning. 

00:03:46 Gareth Mitchell 

This is surely the first real AI election. The first one that scale at least. But could all these deep fakes also be compromising campaigns, misleading voters, and ultimately distorting democracy? Let's discuss with investigative journalist Nilesh Christopher, Nilesh, welcome along to the podcast. 

00:04:04 Nilesh Christopher 

Thank you for having me Gareth. 

00:04:06 Gareth Mitchell 

So I mentioned that Modi video and I should say it's not an official campaign video, is it? But nonetheless it it it's proving quite a hit. Can you just tell us a bit more about this video? I have seen it. It's a bit bizarre, but weirdly effective as well. So what's going on here? 

00:04:21 Nilesh Christopher 

It's the video of Modi strumming a guitar and and in his own voice singing in multiple different languages. Is that is that the one? 

00:04:28 Gareth Mitchell 

Yep, that's the one I'm thinking of because it's not just in Hindi. I'm glad you said multiple languages cause it's in Tamil, isn't it? In Telugu and and an all kind of AI generated voice clone? 

00:04:39 Nilesh Christopher 

Absolutely. So we have had an influx of free tools that we have which have pre trained models in the voice of different politicians, be it Modi, Bolsonaro, Trump and others. 

00:04:50 Nilesh Christopher 

So creators who want to satirize or build their social media accounts, just about the end of 2023 in December started leveraging these tools and creating AI songs, and the voices of political leaders. Some were satirizing them. Some were valorizing them as well. So what we saw initially was Modi, who speaks the Hindi language and which is extremely popular in India south. 

00:05:18 Nilesh Christopher 

Suddenly you had videos of him singing in southern Indian languages like Tamil, Telugu, Malayalam and others. So in effect the first phase of this was AI normalizing the way in which Modi could speak southern Indian languages, which he previously couldn't. And some enterprising  

00:05:38 Nilesh Christopher 

folks saw this as an opportunity where Modi's BJP, which has traditionally been powerful and and extremely popular in the north, could use these audios to campaign in local Indian languages in India south. 

00:05:53 Nilesh Christopher 

So memes paved the way for multilingual AI campaigns in Modi's voice. In many of these southern Indian languages. And right now we are witnessing many southern state elections where voice calls in Modi's voice speaking a local dialect or language is being done for canvassing voters and endorsing candidates. 

00:06:13 Gareth Mitchell 

Right. Yeah. Because we so often hear about deep fakes and voice cloning as as tools of satire or even as kind of malicious tools to mock or to discredit politicians of various hues. But here we're seeing an example of it being used both  

00:06:31 Gareth Mitchell 

actively by candidates as a campaigning tool, but also by their followers and supporters as a way of popularising them and kind of doing the campaigning for the candidates. Have I got that right? 

00:06:44 Nilesh Christopher 

You're absolutely right. I call this the year of the AI experiments. Be consultants, proxy pages of politicians or politicians themselves. 

00:06:54 Nilesh Christopher 

Everybody is trying out a different use case with AI and the two things that we have hit on right now is voice clones and a politician's voice for for canvassing votes. And in the second-half you have AI generated images being created that help valorize a certain leader or show them in certain light. 

00:07:15 Nilesh Christopher 

My favorite Gareth, is portraying Modi as a saffron superhero. So in this AI generated image. 

00:07:22 Gareth Mitchell 

So it it's an AI image and he, yeah, he's he doesn't look like his normal prime ministerial self, shall we say. 

00:07:29 Nilesh Christopher 

No, he isn't. So in this AI generated image shared by one of BJP's proxy pages, there is a a saffron colored flag, a flag with a, with the  

00:07:42 Nilesh Christopher 

symbol OM on it, which is a Hindu spiritual symbol, and saffron is a color often associated with Hindus and right below that flag, you could see Modi with his white wavy hair and a saffron cape like almost like a superhero walking down there. And it was shared on social media, especially on Instagram. And it received over 2 million views. 

00:08:02 Nilesh Christopher 

What these images do is cultivate a certain image of Modi as a saviour of Hindus or cast him in a certain light, where his supporters see him as the saviour or, or what do you call the,  

00:08:20 Nilesh Christopher 

the person who is going to help reincarnate Hindu mythology and things like that. So we we've seen AI image tools being co-opted for strategic narrative building in Indian politics. 

00:08:33 Gareth Mitchell 

Angelica, you're quite interested in this whole sphere as a business opportunity, aren't you? 

00:08:38 Angelica Mari 

Yeah, and uh, Nilesh, you visited the firm, right, called Polymath Synthetic Media Solutions in Rajasthan, right.  In Brazil we've seen companies like operating in the space of  

00:08:53 Angelica Mari 

automating not just message blasts but also AI deep faking certain parts of their campaigns and they're seen as rather shady enterprises, but somehow they operate lawfully. I'm curious to know about how close to the edge in terms of legality at these companies operating in India. 

00:09:21 Nilesh Christopher 

No, it's a fascinating question. I've often wondered about as well. We visited what I call a deep fake service provider in a small Indian city of Ajmer. It's a north Indian city in the state of Rajasthan and you have this self-taught, deep faker who tried out a bunch of these Open Source tools and started sort of superimposing people's faces 

00:09:47 Nilesh Christopher 

on, you know, say Mark Zuckerberg on on a f[l]ight or Indian superstar in movies of in and other Hollywood movies. So that's how his journey started. Now he is in a situation where he creates sanctioned deep fakes, both visual deep fakes and audios of politicians to be able to do outreach. 

00:10:07 Nilesh Christopher 

When we visited that place he said he's got over 250 requests from politicians to be able to attack their opponents or do other nefarious stuff. So as someone who's been in this space for quite some time, he says he's actually taken an oath, an ethical AI coalition, and he's a part of it. And he would never indulge himself in creating nefarious deep . 

00:10:32 Nilesh Christopher 

fakes that attack an opponent. What he does is quote unquote positive campaigns. And it's really interesting how service providers who stand to benefit position themselves and  

00:10:45 Nilesh Christopher 

have kind of legitimized their role in sort of muddying the waters in this political election season. And, but that's what we have noticed and he has clocked in $55,000 of revenue over the past three months. 

00:10:58 Angelica Mari 

And uh, are there any examples of uh AI generated content sort of backfiring or causing a negative impact on a campaign that you can think of? 

00:11:11 Nilesh Christopher 

Currently, the person we are talking about, Devendra Singh, he's a 31 year old dude sitting out of a chair. The content that he has created hasn't necessarily sort of veered in the direction which has caused controversies yet. However, we have also seen shadow accounts and pages creating 

00:11:31 Nilesh Christopher 

video or voice clones of politicians saying things they did not. So the police immediately had to sort of jump in and one of the famous examples that has gone viral right now is Bollywood actors endorsing the Congress Party, which is 

00:11:50 Nilesh Christopher 

the chief opposition party to India's BJP, the ruling party. And they had created clones of Bollywood actors endorsing the Congress Party, which were not sanctioned, so police immediately had to sort of register case against the creator and sharer of such audio clones and and they had to act on it. 

00:12:09 Gareth Mitchell 

But if we're talking about candidates themselves, knowingly and indeed willingly producing deep fakes of themselves, you know, for instance, if they want to speak to to voters in one language rather than another. I mean, that's one thing. We might say, well, that's fine, you know, at least this isn't some kind of malicious actor deceiving people in order to disrupt democracy. But there is a big question of trust here. Isn't there, you know? 

00:12:35 Gareth Mitchell 

The whole idea of politicians is haha, we're meant to trust them. If they're willingly and knowingly deceiving people through these voice clones and deep fakes, that's hugely problematic, isn't it, Nilesh? 

00:12:48 Nilesh Christopher 

Yes, to a certain extent. One of the things that I noticed during my reporting, is the pointers around disclosures and identification and labeling of air generated content. The standards are still evolving, so the deep peak service providers who are testing this first wave of AI generated content for political outreach, they are just sort of trying to fix things on the fly  

00:13:15 Nilesh Christopher 

and get consent from the politicians who they are cloning the voice or likeness off, and that's so they're able to cover their bases that like, hey, we sought consent from this politician who whose voice we are cloning but on the other hand, the voter or the beneficiary who's receiving this one way blast voice call doesn't necessarily know that if it's, if they're talking to an AI or a human, and one of the interesting examples that we came across is is a song called i2 Connect. They did about 20 million calls in the lead up  

00:13:54 Nilesh Christopher 

to the elections that's happening right now. This these calls were in Telugu language and it was done for multiple local politicians who wanted to do voter outreach. Some of these calls wished users on special occasions their birthdays. Others were specifically asking voters to vote for them or endorse other candidates in their local areas. 

00:14:16 Nilesh Christopher 

Another interesting example that when they did these sort of calls last December during a state elections, a bunch of people who received these calls in rural India showed up at the party offices thinking these were real and they were extremely delighted that the candidate they know were being addressed by their names, and personalized using AI 

00:14:36 Nilesh Christopher 

they were, of course not aware. So it's sort of we're seeing this slow unravelling of how when the rubber hits the road when AI is deployed at large scale, different communities with different literacies, how they react to this technology going live. 

00:14:56 Gareth Mitchell 

OK. Well, we'll leave it there. Nilesh. Final word from you, Angelica on this. 

00:15:01 Angelica Mari 

It is really interesting. In Brazil we've seen the  Superior Electoral Court actually setting limits to what extent AI is going to be used in campaigns. You have to tell people whether AI is being used in any campaigning materials. 

00:15:24 Angelica Mari 

They they're setting a lot of rules for this, but from what we we hearing just now from Nilesh, I think pretty soon we going to get like equally large countries like Brazil getting to that stage where it's impossible to put the genie back in the bottle. Right. 

00:15:46 Angelica Mari 

And and you have like a businessman overtly making millions out of this. And people will just have to be informed, if not by the government, by NGO's or activist groups. And how you navigate this scenario where you will be increasingly hard to detect what's real and what's not. 

00:16:11 Gareth Mitchell 

OK, Angelica, thanks very much. More from Nilesh Christopher in the extended version of this podcast for our subscribers. But now let's move on to something pretty related really because of course 1/4 of the world has experienced election Internet censorship this year. That's according to new data from Top10VPN, in a year when 60 countries are going to the polls. 

00:16:35 Gareth Mitchell 

90% of voters in countries that have had elections so far in 2024 live in countries where Internet censorship is on the rise. So that's ten countries, including the likes of Russia and Indonesia. Let's hear more now from Simon Migliano, who's head of research at Top10VPN, and you and your colleagues have been compiling these figures haven't you, Simon? Welcome to the podcast. 

00:16:59 Simon Migliano 

Hi Gareth. Thanks for having me on. 

00:17:00 Gareth Mitchell 

So I mentioned a couple of these ten countries that come to light in your report, so we've already mentioned there Russia and Indonesia. So which of the other countries where we're really seeing this censorship that's happening on quite a scale. 

00:17:15 Simon Migliano 

Some of the biggest ones are in the Indian subcontinent, so India, Pakistan, Bangladesh. But also elsewhere, such as Taiwan, Senegal as well, and then Azerbaijan, the Comoros and and the Maldives, those are the 10 countries that have experienced Internet censorship around elections so far this year. 

00:17:34 Gareth Mitchell 

When you talk about Internet censorship, what kind of evidence are you searching for? Are you looking for specific websites that are being blocked, for instance, or social media shutdowns? Say, what are the criteria here? 

00:17:46 Simon Migliano 

So we looked at a quite a broad range of criteria for Internet censorship and interference around elections because it takes many forms.  So it includes things, at one extreme like a full Internet blackout, such as what happened in the Comoros in January. But it can be a lot more targeted. So in India, for example, X or or Twitter, we see take down notices instructing them to remove and suppress posts that were, that involved political speech. 

00:18:16 Simon Migliano 

You know, in Bangladesh a a major news website that's critical of the government was was blocked during the the course of the elections. So there's a whole range. You know, the the Internet doesn't have to be fully shut down for it to be disrupted, it can be throttled. If you slow down the Internet enough it becomes certain, almost useless for the average person. 

00:18:37 Angelica Mari 

In these countries where censorship is increasing, Simon, how are the citizens reacting? Are they using VPN's? Uh private networks? Any other tech to kind of circumvent censorship? 

00:18:52 Simon Migliano 

Certainly it depends on the type of Internet censorship, of course. So somewhere like Pakistan, where X has been blocked since February, we saw a big uptick in the use of VPN's as as citizens resisted the government attempts to control their access to social media content, they they use VPN's. 

00:19:13 Simon Migliano 

But somewhere like Azerbaijan, where there’s an Internet slow down and VPN can't help you. In fact there is very little you can practically do, or very little a normal person can practically do when there's a actual Internet blackout or or or severe throttling. But yes,  VPNs is the is the obvious choice for a lot of people. In 2024, VPN's are pretty mainstream. Most people have heard of them. Although of course, governments are also getting wise to  

00:19:40 Simon Migliano 

that and making it harder in some cases to to download VPNs by blocking them, although it's not in this particular study. Iran was very successful at blocking VPN's during the the recent process there, for example. 

00:19:56 Gareth Mitchell 

Of course, a big part of your research at top ten VPN is measuring the cost of Internet shutdowns, and I know you do that outside election times as well. But within this, maybe it's beyond the scope of this research, but are you able to put figures on, you know, for instance, for any of the countries on the list, any idea of the cost of these shutdowns and the cost of this Internet censorship? 

00:20:21 Simon Migliano 

Yes. So our Internet shutdown research is quite focused around the economic impact, mainly because that makes it quite easy to understand for people and and for the media. Some of these Internet shut downs around elections, or should I say Internet disruptions around elections do have an economic impact? So Pakistan, for example, has cost over 350 million U.S. dollars so far. That's the impact on on their economy. Senegal as well about $7,000,000, slightly more. But a lot of these, so for example that website I mentioned that got blocked in Bangladesh, you can't really put a price tag on that,  

00:20:59 Simon Migliano 

there's not an economic impact. So a lot of these Internet disruptions that have potentially swayed elections that had a significant impact on democracy. It is very hard to assign a a dollar value to them because they're either very targeted or or or quite short, or just very difficult to to tie up with economic indicators. 

00:21:23 Simon Migliano 

But yes, certainly the one in Pakistan is the standout. They've they've inflicted $350 million worth of economic self harm so far this year, just from blocking X. 

00:21:33 Angelica Mari 

What is your take on the role and responsibility of big tech companies in mitigating or amplifying censorship? Do you think that they are collaborating with authoritarian governments, or are they actually working to protect user rights? 

00:21:52 Simon Migliano 

I don't have a lot of sympathy, to be honest, with the big tech companies in general. I don't think they do an awful lot for our privacy and they don't necessarily look after our data but they are in a difficult position when it comes to government interference in elections. So let's take the example of X.  

00:22:13 Simon Migliano 

When they're informed by the Indian Government that they need to suppress certain content, if they don't comply, then they run a very real risk of being kicked out of the country by being blocked, which is a a worse outcome in the long run. 

00:22:29 Nilesh Christopher 

So whilst I'm not a huge fan of the current management of X. I do think they probably did as best as they could, which was to be transparent about the orders that they’v received from the government. They they shared them, they made it clear that they were doing this under duress. There's not a lot more that I think they can do. I think that's a reasonable course of action  

00:22:48 Simon Migliano 

that that X took. Clearly it's in their in their social media platforms interest to be as accessible as possible to as many people as possible but and, but to a degree their hands get tied a little bit. Like I said, if if they if they don't tow the line to a degree then then may be ejected from the country. 

00:23:08 Gareth Mitchell 

Right. OK, Simon and I know we don't have you for long, so let's just finish on this one because your report is it pains to say, look, it's not all bad. Of course there's a lot to worry about here, but you know, you talk about the many other countries where elections are being carried out freely and fairly, as it were, without having Internet censorship and election censorship, and I suppose that there are cases  

00:23:32 Gareth Mitchell 

of citizens being able to usurp the censorship in certain ways, so if we can, I don't know if we can finish on a bit of a positive note, but you are you, you are keen yourself not to say ohh it's it's all terrible. 

00:23:46 Simon Migliano 

Well, of course it's not all terrible, you know, I guess the the upside is that 20 countries out of 30 have had elections go by that are reasonably free and fair, and that that's a positive thing. And you know whilst we talk about the 10 countries with huge populations amongst them, any kind of election interference of course is, is contemptible and and and worth calling out. 

00:24:11 Simon Migliano 

But some of some of the intellectual interference that we've seen has been relatively minor. You know, it hasn't been at the scale of the Comoros where they had no Internet access for over 24 hours. You know, in some places it has been a documentary being censored or, you know, certain tweets being blocked, which is not ideal. 

00:24:30 Simon Migliano 

But yes, it's it's it's not all doom and gloom and certainly some of the elections interference is not quite as severe as some the others, which is a good thing. 

00:24:40 Gareth Mitchell 

Brilliant. Well, we'll leave it there, Simon. Thank you very much indeed. 

00:24:44 Simon Migliano 

You're welcome. Thanks for having me on. 

00:24:45 Gareth Mitchell 

OK. Well look, just before we go, let's just do, if we've got time, just one quick little subscriber number, we'll just sneak this one in. So I need a vote from our subscriber committee here whether we can allocate the number 37 to Jan Willem, 

00:25:02 Gareth Mitchell 

who says, great programme, thanks. Well, that's definitely going to get you somewhere in the queue, I think, if you complement the programme. But Jan Willem, we're not over the line yet with your subscriber number. We need to get Angelica and at least Ania to agree to this. Can we put that through Angelica? 

00:25:24 Angelica Mari 

Yeah, I, uh, little moment of silence here. There's no drone. 

00:25:29 Gareth Mitchell 

Well, you have to think about it. I know these these are very big and important decisions and we don't take them lightly. So I really understand that pause for thought. 

00:25:35 Angelica Mari 

Yes. But I wanted to, I wanted to dig up a little bit of trivia because he didn't explain any reason why he would want the number 37. But it's my duty here to bring random facts into Somewhere on Earth. 37 is the median value for the 2nd prime factor of an integer. 

00:26:01 Angelica Mari 

So that's the probability well. 

00:26:02 Gareth Mitchell 

How the heck did you know that? 

00:26:05 Gareth Mitchell 

Do you have one of these facts for every single number that we could run by you. 

00:26:09 Angelica Mari 

Well, I I tried to be the random fact generator of this programme, it is my role. 

00:26:17 Gareth Mitchell 

You've certainly done that. Well, maybe that's why Jan Willem has asked for that number then. But that's good knowledge. But well um. Well, I OK. So we've got, I think a vote. Angelica is kind of voting for herself in a way, I think by giving the Jan Willem a very good reason. 

00:26:33 Angelica Mari 

I won't claim that, I won't claim that number though. You can have it Jan. 

00:26:37 Gareth Mitchell 

I’d be tempted, if I had that kind of knowledge. Ania, how about you? Cause you get the casting vote here. This is all hanging on you. Can we let this through? Especially,we have to take into account that Jan Willem didn't give their own reason for wanting this number? It might be a bit tenuous. So this is on a bit of a knife edge, isn't it? 

00:26:55 Angelica Mari 

Well, he did say liked the programme, so I think we should say yes. 

00:26:59 Gareth Mitchell 

Fair enough. OK, we'll, we'll, we'll have that then. OK. So that's on the database. We will do a few more of these when we have a little bit more time going forward and you don't dear listener always have to say what a great programme, although as you can as you just heard, it probably helps. And we want the reasons behind your numbers as well please. So there we are we'll  

00:27:18 Gareth Mitchell 

leave it there. So thank you very much indeed to you, Angelica, to Nilesh, from whom we're going to hear a little bit in a few moments, for those who subscribe to us. Thank you to you,  if you're a subscriber as well. And we have at the controls today, Keziah. 

00:27:34 Gareth Mitchell 

And our production manager is Liz Tuohy, the producer/editor and casting voter on the listener subscriber committee, Ania, is there keeping us all going? And I'm Gareth, and thank you very much to all the people that Lanson's Team Farner for making us so welcome, and keeping the podcast sounding great. See you next time. Bye bye.