Deepfake and artificial intelligence (AI) powered scams have caused financial losses nearing RM4.2 billion globally to date. In the first half of 2025 alone, losses from deepfake fraud reached RM1.92 billion, a stark increase from RM1.68 billion for 2024, and a mere RM601 million total over the period 2019 to 2023.
This is an alarmingly exponential increase compared to when hackers and scammers did not use Generative AI. Previously, access to such services was simplified to help it spread. But now, with deepfake scam losses expected to only get worse, is it finally time to restrict access to subscription for Generative AI? How?
How bad is it exactly? Surely not in Malaysia too?
In February 2024, a finance worker in Hong Kong was duped into authorising 15 transfers totalling RM117.5 million after participating in a video call where all colleagues, including the Chief Financial Officer, were AI-generated deepfakes.
In Malaysia, Malaysian police are actively investigating over 450 deepfake scams, primarily involving voice impersonation, which have resulted in losses of RM2.72 million. Scammers leverage AI to mimic the voices of victims' acquaintances, tricking them into transferring funds to mule accounts. Notable cases include a travel agent in Malaysia losing RM49800 and other victims in Kuala Lumpur and Penang.
Additionally, Malaysian authorities have detected AI-generated deepfake videos featuring prominent figures, including Prime Minister Anwar Ibrahim and even the King of Malaysia, Sultan Ibrahim, promoting fraudulent investment schemes, further misleading the public and causing potential financial harm.
Popular Malaysian singer Siti Nurhaliza's voice and image have also been exploited in AI-generated calls to lure victims with promises of video calls and fictitious rewards. Capital A Berhad and AirAsia have issued public advisories warning against fake, AI-generated videos impersonating their CEO, Tony Fernandes, to promote investment schemes.
Barely hours after the Prime Minister of Malaysia's recent announcement of RM100 aid, scammers came up with their own version with links. If this is any indication, the scammers are becoming faster and faster...perhaps mostly thanks to Generative AI.
The RM100 Sara aid scam came just hours after the announcement
If not now, then when?
The alarming rise in such incidents underscores why it is critically important to start implementing more robust vetting for user subscriptions to generative AI services. Currently, subscribing to many generative AI services is remarkably easy. Most platforms require little more than an email address and a password for free tiers, and sometimes a credit card or payment for paid access, but without any real identity verification.
Hackers and cybercriminals can sign up with burner emails, virtual private networks, or stolen credit card details, maintaining complete anonymity. This ease of access allows malicious actors to rapidly create multiple accounts, generating vast quantities of harmful content without fear of immediate detection or accountability.
You can even sign up for some Generative AI platforms for free
The lack of stringent user vetting acts as a gateway for abuse, enabling the very scams that are now costing billions. While most Generative AI services have integrated guardrails to prevent them from generating on certain topics (how to make bombs, etc), clearly, they aren't working very well.
What can be done?
Recently, many e-wallets and online services in Malaysia required users to perform identity verification via eKYC (electronic Know Your Customer). If they didn't, they weren't allowed to use those services. This could be a step in the right direction for restricting access to Generative AI services.
As such, if you wanted to keep using a Generative AI service like ChatGPT, for example, you would have to do an eKYC process. Granted, we fully expect serious hackers and scammers to have fake or stolen IDs at the ready as well, but it would deter some of the more common scammers at least.
While eKYC registration services might increase the costs for Generative AI service providers, given the amount of losses from scammers and deepfakes, these costs should be trifling in comparison. If Malaysian service providers can do it for millions of Malaysians, why can't a Generative AI service provider do so too?
Let's lock out all those scammers!
Overall, mandatory eKYC is only a step in the right direction. Other measures, such as banning scammer users more strictly and adding human investigators and checkers as part of the subscription process, could also be better at stopping these scammers in the first place.
What do you think? Is it finally time for Generative AI service providers to get a bit more strict with their subscription security? Or do you have a different suggestion? Share your thoughts in the comments below and stay tuned to TechNave.com for more articles on tech.
COMMENTS