Digital Deception: Understanding and Avoiding Generative AI Fraud

Financial fraud is an ever-increasing danger that affects everyone, regardless of age or status. We all know someone who has been affected by financial fraud or have been victims of financial fraud ourselves. The Federal Trade Commission reported that in 2023, Americans lost $10 billion to fraud. Of that $10 Billion, $4.6 billion was reportedly lost in investment scams, and $2.7 billion was lost from imposter scams. These are large numbers, but experts believe that the true numbers are much higher due to underreporting.

According to a Gallup poll from 2023, young adults from ages 18 to 29 experience the highest rate of victimization by scams, but financial losses from scams tend to be much higher for older adults. One of the ways it has become easier for scammers to target people is through the use of technology, specifically generative AI.

What is generative AI, and how do scammers use it?

Generative AI is a type of artificial intelligence that generates high-quality text, images, audio, videos, and other content. The AI processes large quantities of information, which it is fed during a training process; it is taught to recognize patterns and structures in this data and then generate new data with similar characteristics. Generative AI can be a very useful tool in many different fields and can increase the speed of product development, improve productivity, or enhance customer experiences. As with any new technological development, it has the potential to be used for nefarious purposes as well.

Scammers are using generative AI to increase the scale of their attacks, improve their sophistication, and automate scam and fraud creation. By using AI, scammers are now able to quickly produce scams that are more relevant and convincing to you. A scammer might, for example, use generative AI to create an email that mimics the writing style and tone of your boss and is specific to your company and what you do.

One of the new and more alarming ways in which generative AI is being used is to create deepfakes of videos and voices. Deepfakes are realistic, manipulated videos or audio recordings that convincingly mimic a person’s voice or appearance. Scammers can use deepfakes to impersonate executives, public figures, or your loved ones to gain trust and trick people into sending them money or sharing sensitive information.

One example of this has been crypto giveaway scams using deepfake videos of Elon Musk that attempt to convince people to provide their personal financial information and purchase crypto with the promise that Elon Musk will match their purchase. These scams have been done with other celebrities as well and even with government officials. Public figures are easier for generative AI to impersonate because there is a lot of video and audio available for the AI to analyze to create a more convincing deepfake.

Scammers have also used deepfakes of people’s voices to impersonate family members who claim to be in need and coworkers or bosses requesting money transfers and information. In some cases, deep-faked audio and video have even been used successfully to bypass bank authentication software.

A more common way in which scammers are using generative AI is to improve scams that are already being used. Through AI, they are able to create better phishing and spear phishing attacks that are more targeted and sophisticated. Phishing is when a scammer contacts a person through an email or text message that is disguised as being from a reputable source and attempts to collect sensitive data or install malware. Spear phishing is a more specific type of phishing attack that uses personalized emails and messages that appear to be from a known trusted source. Scammers are able to quickly and more efficiently create scam websites, social media accounts and fake businesses in order to trick people into purchasing fake products.

They can also use AI to create seemingly official letters demanding money, which can even be made to look like they are from government agencies such as the IRS.

For the purpose of this article, I requested that ChatGPT 3.5 (a free-to-use generative AI tool) create an official letter requesting a payment of $1,000 to the IRS. The program performed the function in seconds. This letter could then be edited and distributed with instructions on how to pay the scammer. This is not a sophisticated example, but it is a scam that can be set up in minutes and requires minimal effort.

With the use of generative AI, the barrier to entry for scammers is much lower. In fact, scamming has been turned into a subscription service.

Tools like FraudGPT, a similar program to ChatGPT, which removes many safeguards and limitations of ChatGPT, can be purchased as a subscription on the dark web with regular updates. Scammers are even sharing information with each other on best practices for running successful scams.

What is being done to fight Generative AI Fraud?

Companies have started taking additional precautions in order to help fight generative AI fraud.

For example, they have focused additional resources on training and education for their employees to help them be aware of current scams and know what to look out for. Companies have also increased security protocols by using restricted networks, devices, and multi-factor authentication, as well as active, ongoing monitoring for suspicious activity.

AI-based tools are also being used to identify threats. These tools can be trained to detect patterns of fraud and suspicious activity, as well as monitor social media and online platforms for potential scams. What can you do to protect yourself? The way that Generative AI is being used to commit fraud and scams is alarming, especially the use of deepfakes.

Here are some ways to help protect yourself and your data:

• Stay up to date on the latest news about scammers and their most current methods of attack.

• Control what you can control by updating passwords and security software, using multifactor authentication, being careful with links and emails from unknown or unexpected sources, securing your personal information, and limiting what you share online and who you share it with.

• Be aware of how scammers get paid. A clear red flag is someone who insists on being paid with cryptocurrency, a wire transfer, a payment application, or a gift card.

• Take a breath and do not be pressured to act immediately. Honest businesses and organizations will give you time to make a reasoned decision and will rarely demand immediate payment.

• If you believe a communication could be legitimate, contact the person or company directly through a website or phone number you know to be real. Never use a website link, email, or phone number found in a suspicious communication.

• Be critical of everything that you see and read online. If something feels off, it may very well be. Deepfakes can be imperfect. If you pay attention to the details, you can often spot inconsistencies.

• Consider using Anti-Deepfake tools to help protect yourself.

In conclusion, Generative AI is a wonderful tool that can help make our lives easier, but it can also be abused. The best way to protect yourself, as it has always been, is to be aware and sceptical of everything. Do not rush to action but make well-informed, reasoned decisions.

Any opinions are those of Daniel Swain and not necessarily those of Raymond James. The information has been obtained from sources considered to be reliable, but we do not guarantee that the foregoing material is accurate or complete.