AI Tools Fail to Detect Fakes, Raising Concerns Over Accuracy

When a viral photograph of former Philippine lawmaker Elizaldy Co surfaced online amid a corruption scandal, Filipinos turned to an AI-powered chatbot for verification. This tool, however, failed to identify the image as a fake, despite having generated it itself. This incident underscores a troubling blind spot in AI technology as users increasingly rely on chatbots to assess the authenticity of images in real time.

As misinformation continues to proliferate, major tech platforms are reducing human fact-checking. Many AI tools incorrectly label fabricated images as real, complicating the online landscape that is already inundated with AI-generated content. The image of Co, who has been missing since an official investigation began, inaccurately depicted him in Portugal and attracted widespread attention. When questioned by online investigators, Google’s AI mode erroneously confirmed the image’s authenticity. Subsequent investigation by AFP uncovered that the image had been created using Google’s own AI technology.

According to Alon Yamin, chief executive of AI content detection platform Copyleaks, “These models are trained primarily on language patterns and lack the specialized visual understanding needed to accurately identify AI-generated or manipulated imagery.” This highlights a significant limitation; even when images originate from the same generative model, AI chatbots often provide inconsistent assessments.

Failures in Verification

Similar failures have been documented elsewhere. During protests in Pakistan-administered Kashmir earlier in March 2023, social media users circulated a fabricated image supposedly depicting demonstrators with flags and torches. AFP analysis revealed that this image was also created using Google’s Gemini AI model, yet both Gemini and Microsoft’s Copilot inaccurately classified it as genuine.

Rossine Fallorina from the nonprofit Sigla Research Center explained that “this inability to correctly identify AI images stems from the fact that they are programmed only to mimic well.” AI models can generate images that resemble reality but often cannot determine the authenticity of those images.

A study conducted by Columbia University’s Tow Center for Digital Journalism earlier this year tested the verification capabilities of seven AI chatbots, including ChatGPT and Grok. The findings indicated that none of these models accurately identified the origins of ten photographs taken by professional photojournalists.

The Human Factor

In the case of Co’s fabricated photograph, AFP tracked its creator, a web developer from the Philippines, who admitted he generated the image “for fun” using Nano Banana, Gemini’s AI image generator. He expressed shock at the rapid spread of his creation, stating, “I edited my post and added ‘AI generated’ to stop the spread because I was shocked at how many shares it got.”

Such incidents illustrate how AI-generated images can closely mimic real photographs, raising alarms as users increasingly shift from traditional search engines to AI tools for information verification. This trend coincides with Meta’s announcement to end its third-party fact-checking program in the United States, shifting the responsibility of debunking falsehoods to regular users through a model called “Community Notes.”

The role of human fact-checkers is crucial in a landscape where misinformation can escalate rapidly. While researchers recognize that AI models can assist professionals by quickly geolocating images and identifying visual clues, they caution against relying on these technologies alone. Fallorina emphasized, “We can’t rely on AI tools to combat AI in the long run.”

As AI continues to evolve, the challenge remains clear: ensuring the accuracy and reliability of information in a digital world increasingly populated by sophisticated fakes.