AI Chatbots Struggle to Verify Images Amid Growing Concerns

In a troubling incident highlighting the limitations of artificial intelligence, an AI-powered chatbot failed to identify a fabricated image of former Philippine lawmaker Elizaldy Co during a corruption scandal. The image, which falsely depicted Co in Portugal, was generated by the same AI model that was later consulted for verification. This incident raises significant concerns regarding the reliability of AI tools in distinguishing real images from manipulated ones, especially as many users increasingly turn to these chatbots for real-time verification.

As misinformation proliferates online, particularly through social media, the reliance on AI chatbots for fact-checking has intensified. In this case, when online investigators in the Philippines queried Google’s new AI mode about the authenticity of the image, it incorrectly confirmed that the image was genuine. Following this misstep, AFP (Agence France-Presse) conducted an investigation that traced the image back to its creator, a web developer who generated it using Nano Banana, an AI image generator.

Challenges in AI Verification Capabilities

The inability of AI tools to detect their own creations raises questions about their visual debunking capabilities. Alon Yamin, the chief executive of AI content detection platform Copyleaks, noted that current models are primarily trained on language patterns and lack the nuanced visual understanding necessary for accurately identifying AI-generated or manipulated images. “Even when an image originates from a similar generative model, the chatbot often provides inconsistent or overly generalized assessments,” he stated.

This situation is not isolated. During recent protests in Pakistan-administered Kashmir, social media users shared another fabricated image that appeared to showcase demonstrators marching with flags and torches. An analysis by AFP revealed that this image was also created using Google’s Gemini AI model. Both Gemini and Microsoft’s Copilot failed to recognize it as a fake, instead classifying it as genuine.

Research from Columbia University’s Tow Center for Digital Journalism earlier this year corroborates these findings. A study involving seven AI chatbots, including popular models like ChatGPT and Gemini, tested their ability to verify ten images from news events. The results were disheartening, as all seven models failed to accurately identify the origins of the photos.

Impact on Misinformation Spread

The proliferation of AI-generated images on social media has significant implications for public discourse. The web developer behind the image of Elizaldy Co expressed shock at the number of shares it received, admitting, “Sadly, a lot of people believed it.” To mitigate the spread of misinformation, he later edited his post to clarify that the image was AI-generated.

As users increasingly shift from traditional search engines to AI tools for information gathering, the stakes become higher. This trend coincides with Meta‘s announcement earlier this year to end its third-party fact-checking program in the United States. Instead, the company is transitioning to a community-driven model known as “Community Notes,” which relies on everyday users to debunk falsehoods. This shift raises concerns, particularly in polarized societies where accusations of bias against professional fact-checkers are common.

While researchers acknowledge that AI models can assist professional fact-checkers by helping to geolocate images and identify visual cues, they emphasize that these tools cannot replace trained human experts. “We can’t rely on AI tools to combat AI in the long run,” noted Rossine Fallorina from the nonprofit Sigla Research Center.

As AI continues to evolve, its impact on information verification remains a critical issue. The growing reliance on these technologies underscores the need for enhanced capabilities and the importance of human oversight in the fight against misinformation.