Gemini Trifecta Highlights Dangers of Indirect Prompt Injection
New discoveries from network defenders have once again highlighted the ways in which everyday AI agents can be leveraged by threat actors to not just become attack targets, but active threat surfaces. A recent publication from research firm Tenable performed extensive vulnerability studies on the Google Gemini AI agent, with the result of discovering three highly dangerous vulnerabilities, dubbed the Gemini Trifecta by researchers, that seriously endanger the data integrity of users. These vulnerabilities are used to facilitate indirect prompt injection, manipulating what the Gemini agent presents to its user, as well as data exfiltration and retrieval.
The first vulnerability of the Gemini trifecta affects Gemini Cloud Assist. This tool is used to summarize log entries in the Google Cloud Platform, simplifying complex data and delivering recommendations. By inserting attacker-controlled text into a log entry which was summarized by Cloud Assist, the researchers were able to deliver instructions that were executed by Gemini. “To test this, we attacked a mock victim’s Cloud Function and sent a prompt injection input into the User-Agent header with the request to the Cloud Function. This input naturally flowed into Cloud Logging. From there, we simulated a victim reviewing logs via the Gemini integration in GCP’s Log Explorer,” explained Tenable, “to our surprise, Gemini rendered the attacker’s message and inserted the phishing link into its log summary, which was then output to the user.”
The second vulnerability targets Gemini’s Search Personalization Model, a tool that uses user search history to contextualize responses. By using JavaScript from a malicious website, researchers were able to inject malicious search queries into a target user’s browsing history. “When the user interacted with Gemini’s Search Personalization Model,” Tenable reported, “it would process the user’s search queries, including these malicious search queries injected by the attacker, which are essentially prompt injections to Gemini. Since the Gemini model retains the user’s memories, aka ‘Saved Information,’ and the user’s location, the injected queries can access and extract user-specific sensitive data.” In this way, malicious search injections could enable threat actors to harvest personal and corporate data stored as AI memories.
The third and potentially most dangerous of the Gemini Trifecta affects the Gemini Browsing tool, a module that accesses live web content and generates summaries. “This functionality is powerful, but when combined with prompt engineering, it opened a side-channel exfiltration vector,” Tenable warned, “What if we asked Gemini to ‘summarize’ a webpage – where the URL included sensitive data in the query string? Would Gemini fetch a malicious external server with the victim’s sensitive data in the request?” Tenable was able to achieve this outcome largely with the benefit of Gemini’s Show Thinking feature, which revealed the model’s browsing language and the API calls it used to achieve its results. This allowed the researchers to create specially crafted prompts.
These three issues were reported to Google upon discovery and have now been patched, but what they represent indicates significant danger posed by AI in any secure environment that accesses web data. As LLMs currently stand, indirect prompt injection represents a constant danger that cannot be easily detected. Tenable’s recommendations for addressing the issue include assuming as a default that attacker-controlled content will reach AI systems indirectly, implementing layered defenses, including input sanitization, context validation and strict monitoring of tool executions, and regularly performing pen tests on AI-enabled platforms for prompt injection resilience.
