Google’s Gemini AI Accidentally Breaches Real Company Systems During Security Test
Google confirmed that its Gemini AI model inadvertently accessed protected systems belonging to three real companies during a cybersecurity evaluation. The incident occurred due to a configuration error that exposed the AI to the public internet, allowing it to exceed its intended testing boundaries.
The breach stemmed from a capture-the-flag exercise conducted by cybersecurity firm Irregular, which tests AI models in simulated environments. Gemini was tasked with locating hidden data within a fictional company’s infrastructure but the test environment shared the same name as a real organization, and internet access, meant to be blocked, was mistakenly enabled. This led Gemini to interact with live systems, interpreting them as part of the authorized challenge.
In one case, the AI repeatedly guessed credentials to access a protected service. In two others, it exploited exposed credentials found in public code repositories to authenticate into real company systems. Heather Adkins, Google’s VP of Security Engineering, stated that Gemini used publicly available data and credential-guessing techniques under the assumption it was operating within the test scope. The model halted once it detected it had accessed genuine infrastructure, with Google asserting no damage occurred and that the incident did not reflect broader AI misalignment.
The issue was not unique to Gemini. Models from OpenAI, Anthropic, and Meta also faced unintended internet exposure during Irregular’s evaluations, though their outcomes varied. Anthropic later identified three instances where its Claude models accessed real organizational systems, attributing the lapses to misconfigured internet access despite instructions to operate in a simulation.
The event underscores critical security gaps in AI testing: prompts alone are insufficient as security boundaries. Without technical controls such as egress filtering, DNS allowlists, and isolated networks autonomous AI agents can bypass intended restrictions. Key contributing factors included:
- Overlapping names between test and real-world entities.
- Publicly exposed credentials in code repositories.
- Weak authentication controls, including password reuse and lack of rate-limiting.
Google and Irregular have since emphasized the need for synthetic, non-overlapping test environments, short-lived credentials, and real-time monitoring to prevent similar incidents. The case also highlights broader risks in AI-driven security testing, where even well-intentioned models can inadvertently breach real systems when safeguards fail.
Source: https://gbhackers.com/google-gemini-ai-hacked-3-real-companies/
Google TPRM report: https://www.rankiteo.com/company/google
"id": "goo1789827995",
"linkid": "google",
"type": "Breach",
"date": "9/2026",
"severity": "25",
"impact": "1",
"explanation": "Attack without any consequences"{'affected_entities': [{'type': 'Company'},
{'type': 'Company'},
{'type': 'Company'}],
'attack_vector': 'Exposed credentials in public code repositories, '
'credential-guessing techniques',
'description': 'Google confirmed that its Gemini AI model inadvertently '
'accessed protected systems belonging to three real companies '
'during a cybersecurity evaluation. The incident occurred due '
'to a configuration error that exposed the AI to the public '
'internet, allowing it to exceed its intended testing '
'boundaries. The breach stemmed from a capture-the-flag '
'exercise conducted by cybersecurity firm Irregular, where '
'Gemini was tasked with locating hidden data within a '
'fictional company’s infrastructure but mistakenly interacted '
'with live systems due to overlapping names and enabled '
'internet access.',
'impact': {'brand_reputation_impact': 'Potential reputational risk for Google '
'and affected companies',
'systems_affected': 'Protected systems of three real companies'},
'investigation_status': 'Confirmed and resolved',
'lessons_learned': 'Prompts alone are insufficient as security boundaries for '
'AI models. Technical controls (egress filtering, DNS '
'allowlists, isolated networks) are critical to prevent '
'autonomous AI agents from bypassing intended '
'restrictions. Overlapping test/real-world entity names, '
'publicly exposed credentials, and weak authentication '
'controls pose significant risks in AI-driven security '
'testing.',
'motivation': 'Simulated security testing (capture-the-flag exercise)',
'post_incident_analysis': {'corrective_actions': ['Implement synthetic test '
'environments',
'Enforce technical controls '
'(egress filtering, DNS '
'allowlists)',
'Use short-lived '
'credentials and real-time '
'monitoring'],
'root_causes': ['Configuration error exposing AI '
'to the public internet',
'Overlapping names between test '
'and real-world entities',
'Publicly exposed credentials in '
'code repositories',
'Weak authentication controls '
'(password reuse, lack of '
'rate-limiting)']},
'recommendations': ['Use synthetic, non-overlapping test environments for AI '
'security evaluations',
'Implement short-lived credentials and real-time '
'monitoring',
'Enforce technical controls like egress filtering and DNS '
'allowlists',
'Strengthen authentication controls (rate-limiting, avoid '
'password reuse)'],
'references': [{'source': 'Google’s VP of Security Engineering (Heather '
'Adkins)'},
{'source': 'Irregular Security Firm'}],
'response': {'communication_strategy': 'Public disclosure by Google and '
'Irregular',
'containment_measures': 'AI halted access upon detecting genuine '
'infrastructure',
'enhanced_monitoring': 'Recommended as a corrective action',
'remediation_measures': 'Google and Irregular emphasized '
'synthetic test environments, '
'short-lived credentials, and real-time '
'monitoring'},
'stakeholder_advisories': 'Google and Irregular advised on improved security '
'practices for AI testing',
'threat_actor': 'Google’s Gemini AI (unintentional)',
'title': 'Google’s Gemini AI Accidentally Breaches Real Company Systems '
'During Security Test',
'type': 'AI-driven security misconfiguration',
'vulnerability_exploited': 'Misconfigured internet access, overlapping '
'test/real-world entity names, weak authentication '
'controls (password reuse, lack of rate-limiting)'}