LLM Domain Hallucinations Enable Phantom Squatting Attacks on Software Supply Chains
Attackers are exploiting large language models' tendency to generate non-existent domain names and registering these hallucinated domains to intercept developer traffic and compromise software supply chains. This technique bridges AI model weaknesses with domain squatting to create plausible but fake package repositories and dependencies.
Affected
Unit 42 has identified a novel supply chain attack vector exploiting a fundamental weakness in large language models: their tendency to hallucinate plausible-sounding but non-existent domain names when developers query them for package sources, library repositories, or dependency information. Attackers register these hallucinated domains before legitimate alternatives exist, positioning themselves to serve malicious packages or redirect traffic to compromised repositories. This technique is particularly dangerous because it exploits trust in two directions: developers trust LLM suggestions as authoritative, and the domains appear legitimate because they were suggested by the AI model.
The attack surface is significant because LLMs are increasingly embedded in developer workflows through GitHub Copilot, IDE extensions, and standalone assistants. When developers ask an LLM to recommend a package or library source, they may not validate the domain against official registries. An attacker who registers a hallucinated domain can then distribute typosquatted or compromised packages, publish malicious updates to legitimate-sounding libraries, or execute credential harvesting attacks against developers authenticating to the fake repository. The probabilistic nature of LLM outputs means hallucinations occur frequently enough to make this scalable: attackers can monitor common hallucinated domains and register high-probability targets.
This attack pattern differs from traditional typosquatting by leveraging the perceived authority of AI suggestions. A developer might manually scrutinise a domain that looks slightly wrong, but is less likely to question a domain directly suggested by a language model they trust. Additionally, the supply chain context amplifies impact: a single compromised package can affect thousands of downstream consumers before detection, particularly if the attacker targets popular open source projects or internal organisational repositories.
Defenders should implement multiple controls: developers must validate package sources against official registries and project documentation rather than trusting LLM suggestions directly; organisations should block or audit package installations from non-authorised repositories; package managers should implement additional verification mechanisms such as signed releases and build reproducibility. Security teams should also monitor domain registrations matching known LLM hallucination patterns and establish incident response procedures for compromised package detection. For LLM providers, this highlights the need for more constrained outputs when providing specific technical identifiers like domain names, potentially requiring grounding against verified data sources.
The phantom squatting technique represents a genuine evolution of supply chain attacks by weaponising a specific AI model characteristic. It is likely to become more prevalent as LLMs become standard in developer tooling, making this an emerging rather than theoretical risk. Organisations with strong software composition analysis programmes and strict package provenance requirements will have significantly reduced exposure, but environments with permissive dependency management practices face material risk.
Sources