Skip to content

Slopsquatting, how the speed of AI-based development exploits an old attack vector, and how we could prevent it.

Goal

To propose and implement a tool usable by developers during artificial intelligence-assisted programming to prevent slopsquatting vulnerabilities.

Sources and Notes

Here we're diving into a well known AI trouble: model hallucination and how it's affecting the AI-assisted software development. For this purpose, I'll be supporting this article in a research paper published on March 2, 2025 by USENIX entitled "We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs"

On "Slopsquatting" term: this is a pretty new and colloquial term invented by the low-code development community, there is no "slopsquatting" references into Usenix's study. This article is my own elaboration, as Jose Pepe I have no relationship with Usenix and I am just a user of some of the tools I mention here.

How to read

The objective of this proposal is to understand how to use AI for software development in high-speed delivery cycles while mitigating one of its most important risks. You won't know if this tool is useful to you until you understand the problem we face as AI-assisted developers, so the order of reading will be as follows: first (1) I will explain and evaluate the problem based on the study published on March 2, 2025 by USENIX entitled "We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs", and then (2) I will present and explain the tool I developed as a potential preventive aid.

1. The problem

Why does it affect you as a developer or businessman?

Attention and prevention of "slopsquatting" is a main concern for the use of LLMs in programming tasks because almost one fifth of the generated packets are hallucinations and probable attack vectors. Consider that: about 97% of programmers use AI for programming tasks today, and approximately 30% of the code that is written every day is Artificial Intelligence-generated.

Due to the high speed of delivery demanded in certain market areas, AI tools are increasingly relied upon to generate code. If the risks are not known then the damage it can cause is incalculable: for a developer it can mean being responsible for having generated a backdoor into the organization where he's working, and for a company it can mean the breach of their systems and loss of trust by the market. Trust without understanding can even mean that the speed of development plummets due to more time and personnel dedicated to auditing and maintaining AI-generated code.

One of these issues called "model hallucination" derivates in a security threat that has been named "Slopsquatting" by some forums of the online dev-community and technology company newsrooms, and is affecting one in five packets written in code generated by open-source LLM. Using Usenix's study words: "at least 5.2% for commercial models and 21.7% for open-source models".

How it works?

In simple terms, slopsquatting consist on the LLM generating a code using a named external package, and this external package in reality either does not exist or is fraudulent, so the incorporation of the obtained code will generate either a broken installation of the working project, or the installation of malicious code in the development environment and potentially in the project's file repository.

The problem is observed when instructing an LLM to perform a programming task that is likely to require the use of an external module (like a project dependency), and especially when using a language whose package management is done through a centralized repository. For this article we take the case of Javascript and its package manager NPM.

In which LLMs does this happen? The percentage of package hallucination is lower in commercial extended language models, while it is higher in open source ones.

In which kind of packages does this happen? Slopsquatting requires that an LLM be widely used by communities of developers (ie ChatGPT) and also that the package name to be used is a common result of the hallucination of that LLM in particular.

Slopsquatting is an infection technique within the “packet confusion attacks” that consists of distributing malicious code using package names resulting from the hallucination of an LLM. It should be clarified that by confusion I'm not only referring to confusion on the part of the programmer; while the programmer may be confused, a malicious actor will directly attempt to confuse the package manager's dependency conflict resolver, and may not only succeed in installing the malicious package based on human confusion, but by forcing it.

It should also be noted that the injection of this type of package can occur anywhere in the dependency tree, it is a difficult problem to control, and for a developer it can be impossible depending on the project he is working on.

How does it look from the attacker's point of view?

The source article concludes that packet hallucination is a repeatable result. This means that an attacker can rely on the probability of repetition of a number of hallucinated packet names as an entry point to an attack.

In addition, for each LLM there is a likely set of hallucinated package names, while it is unlikely that two or more LLMs will match on a hallucination of package names.

Also, it is more likely that packet hallucination will generate packet names more different from those that exist (instead of using a name like “goolge” to brand-hack a google package, they would use a rather different word.) so an attacker should look for repeatability within an LLM to obtain a likely attack vector rather than trying typos.

It is intuitive to think that an attacker would try to get, for each massively used LLM, the set of names of hallucinated packages, and then start publishing using multiple accesses to centralized repositories (such as npm) packages with code that could, for example if it is a networked device, query DNS to a domain under its control, whose requests may leak basic information such as the computer name, the path in the file system....

LLMs and hallucinations

What is a hallucination and why is it caused? In terms of LLMs, hallucination consists of returning a result that does not correspond to the instruction given by the user of the model. The hallucination can be due to problems with the training data, problems with the training procedures, or problems during inference. The hallucination that gives rise to Slopsquatting is the "package hallucination", which is a type of "fact hallucination".

Hallucination during inference is related to verbosity and model temperature, both of which make the model more prone to hallucination.

Hallucination is a type of behavior of an LLM in the inference phase that also involves other associated behaviors, e.g. a certain amount of verbosity, certain grammar, etc.

About the study that supports this article, as well as the other cited studies on AI applied to software development are done using datasets of development tasks including public and massive sources such as stack overflow.

Which can't be done as a diagnosis?

The comparison between known package names and the names generated by an LLM is not a solution, because a malicious user may have published a malicious package before this comparison is made. An attacker can even pretend that his malicious package is being downloaded and incorporated into the dependency chain of the node repository Filtering the LLM output based on reserved import words in languages (such as require) is not useful for getting malicious module injections because, from the filtering, we will get packages and not modules which are the ones containing the payload of the attack.

Which can be done as prevention?

As a Client after prompting, we will first need to know the current state of the LLMs we use as tools for programming. For example, we will need to know data such as Codellama 7B is the model most prone to packet hallucinations, and javascript is more prone to packet hallucination than python. Also that a model is highly likely to detect its own packet hallucinations and self-correct, even if we do not ask it to do so. It is a good idea to ask for the resolution of a problem in code and then check the central repository database, and ask the model for a correction if necessary, even asking to use specific packages. A 'curious' thing on Javascript and Python, is that when an llms is given a javascript programming task and it hallucinates, it is likely to return code using a package that does not belong to npm but to python PyPI (mainly) or another language's package manager.

2. My proposal: Which can be done as mitigation?

There are things that are possible for both an LLM client and a supplier. I'm focusing on the client, as that is my current role in using AI for work.

Here in mitigation, as in prevention, is the pre-generation code defense, and post generation.

The pre-generation slopsquatting defense, as LLM clients, will focus on the prompt we provide as input; here, the article clarifies that RAG is the client-oriented technique which provides the best results on mitigation. Also points on that Packet hallucination mitigation strategies affect the quality of the code generated by the LLMs.

My proposal focuses on post-generation mitigation, as I assume that AI will continue to be used for code generation while LLM providers try to improve hallucination statistics, so we have to find how to maintain productivity while ensuring safety.

And, in post generation, my proposal has two steps:

  1. firstly to identify suspicious package traits
  2. then to force the programmer to recognize and validate such a package, before it is installed in the development environment and potentially incorporated into the project repository.

There are scripts and hooks that call the packages at the very moment when the installation is finished, that is the risk that I try to avoid with the proposal. Reliability is returned to the programmer in a way that maintains productivity; AI is still used for code generation while its development environment forces the programmer to take responsibility for his product.

Proposal Explanation on Node

The mechanisms by which malicious packages could execute in a development environment

First let's do a brief overview of how npm works in the contexts of downloading and installing packages, and executing commands.

NPM is composed by a centralized repository, whose url is https://registry.npmjs.org/, plus a version control system for dependencies consistency and a CLI client.

When working with external packages, cli client will search online that package across several web protocols such as git, http, etc and, once it's found, it gets downloaded and then installed. When working with packages into working directory, then npm will install it directly from the named file or flder. If you don't pass any parameter to ´npm install´ then it's gonna search in the current working directory for a file called package.json, after read that npm will read the package-lock.json if exists, and then it's will download all required packages (and its dependencies) as tgz packages and finally uncompress all those into node_modules/. node_modules is a directory used as a cache memory shared by the whole package.

This is when the phenomenon called “dependency resolution” takes place, which is the automatic identification of each required resource, its provider and its version, in order to achieve a non-corrupt installation for the whole environment where the main package (working project) is going to be executed (and remember that the dependencies of a package have a whole chain of necessary sub-dependencies, it's really a miracle that we still exist, isn't it). There are also nested dependency trees, created by npm to solve the construction of two or more dependencies that use different versions of the same dependency. Once the complete dependency tree of the main package is installed, the npm hoisting algorithm is run, which selects which particular version or package to use at runtime in case a dependency is used in the dependency tree by more than one importer having multiple versions of the same dependency, this is for performance sake.

The current package management security controls that might already mitigate these risks

Into NPM there already are some security controls:

  1. NPM audit: Npm can detect known dependencies vulnerabilities, but it requires those has been reported before. No completely useful here.

  2. Using scopes: If you're using well defined scope-limited packages you can reduce collision risk or generic names.

  3. Account verification: Maintainers has to have 2FA for publish packages, and NPM is showing if a package has an actived 2FA or not.

For us, Node Developers, there are some common practices helping to reduce these risks:

  1. Locking specify version of dependencies: If joined with a careful revision of each dependency update into the project (including a node_modules revision), then you can know what dependencies are you using and check their reliability.

  2. Dependency Analysis tools (for SCA and other techniques) For CICD, or just a well configured development environment, there are code auditoring tools for detect no reported vulnerabilities, npm has no such mechanism by itself.

  3. Blocking unreliable packages in CICD Configurating policies, commonly using npmrc, for ensure packages integrity using some hash mark, check package popularity, ensure that package's repository has active maintenance... and so.

The proposed mitigation solution, just thinking on it!

My proposal, a mitigation proposal for when the hallucinated package is already in the development environment but has not yet been installed, is to define reliability criteria, and force the developer to audit the package that the system determines to be suspicious based on those criteria so that the developer can decide whether or not to install the new suspicious dependencies.

This is a banality, but a useful one as it allows to exploit AI and keep the most useful of the human as a developer.

An example: If you had a continuous integration and delivery flow with a highly modularized and decoupled system, and you wanted to accelerate the speed of development with an automatic scheduling and integration agent, you could sandbox the module in which this agent works into an environment that emulates the target production environment, use the minimal permissions to access external resources that allow you to deliver to the user a usable iteration from their point of view but constrained from the back end, and send directly the integration of new deliveries to that environment where after running (long enough) it can be freed from that box of constraints to simply serve in production. For this you should set up feature flagging that allows you to also do incremental delivery. Using the tool I proppose, you could be informed by notifications that your code had malicious (or suspicious) packages and since you are in sandbox you could quarantine them or kill the test environment, just to review it later.

The workflow impact of the proposed mitigation solution

The best impact would be greater reliability when using AI assistants for coding, knowing that you won’t need to audit all the code the model provides, as you’ll focus only on what appears suspicious.

The worst impact would be dependency conflicts arising from a quarantine system. New packages are generated every day by building on top of previous ones in the centralized NPM repository, so this proposal MUST be implemented retroactively. What should be done about malicious packages that are already part of the dependency chains of the tools we use?

Without a deep reflection, I can also mention that adding another step to an integration flow (as I’m proposing) could lengthen release times. I might also think this solution could clash with the promise of trends like vibe-coding due to the constant auditing I’m suggesting.

Ending ideas on proposal

For me, you are only productive using LLM when what you invest in auditing the result is less than what you invest in developing the code without LLM. Today AI is a toolbox that we have to know and sometimes know how to use well. Failure to do so limits our development opportunities and reduces the chances of product success. With patience, attention and the love that we #geeks always have for these things, we move forward steadily.

In the tool I'm thinking here, there are several missing points. What about false positives? What to do when you realise about a malicious package? How large should a project be to avoid creating a bottleneck in human validation?

I'd love to hear critiques and counterproposals. In the coming days, if this article receives feedback, I'll implement an open-source PoC.

Contact

You can send me an email to: jose.s.contacto@gmail.com

Also you can send me a WhatsApp to: +54 9 11 2401-0760

Link to article: We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs