What's Not Being Said

The responses to #leftpadgate have mostly been coming from outsiders to JavaScript and Node.js. What follows is my Node.js dev perspective.

Immutable packages and Docker

One of the complaints about packages living in the npm registry is that they aren't immutable. While immutability is usually a descriptive enough term, in this case its leading to some confusion about the reality of the npm registry. Its true that you can run npm unpublish and have a package deleted, you cannot, however, republish the same version of a package twice. If you attempt to do this, you will get a 403 with the following message:

You cannot publish over the previously published version

Therefore, I think its more helpful to be clear when describing the issue. Instead of saying that npm should provide an immutable registry, we should be saying that a registry shouldn't allow a package to be deleted. Moreover, a registry shouldn't allow the removal of a package to break other packages.

Docker Hub also allows you to delete a published image.

If you try to pull from a deleted image you will get an error:

Error: image d0cker/blah not found

Additionally, you can create a different image with the same name and tag and publish it to the same location in the registry.

Why doesn't docker hub have this same issue? In some ways it does, you can prevent a docker pull command from succeeding. However, a published docker image that depends on an unpublished docker image won't suddenly break. This is different because the dependency is resolved at build time: the layer composing the parent image become part of the current image, and are pulled by their digest.

The Node.js package.json offers a similar solution, you can use bundledDependencies to bundle the code of your dependencies into your published tarball. In fact, ESLint tried to move to bundledDependencies, something that was proposed even before the recent event. However, in a more recent version of ESLint, they did revert that decision. Its not entirely clear yet if bundledDependencies are a straightforward solution.

I am personally in favor of being able to delete a package from a registry. It only makes sense when you consider that we are people, and people make mistakes. Even more, if you can't delete a package from a registry, and that package has known vulnerabilities in it, you are now doing more harm than good by allowing it to be served.

All of that being said, npm does need to change their implementation. A developer shouldn't be able delete a package that others are using, unless they have a valid reason to do so. In which case, npm unpublish should prompt them for reasoning, which will submit an issue to npm support for review. Here is one proposal for how this work can begin to be done.

Update - npm has a new unpublish policy

See http://blog.npmjs.org/post/141905368000/changes-to-npms-unpublish-policy

The new policy should fix almost every future issue. The only edge case is if you happen to depend on a package thats less than 24 hours old. The solution to that problem is even simpler, don't depend on a version of a package thats less than 24 hours old. Now we just need for npm to add a warning when someone attempts to install a version thats less than 24 hours old.

Fragility

Another common complaint about left-pad and many of the modules on npm is that they are too small and really shouldn't live out in the wild. Instead, they should be baked into JavaScript or your projects central code. I definitely agree with this sentiment. That being said, there is definitely merit to break out functions that shouldn't change from code that likely will.

One of the major selling points with Docker is the layered approach to building an image. This has the benefit of only needing to pull down the layers of your image that have changed. In the same way, when you separate functions that won't change into published modules, you are able to make upgrades easier as you should only need to worry about enhancements in more complex modules. There is also the benefit of having these more stable modules cached, so when you run npm update you only pull down modules that are frequent to change.

There is always a risk with depending on any code that you don't control or at least have access to. This is one of the reasons why most of the dependencies that hapi has are under the same github org. hapi strives to limit external dependencies that are beyond the control of the core team.

I am not going to say that there isn't fragility by some or even many of the packages published on npm. What I will say is that fragility is not inherent in npm or Node.js, as some people are implying. As with any sizeable diverse developer community, there will be software with varying degrees of fragility.

The JavaScript Ecosystem

Another common response to the event is people framing the discussion around the JavaScript ecosystem, the Node.js ecosystem, or the npm ecosystem. While all of this is true, it doesn't cut to the heart of the situation. The modules that were most impacted were React, Babel, and atom. These are certainly popular modules, but in reality, are not modules that are used by hapi, express, or restify. In which case, I think its more accurate to frame the discussion around the specific communities that were impacted.

This reframing of the discussion is important, as we should be clear about which Node.js sub-communities were impacted. Otherwise, there is a potential for people to think that all of Node.js, npm, or the Internet suddenly broke.

Resolution

If this event dissuades anyone from adopting Node.js, I have to wonder if there was any hope for them to even adopt Node.js in the first place. In reality, this event reveals that Node.js is becoming more and more popular, and that its leaders are well equipped to deal with edge case events!

There is perhaps an even greater revelation brought about by this event, which is that many Node.js outsiders are necessarily fearful of its growing popularity.

Even still, we in the Node.js community have work to do. I have put forward a couple of proposals, as have others in the community. Hopefully, this will serve as a learning experience for everyone and we can do something to prevent it from occuring again.

Credits

A special thank you to Matteo Collina and Colin Ihrig for reviewing this post.