JSJ 266 NPM 5.0 with Rebecca Turner

JavaScript Jabber - Un podcast de Charles M Wood - Les mardis

Catégories:

On today’s episode of JavaScript Jabber, Charles Max Wood and panelist Joe Eames chat with Rebecca Turner, tech lead for NPM, a popular Javascript package manager with the worlds largest software registry. Learn about the newly released NPM 5 including a few of the updated features. Stay tuned!

[1:58] Was the release of node JS 8 tied to NPM5?

Features in NPM5 have been in planning for 2 years now.
Planned on getting it out earlier this year.
Node 8 was coming out and got pushed out a month.
Putting NPM5 into Node 8 became doable.
Pushed really hard to get NPM5 into Node 8 so that users would get NPM5 and updates to NPM5.

[2:58] Why would it matter? NPM doesn’t care right?

Right you can use NPM5 with any version of node.
Most people don’t update NPM, but upgrade Node.
So releasing them together allowed for when people updated Node they would get NPM 5.

[3:29] How does the upgrade process work if you’re using NVM or some node version manager?

Depends. Different approaches for each
NVM gets a fresh copy of Node with new globals. NVM5 and Node 8 are bundled.
For some, If you manually upgrade NVM you’ll always have to manually. It will keep the one you manually upgraded to.

[4:16] Why NPM 5?

It’s night and day faster.
3 to 5 times speed up is not uncommon.
Most package managers are slow.
NPM 5 is still growing. Will get even faster.

[5:18] How did you make it faster?

The NPM’s cache is old. It’s very slow. Appalling slow.
Rewrote cache
Saw huge performance gains

[5:49] What is the function of the cache?

Cache makes it so you don’t have to reinstall modules from the internet.
It has registry information too.
It will now obey http headers for timing out cache.

[6:50] Other things that made it faster?

Had a log file for a long time. It was called shrinkwrap.
NPM 5 makes it default.
Renamed it to packagelog.json
Exactly like shrinkwrap package file seen before
In combo with cache, it makes it really fast.
Stores information about what the tree should look like and it’s general structure.
It doesn’t have to go back and learn versions of packages.

[7:50] Can you turn the default Packagelog.json off?

Yes. Just:
Set packagelog=false in the npmrc

[8:01] Why make it default? Why wasn’t it default before?

It Didn’t have it before. Shrinkwrap was added as a separate project enfolded in NPM and wasn’t core to the design of NPM.
Most people would now benefit from it. Not many scenarios where you wouldn’t want one.
Teams not using the same tools causes headaches and issues.

[9:38] Where does not having a lock show up as a problem?

It records the versions of the packages installed and where NPM put them so that when you clone a project down you will have exactly the same versions across machines.
Collaborators have the exact same version.
Protects from issues after people introduce changes and patch releases.
NPM being faster is just a bonus.
Store the sha512 of the package that was installed in the glock file so that we can verify it when you install. It’s Bit for bit what you had previously.

[11:12] Could you solve that by setting the package version as the same version as the .Json file?

No. That will lock down the versions of the modules that you install personally, not the dependancies, or transitive dependancies.
Package log allows you to look into the head of the installer. This is what the install looks like.

[12:16] Defaulting the log file speed things up? How?

It doesn’t have to figure out dependences or the tree which makes it faster.
Shrinkwrap command is still there, it renames it to shrinkwrap but shrinkwrap cannot be published.
For application level things or big libraries, using shrinkwrap to lock down versions is popular.

[13:42] You’ve Adopted specifications in a ROC process. When did you guys do that?

Did it in January
Have been using them internally for years. Inviting people into the process.
Specifications
Written in the form of “Here is the problem and here are the solutions.”
Spec folder in NPM docs, things being added to that as they specify how things work.
Spec tests have been great.

[14:59] The update adds new tools. Will there be new things in registry as well?

Yes.
Information about a package from registry, it returns document that has info about every version and package json data and full readme for every version.
It gets very large.
New API to request smaller version of that document.
Reduces bandwidth, lower download size, makes it substantially faster.
Used to be hashed with sha1, With this update it will be hashed with sha512 as well as sha1 for older clients.

[16:20] Will you be stopping support for older versions?

LTS version of NPM was a thing for a while. They stopped doing that.
Two models, people either use whatever version came with Node or they update to the latest.
The NPM team is really small. Hard to maintain old NPM branches.
Supports current versions and that’s pretty much it.
If there are big problems they will fix old versions. Patches , etc.

[17:36] Will there ever be problems with that?

Older versions should continue to work. Shouldn’t break any of that.
Can’t upgrade from 0.8.
It does break with different Node version
Does not support Node versions 0.10 or 0.12.

[18:47] How do you upgrade to NPM?

sudo npm install -gmpm
Yes, you may not need sudo. depend on what you’re on.

[19:07] How long has it been since version 4?

Last October is when it came out.

[19:24] Do you already have plans for version 6?

Yes!
More releases than before coming up.
Finally deprecating old features that are only used in a few packages out of the whole registry.
Running tests on getting rid of things.

[20:50] Self healing cache. What is it and why do we want it?

Users are sometimes showing up where installs are broken and tarbols are corrupted.
This happens sometimes with complicated containerization setups makes it more likely. It’s unclear where the problem actually is.
CaCache - content addressable cache. Take the hash of your package and use it to look up address to look it up in the cache.
Compares the Tarbol using an address to look it up in the cache.
Compares to see if it’s old. Trashes old and downloads updated one.
Came out with the cache. Free side effect of the new cache.

[23:14] New information output as part of the update?

NPM has always gave back you the tree from what you just installed.
Now, trees can be larger and displaying that much information is not useful.
User patch - gives you specifically what you asked for.
Information it shows will be something like: “I installed 50 items, updated 7, deleted 2.”

[24:23] Did you personally put that together?

Yes, threw it together and then got feedback from users and went with it.
Often unplanned features will get made and will be thrown out to get feedback.
Another new things ls output now shows you modules that were deduped. Shows logical tree and it’s relationships and what was deduped.

[25:27] You came up to node 4 syntax. Why not go to node 8?

To allow people with just node 4 be able to use NPM.
Many projects still run Node 4. Once a project has been deployed, people generally don’t touch it.

[26:20] Other new features? What about the File Specifier?

File specifier is new. File paths can be in package json, usually put inside pointing to something inside your package.
It will copy from there to your node modules.
Just a node module symlink.
Much faster. Verifiable that what’s in your node modules matches the source. If it’s pointing at the right place it’s correct. If not, then it’s not.
Earlier, sometimes it was hard to tell.

[27:38] Anything else as part of the NPM 5 release? Who do you think will be most affected by it?

For the most part, people notice three things:
1st. no giant tree at the end
2nd. Much faster
3rd. Package lock.

[28:14] If it’s locked, how do you update it?

Run npm installer and then npm update
Used to be scary, but works well now.
Updates to latest semver, matches semver to package json to all node modules.
Updates package lock at the same time
Summary in Git shows what’s changed.

[28:59] Did Yarn come into play with your decisions with this release?

The plans have been in play for a long time for this update.
Yarn’s inclusion of similar features and the feedback was an indicator that some of the features were valuable.

[29:53] Other plans to incorporate features similar to yarn?

Features are already pretty close.
There are other alternative package managers out there.
PMPM interesting because when it installs it doesn’t copy all the files. It creates hard links.

[30:28] Does PMPM and Yarn use NPM registry?

Yes! Other than CNPM. The NPM client used in China.
CNPM Registry mirror behind firewall. Have their own client to their registry. Their registry is a copy of ours.

[31:15] What about RNPM?

I wouldn’t be surprised.

[31:45] “Won’t you come and say something controversial about your competitor?”

We all want it to be collaborative.
When we were writing our new cache, we also helped Yarn with their cache and sped things up tremendously.

Picks