Vladimir Klepov as a Coder

How we made our pre-commit check 7x faster

As a guy who’s somewhat responsible for a large chunk of front-end development infrastructure at our company, I’ve spent the last couple of months woried about the performance of our pre-commit checks. We have around 50 projects on a standard react + typescript stack, and a corresponding set of pre-commit checks: eslint + stylelint + tsc + sometimes, jest. This suite was taking anywhere from 10s on a starter project to 50s on a monstrous app — not fun. I set out to fix this — and I did.

Cache your linters

The quick fix was to add --cache flag to eslint and stylelint calls. These tools process one file at a time, and caching makes them run very fast (around 1s for a normal commit instead of 10+). A quick github search makes me sad, because few people seem to do this. Also don’t forget to gitignore .stylelintcache and .eslintcache. Gain: 50 -> 30s.

Run the checks concurrently

Most checks were written like eslint src && stylelint src/**/*.css && tsc --noEmit — I assume the code was just being copied over. It’s a waste for multi-core developer machines, and has an extra drawback of being unusable on windows (I don’t think many front-end devs run windows, anyways). Making the checks run in parallel using concurrently or npm-run-all essentially makes the check run as fast as the slowest check — in our case, we were getting linters and jest for free, and tsc became the limiting factor. Gain: 30 -> 28s.

Cache tsc

tsc --noEmit sounds like the way to go if you run tsc to type-check your code, not to build anything. However, it was impossible to combine --noEmit with --incremental for a long time, leaving you with no caching and slow builds. Luckily, TS 4.0+ supports this combination — just drop an --incremental flag and save time. If you’re not ready to upgrade, a workaround exists — you want the check to be faster, not to write exactly zero files, don’t you? Gain: 28 -> 7s.

Do not break jest dependency detection

Lastly, I wanted to cover several ways to speed up jest if you happen to run it in your pre-commit (this is pretty rare). Obviously, you want to use jest --onlyChanged (or jest -o) to test only the files changed in the commit, not all the project. jest uses simple file-based dependency detection, no tree-shaking or anything — if you change file A, all the files that import A may have changed, and so on, and jest must run the tests for all the files that depend on A, too. You can work with this if you follow 2 rules:

  1. Do no import index.js inside your project — this erases granular change checks for individual modules re-exported via index. In the worst case, if you import from a root-level index, every change triggers all the tests.
  2. Break frequently changed files into smaller chunks. Granted, it’s good to use smaller modules in any case, but I bet you could start with your utils.js that contains 200 helpers. This will allow jest to make better guesses about what actually changed.

When pre-commit checks get slower, I see a lot of pressure to drop some checks and move them to CI. If you stick with slow checks instead, rest assured many developers will just --no-verify when commiting, which is probably not what you wanted to achieve. Lukily, you can easily make your pre-commit checks run in under 10 seconds:

  1. Use eslint --cache and stylelint --cache
  2. Run tsc with --incremental flag, or use a workaround for TS <4.0
  3. Parallelize the checks using concurrently or npm-run-all
  4. Use jest -o, don’t import index, and use smaller modules.

This can be done in 15 minutes, really. I’ve run some calculations for you — if you manage to strip 30s off your check time, assuming you make 5 commits a day and have a 3-person team (all this sound plausible), you’re saving your team 3 * 5 * 0.5 * 250 / 60 = 31 hours a year, that’s almost a week to spend better than waiting for pre-commit cheks. I really really hope you go and see if you can apply some of these techniques right now.

More? All articles ever
Written in by your friend, Vladimir. Follow me on Twitter to get post updates. I have RSS, too. And you can buy me a coffee!
Older
Cleaner ways to build dynamic JS arrays
Newer
useLayoutEffect is a bad place for deriving state