Fix require time performance

[chris-morgan]

Describe the issue. Is it a bug or a feature request (new rule, new option, etc.)?

Loading stylelint (require('stylelint')) is very slow. It takes over a second for me when there’s nothing else installed, and when there are various other things installed it increases (e.g. two seconds).

I care about this because in a Makefile world where you spin up the tooling for each file, this is a >1 second delay per file which rapidly changes from unpleasant to awful.

Steps to reproduce:

$ cd $(mktemp -d)
$ yarn add stylelint
[…]
$ node -e "s=Date.now();require('stylelint');console.log(Date.now()-s)"
1183
$ node -e "s=Date.now();require('stylelint');console.log(Date.now()-s)"
1128
$ node -e "s=Date.now();require('stylelint');console.log(Date.now()-s)"
1155

Which rule, if any, is this issue related to?

n/a

What CSS is needed to reproduce this issue?

n/a

What stylelint configuration is needed to reproduce this issue?

n/a

Which version of stylelint are you using?

7.9.0

How are you running stylelint: CLI, PostCSS plugin, Node API?

n/a

Does your issue relate to non-standard syntax (e.g. SCSS, nesting, etc.)?

n/a

What did you expect to happen?

require('stylelint') to be fast (like, well under 100ms).

What actually happened (e.g. what warnings or errors you are getting)?

It took a long time to import stylelint, a second or two.

[davidtheclark]

@chris-morgan Can you contrast this with other programs of similar complexity? Is there anything actionable that can be done?

[chris-morgan]

@davidtheclark Well, here are a bunch of other things I was trying with postcss:

postcss: 109 postcss-cssnext: 67 postcss-custom-properties: 3 postcss-at-rules-variables: 2 postcss-conditionals: 21 postcss-clean: 204 autoprefixer: 267 stylelint: 1887

And I consider postcss-clean and autoprefixer to be far too slow (over 200ms) as well. Seriously, 100ms is a long time.

I don’t know why stylelint is so slow at present, but it is very slow and must be doing something that it shouldn’t be!

[davidtheclark]

@chris-morgan Can you compare it to a module of similar size? I imagine stylelint involves many, many more files than any of those modules you've listed.

[chris-morgan]

Putting a bit of instrumentation into require I see that the 175 rules, each in their own module/file, consume the largest fraction of the time. I imagine there’s quite a bit of overhead in loading the modules. Rolling all of stylelint up into one file for the npm distribution seems to me to be a reasonable solution, or remedy at least (I don’t know how long it will take to import after that, we can reassess after that). Hundreds of files is simply terrible for performance.

[hudochenkov]

@davidtheclark I compared with eslint using the method above:

stylelint	eslint
959	262
1021	280
853	289
880	276
857	297

eslint is pretty big. Maybe we have slow dependencies, or there are other places to improve the speed.

[chris-morgan]

Rolling the ~275 files of stylelint into one with rollup.js produced a saving of about one millisecond per file, halving the loading time spent in stylelint. That then leaves stylelint’s dependencies unrolled; I estimate another 150ms could be saved there if they were to roll themselves up. (Fun fact: cosmiconfig adds over 100ms of loading time, just by itself; its dependency js-yaml is around 80ms of that. A few others are almost as bad.)

A 40–50% reduction in load time purely from rolling everything up is not to be sneezed at.

After doing this, stylelint would still be far too slow to start for my liking, but it’s a start, at the least.

[hudochenkov]

@chris-morgan how did you measure dependencies loading time? Like you did with cosmicconfig. I would like to learn :)

[chris-morgan]

@hudochenkov A very hacked-together script which produces a kind-of-back-to-front tree. I think I’ll make a nicer version of this which does a bit of analysis, it’ll do nicely as my first package to go on npm.

var orig = require.extensions['.js'];
var level = 0;
var cwd = process.cwd();
var toPrint = [];
require.extensions['.js'] = function (module, path) {
    if (path.startsWith(cwd)) {
        path = '.' + path.substr(cwd.length);
    }
    var start = Date.now();
    level++;
    var out = orig.apply(this, arguments);
    level--;
    toPrint.push('│ '.repeat(level) + '┌ ' + path + ' \x1b[32m' + (Date.now() - start) + '\x1b[m');
    if (!level) {
        console.log(toPrint.join('\n'));
        toPrint.splice(0);
    }
    return out;
}

require('stylelint');  // Or ./stylelint-bundle.js after generating it

Then I just worked it over a bit in Vim with regular expressions, :sort n, :%!uniq and other things like that.

[hudochenkov]

master	v8
837	694
838	699
822	694
817	690
869	678

v8 is faster because we eliminated bunch of rules depended on third-party modules.

[alexander-akait]

@hudochenkov What tool do you use to output the time? :smile:

[hudochenkov]

@evilebottnawi $ node perf.js :)

console.time('stylelint');
const stylelint = require('./stylelint');
console.timeEnd('stylelint');

[alexander-akait]

@hudochenkov Thanks) I just thought that this could be some kind of other tool, on the hooks require for example :smile:

[alexander-akait]

@hudochenkov also we can try using lazy require for some modules, but it is need investigation

[jeddy3]

Ok, so what's actionable here?

The good news is v8 is only about 2.5x slower than eslint (a similarly sized project), but it sounds like there are two possible areas of improvement: the requiring of our individual source files and our dependencies (in particular, autoprefixer and js-yaml (within cosmiconfig)).

We could try rolling-up the code as part of the publishing process. What I'd like to know is will this complicate using stylelint as a pinned-github-tag dependencies. One of the reasons we target node@4, and no longer transpile our code, is the simplify that process.

autoprefixer includes the caniuse database, which accounts for a significant chunk of stylelint's weight. There's an upcoming PR to migrate autoprefixer's dependency browserlist to caniuse-lite. I suggest waiting to see if that helps.

Regarding js-yaml, perhaps we could open an issue there to see if there's anything that can do about their require time?

Does any performance-orientated person want to pick this up?

[ben-eb]

@jeddy3 caniuse-lite is much smaller on disk, though it remains to be seen if it will improve performance. I hope that it does, but of course there is a small overhead to unpacking the data when the module is run. The module is optimised for size not performance (indeed, it's about 7 times smaller).

• https://travis-ci.org/postcss/autoprefixer/jobs/219404429 - caniuse-db
• https://travis-ci.org/postcss/autoprefixer/jobs/220033486 - caniuse-lite

At the very least I hope we can start a conversation about providing alternative ways to consume caniuse data through subsets like this one.

[davidtheclark]

After reading https://pouchdb.com/2016/01/13/pouchdb-5.2.0-a-better-build-system-with-rollup.html and https://medium.com/@Rich_Harris/how-to-not-break-the-internet-with-this-one-weird-trick-e3e2d57fee28, I'm thinking that the only way to solve this would be to create a bundle with Rollup, and that might be just fine, or at least worth an experiment. Because we wouldn't be transpiling any code (except imports/exports), just concatenating, the built file would still be legible and debuggable (just long).

What I'd like to know is will this complicate using stylelint as a pinned-github-tag dependencies.

I don't know of a way to ease this — but I also don't know if it's worth worrying about too much.

So, when somebody has the time and wants to take a shot at this problem, I suggest looking codemodding to ES2015 module syntax and using Rollup.

[m-allanson]

I experimented with this the other day, we should be able to use rollup-plugin-commonjs to avoid having to change any existing stylelint code.

This is a rollup config that worked for me on stylelint/master:

"use strict"
const commonjs = require("rollup-plugin-commonjs")
const json = require("rollup-plugin-json")

module.exports = {
  entry: "lib/index.js",
  dest: "dist/index.js",
  moduleName: "stylelint",
  format: "cjs",
  plugins: [
    commonjs(),
    json(),
  ],
}

Here are the before and after results of requiring the bundled and unbundled stylelint 5 times each:

node -e "s=Date.now();require('./lib/index.js');console.log(Date.now()-s)"
800
749
741
762
731
===
mean 756

node -e "s=Date.now();require('./dist/index.js');console.log(Date.now()-s)"
613
673
611
620
665
===
mean 636

However I was unable to succesfully rollup all stylelint's dependencies into one file (using rollup-plugin-node-resolve, the resulting bundle would error when run. I assume that would bring further improvements.

[jeddy3]

I experimented with this the other day, we should be able to use rollup-plugin-commonjs to avoid having to change any existing stylelint code.

That's awesome.

Here are the before and after results of requiring the bundled and unbundled stylelint 5 times each

100ms improvement isn't too shabby. I think we should roll with it.

However I was unable to succesfully rollup all stylelint's dependencies into one file (using rollup-plugin-node-resolve, the resulting bundle would error when run. I assume that would bring further improvements.

I think we can definitely proceed with rolling up our files for now, and then look to this as an additionally improvement.

[Bigdragon13th]

I experience this issue as well, I'm using styelint with gulp and it's very slow. You can see my time from time-require here

Start time: (2017-06-23 09:15:03 UTC) [treshold=1%,sorted]
 # [order]  module                                                   time  %
 1 [10299]  /Users/big/Documents/www/c...-boilerplate/gulpfile.js)  44.7s  ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 73%
 2  [8425]  gulp-stylelint (node_modul...-stylelint/dist/index.js)   9.1s  ▇▇▇▇▇▇▇▇▇ 15%
 3  [8396]  stylelint (node_modules/stylelint/lib/index.js)          8.8s  ▇▇▇▇▇▇▇▇ 14%
 4 [10259]  gulp-pleeease (node_modules/gulp-pleeease/index.js)      6.1s  ▇▇▇▇▇▇ 10%
 5 [10221]  pleeease (node_modules/pleeease/lib/pleeease.js)         5.9s  ▇▇▇▇▇▇ 10%
 6 [10216]  ../lib/processors (node_mo...eeease/lib/processors.js)   5.8s  ▇▇▇▇▇▇ 10%
 7  [7862]  ./utils/checkAgainstRule (...tils/checkAgainstRule.js)   5.8s  ▇▇▇▇▇▇ 9%
 8  [7860]  ../normalizeRuleSettings (...normalizeRuleSettings.js)   5.8s  ▇▇▇▇▇▇ 9%
 9  [7859]  ./rules (node_modules/stylelint/lib/rules/index.js)      5.7s  ▇▇▇▇▇▇ 9%
10  [9096]  gulp-autoprefixer (node_mo...lp-autoprefixer/index.js)   4.9s  ▇▇▇▇▇ 8%
11  [9094]  autoprefixer (node_modules...ixer/lib/autoprefixer.js)   4.8s  ▇▇▇▇▇ 8%
12  [8879]  browserslist (node_modules/browserslist/index.js)        3.5s  ▇▇▇▇ 6%
13  [8877]  caniuse-lite (node_modules...e/dist/unpacker/index.js)   3.4s  ▇▇▇ 6%
14 [14140]  ./list (node_modules/bower/lib/commands/list.js)         3.4s  ▇▇▇ 6%
15 [14136]  ../core/Project (node_modu...ower/lib/core/Project.js)   3.4s  ▇▇▇ 5%
16  [8876]  ../../data/features (node_...se-lite/data/features.js)   3.3s  ▇▇▇ 5%
17 [14126]  ./Manager (node_modules/bower/lib/core/Manager.js)       3.2s  ▇▇▇ 5%
18  [1750]  gulp-awspublish (node_modu...-awspublish/lib/index.js)   3.2s  ▇▇▇ 5%
19 [14114]  ./PackageRepository (node_...ore/PackageRepository.js)     3s  ▇▇▇ 5%
20  [1693]  aws-sdk (node_modules/aws-sdk/lib/aws.js)                2.8s  ▇▇▇ 5%
21  [4886]  babelify (node_modules/babelify/index.js)                2.8s  ▇▇▇ 5%
22  [4884]  babel-core (node_modules/babel-core/index.js)            2.8s  ▇▇▇ 5%
23  [4883]  ./lib/api/node.js (node_mo...bel-core/lib/api/node.js)   2.7s  ▇▇▇ 4%
24  [4859]  ../transformation/file (no...sformation/file/index.js)   2.7s  ▇▇▇ 4%
25   [358]  gulp (node_modules/gulp/index.js)                        2.3s  ▇▇▇ 4%
26  [3772]  debowerify (node_modules/debowerify/index.js)            2.2s  ▇▇ 4%
27  [3766]  bower (node_modules/bower/lib/index.js)                  2.1s  ▇▇ 3%
28  [2343]  hipchat-notify (node_modules/hipchat-notify/index.js)      2s  ▇▇ 3%
29 [14112]  ./resolverFactory (node_mo.../core/resolverFactory.js)     2s  ▇▇ 3%
30 [14110]  ./resolvers (node_modules/.../core/resolvers/index.js)     2s  ▇▇ 3%
31  [2340]  request (node_modules/request/index.js)                    2s  ▇▇ 3%
32  [1135]  ./node_loader (node_module...s-sdk/lib/node_loader.js)   1.8s  ▇▇ 3%
33  [2339]  ./request (node_modules/request/request.js)              1.7s  ▇▇ 3%
34  [2873]  gulp-eslint (node_modules/gulp-eslint/index.js)          1.7s  ▇▇ 3%
35  [1096]  ./core (node_modules/aws-sdk/lib/core.js)                1.7s  ▇▇ 3%
36 [14075]  ./GitHubResolver (node_mod...olvers/GitHubResolver.js)   1.7s  ▇▇ 3%
37  [2867]  eslint (node_modules/eslint/lib/api.js)                  1.6s  ▇▇ 3%
38  [3999]  watchify (node_modules/watchify/index.js)                1.6s  ▇▇ 3%
39  [3996]  chokidar (node_modules/chokidar/index.js)                1.5s  ▇▇ 3%
40  [3626]  mout (node_modules/mout/index.js)                        1.4s  ▇▇ 2%
41  [3068]  browserify (node_modules/browserify/index.js)            1.4s  ▇▇ 2%
42   [996]  ./xml/builder (node_module...s-sdk/lib/xml/builder.js)   1.3s  ▇▇ 2%
43 [10031]  autoprefixer (node_modules...ixer/lib/autoprefixer.js)   1.3s  ▇▇ 2%
44   [995]  xmlbuilder (node_modules/xmlbuilder/lib/index.js)        1.3s  ▇▇ 2%
45  [4586]  babel-helpers (node_module...bel-helpers/lib/index.js)   1.2s  ▇▇ 2%
46  [6100]  ./at-rule-no-vendor-prefix...o-vendor-prefix/index.js)   1.2s  ▇▇ 2%
47  [6095]  ../../utils/isAutoprefixab...tils/isAutoprefixable.js)   1.2s  ▇▇ 2%
48  [5648]  gulp-less (node_modules/gulp-less/index.js)              1.2s  ▇▇ 2%
49  [4585]  ./helpers (node_modules/babel-helpers/lib/helpers.js)    1.2s  ▇▇ 2%
50  [4584]  babel-template (node_modul...el-template/lib/index.js)   1.1s  ▇▇ 2%
51   [184]  gulp-util (node_modules/gulp-util/index.js)              1.1s  ▇ 2%
52   [357]  vinyl-fs (node_modules/vinyl-fs/index.js)                  1s  ▇ 2%
53  [1692]  ../clients/all (node_modules/aws-sdk/clients/all.js)    996ms  ▇ 2%
54  [9035]  ./prefixes (node_modules/a...prefixer/lib/prefixes.js)  989ms  ▇ 2%
55  [9198]  pixrem (node_modules/pixrem/lib/pixrem.js)              986ms  ▇ 2%
56  [2618]  ./eslint (node_modules/eslint/lib/eslint.js)            973ms  ▇ 2%
57  [8108]  ./testUtils/createRuleTest...tils/createRuleTester.js)  970ms  ▇ 2%
58  [2456]  gulp-modernizr (node_modules/gulp-modernizr/index.js)   943ms  ▇ 2%
59  [8341]  ./formatters (node_modules.../lib/formatters/index.js)  928ms  ▇ 2%
60  [5260]  gulp-uglify (node_modules/gulp-uglify/index.js)         911ms  ▇ 1%
61  [4581]  babel-traverse (node_modul...el-traverse/lib/index.js)  904ms  ▇ 1%
62 [14073]  ../../util/extract (node_m...ower/lib/util/extract.js)  888ms  ▇ 1%
63   [994]  ./XMLBuilder (node_modules...uilder/lib/XMLBuilder.js)  881ms  ▇ 1%
64  [3995]  ./lib/fsevents-handler (no.../lib/fsevents-handler.js)  877ms  ▇ 1%
65  [3994]  fsevents (node_modules/fsevents/fsevents.js)            862ms  ▇ 1%
66  [4557]  ./path (node_modules/babel...averse/lib/path/index.js)  858ms  ▇ 1%
67  [2448]  gulp-util (node_modules/gu...dules/gulp-util/index.js)  833ms  ▇ 1%
68   [311]  ./lib/src (node_modules/vinyl-fs/lib/src/index.js)      813ms  ▇ 1%
69  [3988]  node-pre-gyp (node_modules...-gyp/lib/node-pre-gyp.js)  804ms  ▇ 1%
70  [1846]  gulp-scp2 (node_modules/gulp-scp2/index.js)             801ms  ▇ 1%
71  [5647]  /Users/big/Documents/www/c...de_modules/less/index.js)  780ms  ▇ 1%
72  [8336]  ./stringFormatter (node_mo...tters/stringFormatter.js)  779ms  ▇ 1%
73  [5646]  ./lib/less-node (node_modu...s/lib/less-node/index.js)  771ms  ▇ 1%
74 [13972]  ../../util/download (node_...wer/lib/util/download.js)  764ms  ▇ 1%
75  [9192]  postcss (node_modules/pixr...s/postcss/lib/postcss.js)  763ms  ▇ 1%
76  [7059]  ./no-browser-hacks (node_m...o-browser-hacks/index.js)  755ms  ▇ 1%
77  [7058]  stylehacks (node_modules/stylehacks/dist/index.js)      742ms  ▇ 1%
78 [13966]  request (node_modules/bowe...modules/request/index.js)  739ms  ▇ 1%
79  [5247]  ./composer (node_modules/gulp-uglify/composer.js)       719ms  ▇ 1%
80  [1839]  scp2 (node_modules/scp2/index.js)                       707ms  ▇ 1%
81  [5246]  ./lib/minify (node_modules/gulp-uglify/lib/minify.js)   705ms  ▇ 1%
82 [13965]  ./request (node_modules/bo...dules/request/request.js)  695ms  ▇ 1%
83  [1838]  ./lib/scp (node_modules/scp2/lib/scp.js)                695ms  ▇ 1%
84  [7250]  ./no-indistinguishable-col...uishable-colors/index.js)  687ms  ▇ 1%
85  [8328]  table (node_modules/stylel...ules/table/dist/index.js)  683ms  ▇ 1%
86  [7249]  colorguard (node_modules/colorguard/index.js)           677ms  ▇ 1%
87  [7248]  ./lib/colorguard (node_mod...rguard/lib/colorguard.js)  652ms  ▇ 1%
88  [2855]  ./cli-engine (node_modules/eslint/lib/cli-engine.js)    628ms  ▇ 1%
89  [1837]  ./client (node_modules/scp2/lib/client.js)              626ms  ▇ 1%
90   [993]  ./XMLElement (node_modules...uilder/lib/XMLElement.js)  626ms  ▇ 1%
91 [13640]  bower-registry-client (nod...egistry-client/Client.js)  623ms  ▇ 1%
92  [8255]  ./createStylelint (node_mo...t/lib/createStylelint.js)  621ms  ▇ 1%
Total require(): 27647
Total time: 1m 1.1s

As you can see, Stylelint is top of the list here.

[alexander-akait]

@Bigdragon13th maybe your don't ignore some files (tests, examples, build directory and etc stuff)?

[Bigdragon13th]

@evilebottnawi It's not runtime that is slow, it's the loading time stylelint use when we do require("stylelint") that is very slow.

[Mottie]

Has anyone successfully bundled Stylelint using rollup? I created a stylelint-bundler repo reporting my attempts, work-arounds and failure at making a stand-alone version of Stylelint. Maybe someone with more experience with rollup can help? Paging @bfred-it 😉.

Also, the namedColorData file is 61KB. I think all the func definitions aren't necessary as conversion between the color formats shouldn't be that difficult.

[fregante]

@Mottie I gave up on rollup for complex tasks. Try webpack or backpack https://github.com/jaredpalmer/backpack/

[jeddy3]

@Mottie Big thanks for making a start on attempting to bundle stylelint using rollup, and for documenting your progress in your repo. It looks to be a tricky one!

Has anyone successfully bundled Stylelint using rollup?

I don't believe any of the core team have had time to look into this, and so any progress you make would be greatly appreciated. Good performance is one of the goals of stylelint.

I understand that prettier uses rollup for their build. Perhaps you'll find some inspiration there to overcome the blockers you've encountered.

Also, the namedColorData file is 61KB. I think all the func definitions aren't necessary as conversion between the color formats shouldn't be that difficult.

Agreed. Feel free to contribute that optimisation. You can use the built-in benchmarking tools to gauge the performance impact on the rules that use this data.

[m-allanson]

This may be useful for anyone that wants to dig in to this issue: https://blog.sqreen.io/diy-node-apm/

And https://github.com/davidmarkclements/0x

[fregante]

Here's a node-specific way to speed up exactly this: load times caused by lots of requires

https://github.com/dominictarr/noderify

[jeddy3]

There's a new compiler/bundler on the block, ncc. Perhaps we'll have more success with that as:

How Is This Different from Webpack, Rollup, Parcel?

We wanted to focus on a development experience that maximizes productivity, followed the semantics of the Node.js platform out of the box and mirrored the conventions of other well-designed and battle-tested languages.

This means that it should do the right thing out of the box, with no extra configuration.

[ntwb]

Neat, Zeit continue to produce fantastic products.

That said, it's very shiny, 14 days.

For sure it would be interesting to see how stylelint would package up using ncc along with some performance testing.

[vankop]

I don't think this issue still relevant.

[chris-morgan]

Nope, not fixed at all. On my Surface Book, using Node 8.10.0 on Ubuntu/WSL, I’m seeing best-case figures of 423ms in best-performance mode, and 1011ms in default mode (when on battery). eslint also shows poorly at 184ms and 422ms.

Even your figures of 257ms and 210ms I still consider to be very slow for importing things like this. It’s a big improvement over times more than a second, but as I mentioned, I expect import times well under 100ms.

[vankop]

So lets deep dive:

For me on Node.js 10 LTS OSX running script below in repo root:

cat package.json | jq -r '.dependencies | keys | @tsv' | node -e "require('fs').readFileSync('/dev/stdin', 'utf-8').split('\t').map(a => a.trim()).forEach(a => {console.time(a);require(a);console.timeEnd(a);})"

it results in (after several runs in loop):

autoprefixer: 214.679ms
balanced-match: 0.448ms
chalk: 0.117ms
cosmiconfig: 28.108ms
debug: 2.184ms
execall: 0.934ms
file-entry-cache: 11.034ms
get-stdin: 0.745ms
global-modules: 4.689ms
globby: 78.102ms
globjoin: 0.312ms
html-tags: 2.088ms
ignore: 0.702ms
import-lazy: 0.381ms
imurmurhash: 0.553ms
known-css-properties: 0.827ms
leven: 0.334ms
lodash: 17.709ms
log-symbols: 1.176ms
mathml-tag-names: 0.425ms
meow: 28.433ms
micromatch: 7.088ms
normalize-selector: 0.936ms
postcss: 0.293ms
postcss-html: 10.530ms
postcss-jsx: 80.648ms
postcss-less: 2.009ms
postcss-markdown: 45.007ms
postcss-media-query-parser: 1.145ms
postcss-reporter: 1.449ms
postcss-resolve-nested-selector: 0.276ms
postcss-safe-parser: 0.836ms
postcss-sass: 23.917ms
postcss-scss: 4.282ms
postcss-selector-parser: 17.014ms
postcss-syntax: 0.950ms
postcss-value-parser: 0.038ms
resolve-from: 0.261ms
signal-exit: 0.035ms
slash: 0.244ms
specificity: 0.439ms
string-width: 1.031ms
strip-ansi: 0.101ms
style-search: 0.345ms
sugarss: 2.752ms
svg-tags: 0.881ms
table: 46.021ms

So looks like stylelint dependencies loading to long. List of long loading deps:

autoprefixer ~ 200ms
globby ~ 80ms
postcss-markdown ~ 45ms
postcss-jsx ~ 80ms
table ~ 45ms
cosmiconfig ~ 30ms

[vankop]

postcss-markdown loading only if syntax set to markdown postcss-jsx loading only if sytax set to css-in-js autoprefixer loads only in rules globby loads table it loads, but does not presented in require.cache 🙈 cosmiconfig loads

List of loaded modules after require('stylelint'): https://gist.github.com/vankop/a2496e6bc28c705f42dbd1f3da05fa64

All timings: https://gist.github.com/vankop/f83bdeda2cae617b4a801d8e210454ef script: https://gist.github.com/vankop/f83bdeda2cae617b4a801d8e210454ef#file-timing-js

[vankop]

Actually, I don't know what to do more =) Maybe we can use something instead of globby and load lazily micromatch then ( currently stylelint and fast-glob has dependency on it, fast-glob is dependency of globby)

micromatch and its deps take 50ms to load (globby - micromatch ~20ms)

[vankop]

Wow, looks like it is problem of micromatch@3 https://github.com/micromatch/micromatch/issues/137

In version 4 it loads ~10ms

[vankop]

https://github.com/stylelint/stylelint/pull/4254 should solve problem with micromatch

Applying ☝️this speeds up loading by 50ms

[ntwb]

4254 should solve problem with micromatch

If someone wants to help get the Windows issues fixed, we can merge it.

I've had a couple of go's at this and not got far myself.

[vankop]

eslint uses v8-compile-cache, maybe try to use it in stylelint also https://github.com/eslint/eslint/pull/11921

[alexander-akait]

@vankop good idea, let's do it

[vankop]

After all PR provided here (without v8-compile-cache) script provided here loads ~/lib/index for me in 140ms 🥳 🚀 (Previously 260ms) Using v8-compile-cache I hope will speed up stylelint to ~120ms maybe less

[jeddy3]

@vankop Thanks for looking into this.

Nope, not fixed at all. It’s a big improvement over times more than a second, but as I mentioned, I expect import times well under 100ms

Yes, I think there is still work to do on this. If we compare three similar tools:

$ node -e "s=Date.now();require('stylelint');console.log(Date.now()-s)"
312
$ node -e "s=Date.now();require('eslint');console.log(Date.now()-s)"
159
$ node -e "s=Date.now();require('prettier');console.log(Date.now()-s)"
91

Prettier is striking just under 100ms. I think that should be our target too. Prettier has vastly more contributors and resources than us, but we can hopefully borrow from their approach to get us somewhere near their performance.

Actually, I don't know what to do more

I suspect the next step is to bundle/roll-up things for our published package. This was suggested way back, but we haven't succeeded at it yet. We can now look to how prettier does it for inspiration. It appears they create a number of bundles. For example, there are ones for their standalone, bin and each parser:

// Contents of Prettier's published package
bin-prettier.js
doc.js
index.js
parser-angular.js
parser-babylon.js
parser-flow.js
parser-glimmer.js
parser-graphql.js
parser-html.js
parser-markdown.js
parser-postcss.js
parser-typescript.js
parser-yaml.js
standalone.js
third-party.js
LICENSE
package.json
README.md

We'll want to keep the code base node@6 compatible (e.g. by using commonjs requires) so that stylelint can be installed directly from GitHub without a compilation step. I think optimising a build for the published package should be considered as an enhancement.

I think it's an interesting piece of work for anyone who has the time.

[vankop]

@jeddy3 After playing with bash have calculated some interesting stats: For stylelint command (i think OSX only): find -E lib -type f -name "*.js" -not -regex '(.*\.test\.js)|(.*__tests__.*)' | xargs wc -l Results in 31126 LOCs find -E lib -type f -regex 'lib/rules/.*/[^/]*\.js' -not -regex '.*__tests__.*' | xargs wc -l Results in 18092 LOCs For prettier command: find src -name '*.js' | xargs wc -l Results in 26931 LOCs

So I think, it could be some optimizations still without bundling =) Since rules loads on demand

[jeddy3]

it could be some optimizations still without bundling =)

That's good to know. What did you have in mind?

[vankop]

I will create PR to cosmiconfig to load js-yaml lazily, too.

[jimmywarting]

gosh, just take a look at this dependency graph https://npm.anvaka.com/#/view/2d/stylelint

[XhmikosR]

I think after the recent changes, things should improve a bit more. Not much, but maybe a bit.

Can someone compare v12.0.0 and latest master please? And perhaps repost the instructions/script? :)

[vankop]

Using script https://gist.github.com/vankop/f83bdeda2cae617b4a801d8e210454ef#file-timing-js

Updated timings: https://gist.github.com/vankop/30f51e6cd07e5c97a52f98c5a69b6be7

and we still need to update to globby@10

[jeddy3]

gosh, just take a look at this dependency graph

That's a rather good visualisation.

I think after the recent changes, things should improve a bit more. Not much, but maybe a bit.

Good stuff.

I feel like this will be an ongoing endeavor to find further optimisations. There are likely areas where we can improve our performance by seeking more streamlined 3rd party packages.

[jeddy3]

I'm seeing some promising results from bundling with ncc. I'm consistently getting a require-time of around 80ms when I:

• bundle out the syntaxes separately
• exclude
  • vendor-prefix rules
  • function-calc-no-invalid
  • lodash

This is comparable with Prettier's bundled require-time performance. As is the minified bundle size of 1mb.

Without the above exclusions, I'm getting a minified bundle-size of 3.5mb and require-time of 120ms. Our dependencies account for nearly 90% of the bundle, with the syntaxes, lodash and autoprefixer being responsible for the majority of it.

You can view this yourself by running the following command in the stylelint repository and dragging the generated /dist/stats.json onto this visualizer webpage:

npx @zeit/ncc build lib/index.js -o dist --stats-out dist/stats.json

To improve the performance of stylelint and support running it in the browser (https://github.com/stylelint/stylelint/issues/3935) we'll need to:

• adopt single-function lodash modules
• move the vendor prefix rules to a plugin
• perhaps move the function-calc-no-invalid rule (and other validate rules) to a plugin
• bundle stylelint (with the syntaxes bundled separately), like Prettier does

The latter three are breaking changes. I suggest we create a next branch that we can publish as stylelint@next.

In hindsight, we should have removed the vendor prefix rules in 8.0.0, when we removed all the other rules that were thin wrappers across other tools.

I'll create separate issues for everything but the bundling, which can be discussed here.

[alexander-akait]

move the vendor prefix rules to a plugin

Like property-no-vendor-prefix?

perhaps move the function-calc-no-invalid rule (and other validate rules) to a plugin

function-calc-no-invalid is very bad, it is not supported some CSS syntaxes, it was taken from postcss-calc and you can look on issues and problems

[jeddy3]

@evilebottnawi I've referenced your comments in https://github.com/stylelint/stylelint/issues/4730 and https://github.com/stylelint/stylelint/issues/4731, respectively. Let's continue the specific discussions there.