Signed extension loading time too long due to node_modules

[Jolg42]

Hi!

I recently signed a CEP extension with node activated and some node_modules and I was surprised by the time needed to load the extension in Photoshop 2015.5.1, 1 to 3 minutes on some computers at each Photoshop restart when opening the extension.

This is only happening on computers with PlayerDebugMode to 0

I looked at the node_modules folder, there are ~5000 files.

Do you know how to have a reasonable load time (less than 20 seconds)?

[undavide]

Hi Joël, in my experience this has happened when the node_modules folder contained also building tools (Gulp etc) – removing them solved the issue. But frankly I haven't too many dependencies in my own projects, so you might have run into a CEP limit...?

[bbb999]

Signature verification expands, with file count. Include those additional modules, judiciously. ;)

[venpa]

It's the known issue in CEP and planning to have few improvements in the next release. For now, you can consider reducing the number of files in the extension.

[Jolg42]

Thank you all for the help 😉

I reduced the number of files of node_modules to ~700 by checking everything and removing a lodash dependency from a dependency we have with npm-shrinkwrap.json (Lodash is +1000 files!) + filtering useless files (*.md, doc, tests & co).

It looks better but I need to check on more computers to be sure!

@venpa Good to know..! So is this improvement already in the Pre-Release version?

[Jolg42]

I did an homemade filtering but I just found https://github.com/KyleRoss/modclean that is doing the same but better.

[venpa]

@Jolg42 : We are discussing about what can be done to improve the performance of signing verification. At least, it takes 3 to 4 months to get into Prerelease builds if there is a better idea for signing verification.

[Jolg42]

@venpa Ok! Thank you for the details 👍

[johnlouis-swaine-axomic]

@Jolg42 Probably a bit late for you but using a framework like Webpack to consolidate your JS can massively improve this performance (10x +)

[Jolg42]

@johnlouis-swaine-axomic Yes! But only for the "front-end" part.

For Node.js dependencies, it's a lot harder :( My advice for Node.js dependencies is to select them carefully and to check sub-dependencies...! As 1 package can add a few thousand files sometimes!

I also use modclean to clean the node_modules folder (removings "test" folders & useless things).

[jingtaotan]

We recently had to solve this problem too, we had over 10,000 files in our node_modules modclean didn't really help much, because the number of "actual" .js files alone was pretty huge.

In the end, I used module-concat to concatenate all the files in node_modules together with my own JS files into 1 single JS file. And the ZXP signing and verification during ZXP installation/panel loading was much faster.

[Jolg42]

Your output file must be huge :D Thanks for sharing! Do you know if it's compatible with a Webpack or Browserify setup?

[jingtaotan]

Your output file must be huge

Our concatenated single JS file is 2MB in size, the signed ZXP only about 500KB.

I am not sure about using module-concat with Webpack and Browserify. I had in mind to try Webpack/Browserify but they both seemed too complex to setup for what I was trying to do.

In fact the author of module-concat says that he also did not want a heavy-weight approach like Webpack/Browserify. So maybe @Jolg42 you could give one or the other a try.

[Jolg42]

I'm already using Browserify actually, but I didn't try to require/concat node_modules with it. I don't know if this approach is compatible with any package though, like packages requiring binaries...

[zhangusc]

Does anyone have a solution/workaround? I have encountered the same problem. Loading extensions takes minutes as the extension contains thousands of files and the signature verification takes a long time.

[noamraby-ssky]

The only solution I found was to drastically reduce the amount of files. I used webpack but any bundler should do the job.

[zhangusc]

@ErinFinnegan Is it possible for the signing tool (ZxpSignCmd) to exclude a subfolder (e.g. a JS library) in the extension from signing?

[ErinFinnegan]

@zhangusc here's the answer from engineering:

As of now it is not possible to exclude a subfolder in signing using the ZXPSignCmd. All the contents of the input directory provided in signing are packaged inside the signed ZXP.

[lsh246]

For what it's worth, another bottleneck for load times, at least in InDesign, can be the structure of the ExtendScript (jsx) files. We were able to reduce our load times from ~30 seconds to ~2 seconds using the following two methods:

1) The larger the fully unfolded .jsx file pointed to in your manifest.xml file, the longer the load time. It's not just the size of the single .jsx file pointed to in the manifest file, but all the files included by that file. Optimizing that file by removing all unnecessary and redundant includes improved our load times a lot.

2) Compiling the .jsx file pointed to in the manifest file to .jsxbin, and pointing the manifest to that jsxbin file, also reduced our load times.