Packaging for universal mac fails due to pattern error

[lukehaas]

• electron-builder: 23.3.3
• node: 16.14.2
• electron: 19.0.8
• electron type: current
• target: mac universal

I'm encountering an error during the packaging step, it's failing with the following message:

pattern is too long  failedTask=build stackTrace=TypeError: pattern is too long

The issue started occurring when I added esbuild as a dependency - this is a prod dependency for my app. The error goes away if I either remove esbuild or keep it and remove other large dependencies, so it seems like I'm encountering some kind of size limit.

Here is the stack trace:

    at assertValidPattern (/MyProject/node_modules/minimatch/minimatch.js:279:11)
    at minimatch (/MyProject/node_modules/minimatch/minimatch.js:116:3)
    at handleFile (/MyProject/node_modules/asar/lib/asar.js:123:26)
    at next (/MyProject/node_modules/asar/lib/asar.js:148:11)
    at Object.module.exports.createPackageFromFiles (/MyProject/node_modules/asar/lib/asar.js:152:10)
    at Object.module.exports.createPackageWithOptions (/MyProject/node_modules/asar/lib/asar.js:40:25)
    at Object.exports.mergeASARs (/MyProject/node_modules/@electron/universal/src/asar-utils.ts:198:5)
    at exports.makeUniversalApp (/MyProject/node_modules/@electron/universal/src/index.ts:205:7)
    at MacPackager.doPack (/MyProject/node_modules/app-builder-lib/src/macPackager.ts:125:9)
    at MacPackager.pack (/MyProject/node_modules/app-builder-lib/src/macPackager.ts:179:7)
    at Packager.doBuild (/MyProject/node_modules/app-builder-lib/src/packager.ts:441:9)
    at Object.executeFinally (/MyProject/node_modules/builder-util/src/promise.ts:12:14)
    at Packager._build (/MyProject/node_modules/app-builder-lib/src/packager.ts:376:31)
    at Packager.build (/MyProject/node_modules/app-builder-lib/src/packager.ts:337:12)
    at Object.executeFinally (/MyProject/node_modules/builder-util/src/promise.ts:12:14)

[lukehaas]

I've looked a bit further into this error, it results from a maximum pattern limit in minimatch which is set to 1024 * 64. So the issue is that electron-builder is creating a glob pattern that exceeds a length of1024 * 64. I've opened an issue with minimatch to see if this limit can be increased: https://github.com/isaacs/minimatch/issues/173

But there could still be an issue here with glob patterns being larger than they should be.

[mmaietta]

Under Mac config, can you try mergeASAR as false? Looks like you're running into an issue with upstream npm package @electron/universal

[isaacs]

Yeah, 64kb is really excessively large, if you're running into this, I'd suspect something else earlier in the process has gone very off the rails. It's not impossible that you'd end up with a pattern that long, but it is definitely very strange.

[lukehaas]

Thanks, @mmaietta. setting mergeASARs to false worked. Closing as resolved.

[blv-rodrigoerades]

One more thank to @mmaietta. setting mergeASARs to false also did the trick for me. I was adding nodegit library

[itsPG]

Unfortunally, turing mergeASARs off makes x64ArchFiles option broken, (per documents says )

I'm still seeking the other workarounds.

[xieerduos]

Unfortunally, turing mergeASARs off makes x64ArchFiles option broken, (per documents says )

I'm still seeking the other workarounds.

Have you found a solution yet? I also encountered the same problem

[itsPG]

Unfortunally, turing mergeASARs off makes x64ArchFiles option broken, (per documents says ) I'm still seeking the other workarounds.

Have you found a solution yet? I also encountered the same problem

Unfortunately, I gave up building universal app for now.

[xieerduos]

Unfortunally, turing mergeASARs off makes x64ArchFiles option broken, (per documents says ) I'm still seeking the other workarounds.

Have you found a solution yet? I also encountered the same problem

Unfortunately, I gave up building universal app for now.

I just successfully packaged it, here is my configuration

add

    singleArchFiles: '*',
    x64ArchFiles: '*',

const extraResources = [
  {from: './icons', to: 'icons'},
  {
    from: './sdk',
    to: 'sdk',
    filter: process.env.VUE_APP_MAC_ARCH === 'universal' ? '${platform}/universal/**/*' : '${platform}/${arch}/**/*'
  },
];

{
  builderOptions: {
    extraResources,
    mergeASARs: process.env.VUE_APP_MAC_ARCH === 'universal' ? false : undefined,
    singleArchFiles: '*',
    x64ArchFiles: '*',
  }
}

[xieerduos]

Through debugging, it was found that the issue of files not being displayed is due to the need to ignore some files that are not necessary in the production environment. Setting mergeASARs to true can resolve the issue successfully.

The specific error file and code are: node_modules/@electron/universal/dist/cjs/asar-utils.js, line 154 of the code. By checking the output value of resolvedUnpack, it can be seen that the error 'pattern is too long' occurs when there are too many files.

        debug_1.d(`creating archive at ${outputAsarPath}`);
        const resolvedUnpack = Array.from(unpackedFiles).map((file) => path.join(x64Dir, file));
        let unpack;
        if (resolvedUnpack.length > 1) {
            unpack = `{${resolvedUnpack.join(',')}}`;
        }
        else if (resolvedUnpack.length === 1) {
            unpack = resolvedUnpack[0];
        }
        await asar.createPackageWithOptions(x64Dir, outputAsarPath, {
            unpack,
        });

The final solution is to add matching rules to files, ignoring files that are not needed in the production environment, for example:

   files: [
          '**/*',
          '!**/node_modules/*/{CHANGELOG.md,README.md,README,readme.md,readme}',
          '!**/node_modules/*/{test,__tests__,tests,powered-test,example,examples}',
          '!**/node_modules/*.d.ts',
          '!**/node_modules/.bin',
          '!**/*.{iml,o,hprof,orig,pyc,pyo,rbc,swp,csproj,sln,xproj}',
          '!.editorconfig',
          '!**/._*',
          '!**/{.DS_Store,.git,.hg,.svn,CVS,RCS,SCCS,.gitignore,.gitattributes}',
          '!**/{__pycache__,thumbs.db,.flowconfig,.idea,.vs,.nyc_output}',
          '!**/{appveyor.yml,.travis.yml,circle.yml}',
          '!**/{npm-debug.log,yarn.lock,.yarn-integrity,.yarn-metadata.json}',
          'node_modules/@breush/ffi-napi/**/*',
          'node_modules/@breush/ref-napi/**/*',
          '!node_modules/@breush/ffi-napi/prebuilds/**',
          '!node_modules/@breush/ffi-napi/deps/libffi/**',
          'node_modules/@breush/ffi-napi/deps/libffi/include/**',
          '!node_modules/@breush/ffi-napi/src/**',
          '!node_modules/@breush/ffi-napi/LICENSE',
          '!node_modules/@breush/ffi-napi/Dockerfile',
          '!node_modules/@breush/ref-napi/prebuilds/**',
          '!node_modules/@breush/ref-napi/Dockerfile',
          '!node_modules/@breush/ref-napi/LICENSE',
          '!node_modules/sqlite3/**',
          '!node_modules/sqlite3/node_modules/*/*/**.*',
          'node_modules/sqlite3/package.json',
          'node_modules/sqlite3/lib/**',
          'node_modules/sqlite3/deps/**',
          '!**/*/*.md'
        ]