pkgroll bundler output: consider using 'export default' for internal functions to bundle them instead of export them

[nathanjhood]

issue:

• using export = x with import x = require('path/to/x') is correct, for running exported scripts and CLI in NodeJS environment
• for any internal helper functions, declared as individual modules, this syntax prevents them from being bundled where they are being used
• this means that I'd be looking at adding a package.json#exports for every helper function - parseEnv and so forth...
• this is misleading; those helpers should be exposed to consumers, who may wish to tweak or build their own scripts with them, but they are part of the esbuild-scripts library, so should probably just be bundled in the final index.js for consumers

solution:

• use export = x with import x = require('path/to/x') syntax for cli.ts, build.ts, start.ts, test.ts, and other executable scripts
• use export { x as default, x } or just export default x with import {x} from 'path/to/x' or just import x from 'path/to/x' for eveything else, such as parseEnv, getBuildOptions, etc

blockers:

• not totally sure of the final architectural requirements until I've tried the scripts on some actual projects (beware premature optimisation)

plan:

• get at least one script (build is almost done) up and running to a basic level, and try some soft deploy testing locally to see how the bundled esbuild-scripts package behaves
• the above presents multiple testing points, such as being consumed from the <projectRoot>/node_modules/@nathanjhood/esbuild-scripts directory with the CLI, or from the <projectRoot>/scripts directory without a CLI
• make described changes as necessary; another test can be added per file to check the required import syntax, if not already in place (also: consider doing so in an index.test.ts)

[nathanjhood]

Steps taken:

• got build script up and running; CLI sets up the process and args, then spawns the build script as child process
• as a mock-up, this script was able to build the esbuild-scripts project (i.e., itself), so I moved on to consumer testing
• by being smart ;) I have actually been developing this project locally in a sub-directory of a bare-bones React app
• I moved upwards into this consuming project and tried the first attempting at consuming build via the CLI, as intended

Result:

• suspicions about bundler output, highlighted in the opening of this issue, are a confirmed issue

Overview:

in a hypothetical, new React project (as created by, for example, the npm create-react-app package), here called "consumer":

// package.json
{
  "name": "consumer",
  "dependencies": {
  "@nathanjhood/esbuld-scripts": "./packages/@nathanjhood/esbuld-scripts",
  "react": "18.2.0",
  "react-dom": "18.2.0",
  // etc... 
  },
  "scripts": {
  "build": "esbuild-scripts build"
  }
}

The esbuild-scripts repository is located at <projectRoot>/packages/@nathanjhood/esbuld-scripts, as described in the above package.json. It has been pre-built, meaning that it already contains it's own node_modules and dist/*.js, in order to isolate any issues with the underlying mechanisms before worrying too much about where to find the files.

In the "consumer" project root directory, call the aliased esbuild-scripts build command via Yarn, observe output result:

$ yarn build
Starting @nathanjhood/esbuild-scripts v0.1.0 
┊
├──┐  
/***/consumer
[ '/***/.config/nvm/versions/node/v20.16.0/bin/node' ]
[ 'build' ]
[ '/***/.config/nvm/versions/node/v20.16.0/bin/node' ]
[ 'build' ]
│  │  
│  └─ Recieved Script: build
│     │  
node:internal/modules/cjs/loader:1148
  throw err;
  ^

Error: Cannot find module '../config/esbuild/getBuildOptions'
Require stack:
- /***/consumer/node_modules/@nathanjhood/esbuild-scripts/dist/pkgroll_create-require-tKTPlRJW.js
- /***/consumer/node_modules/@nathanjhood/esbuild-scripts/dist/scripts/build.js
    at Module._resolveFilename (node:internal/modules/cjs/loader:1145:15)
    at Module._load (node:internal/modules/cjs/loader:986:27)
    at # etc....

Node.js v20.16.0
│     └─ build time taken: 499.561ms
│  
├─ @nathanjhood/esbuild-scripts time taken: 578.619ms
┊
Done in 1.47s.

It would be wise to continue development of esbuild-scripts from this point forward from the context of a consumer project.

Tests should only be developed in the context of the top-level project; thus I'll need to probably generate some very basic tests in the esbuild-scripts project context, and then dive over to the "consumer" project to see which further issues might arise - and need tests for.

[nathanjhood]

Update:

The good:

• got the CLI exporting to the "consumer" node_modules/bin directory with a symlink, when running yarn install

see: esbuild-scripts is listed

The bad:

• As per suspicion, everything in our package.json#bin is also getting symlinked - all the helper functions, as shown

see: `getClientis also listed, and so isbuild`*

The interesting:

• those helpers were all written to be able to be executed directly - really, just for dev/debugging (was planning to strip the executeability from these before declaring a v0.0.1). This means that in "consumer", I am actually able to run this command:

$ yarn getClientEnvironment

The unfortunate:

• the above command(s) fail due to wrong path imports

So we've arrived at an interesting and unforseen choice;

1. we can allow the helpers to be executable for consumers by keeping the current import/export syntax, but flattening the project directories such that all exectables are in one, flat directory, fixing all import path resolutions
2. we can instead resort to the OG plan by only symlinking the CLI part into node_modules/bin, and having it bundle in all the helper functions it needs by changing the import/export syntaxes, and removing the helpers from package.json#bin

Rather than make a supposedly "logical" decision now, I'd quite like to experiment with this briefly to investigate usage cases, potential issues, paths of least resistance (pun intended) and so forth.

[nathanjhood]

Progress:

• moved all helper functions from package.json#bin.<path/to/file> to package.json#exports.<path/to/file>

Result:

note: the unwanted symlinks to the helpers and build script are gone

The CLI successfully locates the build script, but that script fails due to not resolving a helper function.

Could it be as simple as changing the import statement in the script...?

[nathanjhood]

Progress:

• got the "consumer" project to build using esbuild-scripts <3
• to do so, I had to manually enter <projectRoot>/node_modules/@nathanjhood/esbuild-scripts and change all pkgroll_createRequire.require('path/to/helper') calls to use the regular require('path/to/helper') (ie., not the pkgroll shim)
• I also realized how to export multiple declarations and commonjs/modules using package.json@exports:{} correctly
• for the extra mile, the above shown demo project "consumer" was built entirely using React Native Web :D

So, to take this PR home, I need to set up these exports correctly such that the output, as found in "consumer/node_modules/...." is runnable.

That's a milestone of a bigger milestone - one cornerpiece of the 0.0.1 baseline, in view.

[nathanjhood]

Update:

It's done.

The "consumer" project is building and all is correct. Results are serving as per above, but without any modifications to the node_modules contents. I added the copyPublicFolder function to finish the job.

Main takeaway from this PR was to shim every file with a createRequire(__filename) - __filename, not import.meta.url!!! - so that each one can resolve themselves and their relative imports/requires correctly.

* any mention of import.meta.url automatically converts the file to a module, per NodeJS docs - despite not mentioning the __filename workaround anywhere...

Later, I will probably do something like this in the package.json#exports:

"exports": {
  "src/process/parseArgs.ts": {
    "require": {
      "types": "./dist/process/parseArgs.d.cts",
      "default": "./dist/process/parseArgs.cjs"
    },
    "import": {
      "types": "./dist/process/parseArgs.d.mts",
      "default": "./dist/process/parseArgs.mjs"
    }
  }
  // and all the rest....
}

Just needed to puth the template somewhere so I can use it later to get that done.