Add "how to build/run local version of Auspice CLI" to dev docs

[corneliusroemer]

I'm trying to test @jameshadfield's PR #1655. The new feature applies only to very specific datasets, none of which are in the sample datasets that come with the heroku test build.

So I need to run auspice locally and pass in my own dataset through the CLI.

However, being a JS noob, I can't figure out how. And unfortunately the dev_docs.md don't seem to help me neither.

It would be nice if there was a simple section explaining how to run the auspice CLI based off the current code base, the equivalent of pip install -e . (which would also make the CLI entry point available in path), or something more like pipx not putting it in path but allowing me to pass execute it once, something like that must exist!

I want something like cargo run (Rust) where it runs the CLI off the current code base.

[corneliusroemer]

Figured it out with help of ChatGPT (but would still be nice to dcoument):

npm run (cargo is actually modelled after npm to some extent, so this is no coincidence)

So npm run view is the equivalent of auspice view

There's a shortcut also available: npm start that is the equivalent of npm run view.

To pass arguments I need to add -- first: npm run view -- -h

This is how to test:

npm run view -- --datasetDir <AUSPICE_FOLDER_PATH> 

[tsibley]

You can also run ./auspice.js, e.g. ./auspice.js view --datasetDir ….

npm run is not really equivalent to cargo run; it's only running the shortcuts/aliases defined in package.json, e.g.

https://github.com/nextstrain/auspice/blob/471fab1424be784bccf3d4470a944adabaeb3442/package.json#L17-L35

It does some PATH and other env var manipulations before invoking the aliases, but these are rarely necessary.

The rough equivalent of pipx is npx; the former is vaguely modeled off the latter, but with substantial improvements IMO.

The Auspice install docs cover how to install "as a developer". Note that they recommend (and assume) you create a Conda environment so that the Node installation in use for Auspice is isolated. If you're managing Node installations another way (e.g. nvm), then you may not want to follow the recommendation to npm install --global ..

[corneliusroemer]

Thanks! This dev install part https://docs.nextstrain.org/projects/auspice/en/stable/introduction/install.html#install-auspice-as-a-developer should be linked from dev docs - I didn't find it because I assumed docs would be user facing, not dev facing.

[corneliusroemer]

I'm following the dev installation but get an error:

CONDA_SUBDIR=osx-arm64 mamba create --name auspice nodejs=16
mamba activate auspice
 conda config --env --set subdir osx-arm64
npm install --global .

with error:

npm ERR! WebpackOptionsValidationError: Invalid configuration object. Webpack has been initialised using a configuration object that does not match the API schema.

```console npm ERR! code 1 npm ERR! path /Users/corneliusromer/code/auspice npm ERR! command failed npm ERR! command sh -c -- npm run build npm ERR! > auspice@2.45.2 build npm ERR! > node auspice.js build --verbose npm ERR! npm ERR! [verbose] Generating webpack config. Extensions? false. devMode: false npm ERR! [verbose] Webpack writing output to: /Users/corneliusromer/code/auspice/dist npm ERR! /Users/corneliusromer/code/auspice/node_modules/webpack/lib/webpack.js:31 npm ERR! throw new WebpackOptionsValidationError(webpackOptionsValidationErrors); npm ERR! ^ npm ERR! npm ERR! WebpackOptionsValidationError: Invalid configuration object. Webpack has been initialised using a configuration object that does not match the API schema. npm ERR! - configuration.module.rules[2].type should be one of these: npm ERR! "javascript/auto" | "javascript/dynamic" | "javascript/esm" | "json" | "webassembly/experimental" npm ERR! -> Module type to use for the module npm ERR! - configuration.resolve has an unknown property 'fallback'. These properties are valid: npm ERR! object { alias?, aliasFields?, cachePredicate?, cacheWithContext?, concord?, descriptionFiles?, enforceExtension?, enforceModuleExtension?, extensions?, fileSystem?, ignoreRootsErrors?, mainFields?, mainFiles?, moduleExtensions?, modules?, plugins?, preferAbsolute?, resolver?, roots?, symlinks?, unsafeCache?, useSyncFileSystemCalls? } npm ERR! -> Options for the resolver npm ERR! at webpack (/Users/corneliusromer/code/auspice/node_modules/webpack/lib/webpack.js:31:9) npm ERR! at Object.run (/Users/corneliusromer/code/auspice/cli/build.js:33:20) npm ERR! at Object. (/Users/corneliusromer/code/auspice/auspice.js:33:9) npm ERR! at Module._compile (node:internal/modules/cjs/loader:1165:14) npm ERR! at Object.Module._extensions..js (node:internal/modules/cjs/loader:1219:10) npm ERR! at Module.load (node:internal/modules/cjs/loader:1043:32) npm ERR! at Function.Module._load (node:internal/modules/cjs/loader:878:12) npm ERR! at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12) npm ERR! at node:internal/main/run_main_module:22:47 { npm ERR! details: undefined, npm ERR! missing: undefined, npm ERR! origin: undefined, npm ERR! dependencies: undefined, npm ERR! module: undefined, npm ERR! validationErrors: [ npm ERR! { npm ERR! keyword: 'anyOf', npm ERR! dataPath: '.module', npm ERR! schemaPath: '#/properties/module/anyOf', npm ERR! params: {}, npm ERR! message: 'should match some schema in anyOf', npm ERR! schema: [ { '$ref': '#/definitions/ModuleOptions' } ], npm ERR! parentSchema: { npm ERR! description: 'Options affecting the normal modules (`NormalModuleFactory`).', npm ERR! anyOf: [ { '$ref': '#/definitions/ModuleOptions' } ] npm ERR! }, npm ERR! data: { npm ERR! rules: [ npm ERR! { npm ERR! test: /\.js$/, npm ERR! loader: 'babel-loader', npm ERR! exclude: [Array], npm ERR! options: [Object] npm ERR! }, npm ERR! { test: /\.css$/, use: [Array] }, npm ERR! { npm ERR! test: /\.(gif|png|jpe?g|svg|woff2?|eot|otf|ttf)$/i, npm ERR! type: 'asset/resource' npm ERR! }, npm ERR! { test: /node_modules\/esprima/, use: 'null-loader' } npm ERR! ] npm ERR! }, npm ERR! children: [ npm ERR! { npm ERR! keyword: 'enum', npm ERR! dataPath: '.module.rules[2].type', npm ERR! schemaPath: '#/properties/type/enum', npm ERR! params: { allowedValues: [Array] }, npm ERR! message: 'should be equal to one of the allowed values', npm ERR! schema: [ npm ERR! 'javascript/auto', npm ERR! 'javascript/dynamic', npm ERR! 'javascript/esm', npm ERR! 'json', npm ERR! 'webassembly/experimental' npm ERR! ], npm ERR! parentSchema: { npm ERR! description: 'Module type to use for the module', npm ERR! enum: [Array] npm ERR! }, npm ERR! data: 'asset/resource', npm ERR! children: undefined npm ERR! }, npm ERR! { npm ERR! keyword: 'anyOf', npm ERR! dataPath: '.module.rules[2]', npm ERR! schemaPath: '#/items/anyOf', npm ERR! params: {}, npm ERR! message: 'should match some schema in anyOf', npm ERR! schema: [ [Object] ], npm ERR! parentSchema: { description: 'A rule', anyOf: [Array] }, npm ERR! data: { npm ERR! test: /\.(gif|png|jpe?g|svg|woff2?|eot|otf|ttf)$/i, npm ERR! type: 'asset/resource' npm ERR! }, npm ERR! children: undefined npm ERR! }, npm ERR! { npm ERR! keyword: 'anyOf', npm ERR! dataPath: '.module.rules', npm ERR! schemaPath: '#/properties/rules/anyOf', npm ERR! params: {}, npm ERR! message: 'should match some schema in anyOf', npm ERR! schema: [ [Object] ], npm ERR! parentSchema: { npm ERR! description: 'An array of rules applied for modules.', npm ERR! anyOf: [Array] npm ERR! }, npm ERR! data: [ [Object], [Object], [Object], [Object] ], npm ERR! children: undefined npm ERR! } npm ERR! ] npm ERR! }, npm ERR! { npm ERR! keyword: 'anyOf', npm ERR! dataPath: '.resolve', npm ERR! schemaPath: '#/properties/resolve/anyOf', npm ERR! params: {}, npm ERR! message: 'should match some schema in anyOf', npm ERR! schema: [ { '$ref': '#/definitions/ResolveOptions' } ], npm ERR! parentSchema: { npm ERR! description: 'Options for the resolver', npm ERR! anyOf: [ { '$ref': '#/definitions/ResolveOptions' } ] npm ERR! }, npm ERR! data: { npm ERR! alias: { npm ERR! '@extensions': '/Users/corneliusromer/code/auspice/.null', npm ERR! '@auspice': '/Users/corneliusromer/code/auspice/src', npm ERR! react: '/Users/corneliusromer/code/auspice/node_modules/react', npm ERR! 'react-hot-loader': '/Users/corneliusromer/code/auspice/node_modules/react-hot-loader', npm ERR! 'react-dom': '/Users/corneliusromer/code/auspice/node_modules/@hot-loader/react-dom', npm ERR! 'regenerator-runtime': '/Users/corneliusromer/code/auspice/node_modules/regenerator-runtime', npm ERR! 'core-js': '/Users/corneliusromer/code/auspice/node_modules/core-js', npm ERR! 'styled-components': '/Users/corneliusromer/code/auspice/node_modules/styled-components' npm ERR! }, npm ERR! fallback: { npm ERR! buffer: '/Users/corneliusromer/code/auspice/node_modules/buffer/index.js', npm ERR! fs: false npm ERR! } npm ERR! }, npm ERR! children: [ npm ERR! { npm ERR! keyword: 'additionalProperties', npm ERR! dataPath: '.resolve', npm ERR! schemaPath: '#/additionalProperties', npm ERR! params: { additionalProperty: 'fallback' }, npm ERR! message: 'should NOT have additional properties', npm ERR! schema: false, npm ERR! parentSchema: { npm ERR! type: 'object', npm ERR! additionalProperties: false, npm ERR! properties: [Object] npm ERR! }, npm ERR! data: { alias: [Object], fallback: [Object] }, npm ERR! children: undefined npm ERR! } npm ERR! ] npm ERR! } npm ERR! ] npm ERR! } npm ERR! A complete log of this run can be found in: npm ERR! /Users/corneliusromer/.npm/_logs/2023-03-23T17_17_35_906Z-debug-0.log ```

[tsibley]

The dev docs do link to it, at the top:

https://github.com/nextstrain/auspice/blob/471fab1424be784bccf3d4470a944adabaeb3442/DEV_DOCS.md?plain=1#L24

I think that error indicates you have Webpack 4 not Webpack 5 (c.f. 7892c598fa305c2f1ee21c9aab5be01f15063c28 and efa51a77311099fd9eee9d1ab60a4f3cae88d4db). So that's interesting, because:

$ jq -r .dependencies.webpack package.json
^5.74.0

[tsibley]

Doesn't explain what you're seeing, but this works fine for me:

mamba create --name auspice nodejs=16
conda activate auspice
npm install --global .
npm run build

[corneliusroemer]

The dev docs do link to it, at the top:

Ok, but hidden in a section that I skimmed because I was going to test not contribute. I was looking for a section titled: "Setting up Auspice for development and/or testing" ;)

Something is different on my system. I have nvm installed, but which npm returns what we'd expect:

❯ which npm                                        
/opt/homebrew/Caskroom/miniforge/base/envs/auspice/bin/npm

❯ which node
/opt/homebrew/Caskroom/miniforge/base/envs/auspice/bin/node

Any ideas?

[tsibley]

I think the next step is to interrogate npm about what Webpack version you got and why. Don't have immediate suggestions about what that why might be.

If you don't care about preserving the state of this problem, you could try running npm ci to reinstall deps from scratch.

[corneliusroemer]

npm ci is the solution, works! Thanks for the tip. Maybe we can add that to the docs too, for the noobs like me 🙃

[tsibley]

Well, AFAIK, it shouldn't have been necessary with something like:

mamba create --name auspice nodejs=16
conda activate auspice
npm install --global .
npm run build

which is what's documented. So there's still something missing from our understanding here.

[tsibley]

Ok, here's one issue, which may be related to yours: the current instructions for developers assume you already have a node_modules/ bootstrapped somehow.

rm -rf node_modules/
conda env remove --name auspice
mamba create --name auspice nodejs=16
conda activate auspice
npm install --global .

The last line, because of the prepare and build aliases,

https://github.com/nextstrain/auspice/blob/471fab1424be784bccf3d4470a944adabaeb3442/package.json#L22-L23

tries to run Auspice before any deps are installed, yielding:

npm ERR! code 1
npm ERR! path /home/tom/nextstrain/auspice
npm ERR! command failed
npm ERR! command sh -c -- npm run build
npm ERR! > auspice@2.45.2 build
npm ERR! > node auspice.js build --verbose
npm ERR! node:internal/modules/cjs/loader:998
npm ERR!   throw err;
npm ERR!   ^
npm ERR! 
npm ERR! Error: Cannot find module 'argparse'
npm ERR! Require stack:
npm ERR! - /home/tom/nextstrain/auspice/auspice.js
npm ERR!     at Function.Module._resolveFilename (node:internal/modules/cjs/loader:995:15)
npm ERR!     at Function.Module._load (node:internal/modules/cjs/loader:841:27)
npm ERR!     at Module.require (node:internal/modules/cjs/loader:1067:19)
npm ERR!     at require (node:internal/modules/cjs/helpers:103:18)
npm ERR!     at Object.<anonymous> (/home/tom/nextstrain/auspice/auspice.js:3:18)
npm ERR!     at Module._compile (node:internal/modules/cjs/loader:1165:14)
npm ERR!     at Object.Module._extensions..js (node:internal/modules/cjs/loader:1219:10)
npm ERR!     at Module.load (node:internal/modules/cjs/loader:1043:32)
npm ERR!     at Function.Module._load (node:internal/modules/cjs/loader:878:12)
npm ERR!     at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12) {
npm ERR!   code: 'MODULE_NOT_FOUND',
npm ERR!   requireStack: [ '/home/tom/nextstrain/auspice/auspice.js' ]
npm ERR! }

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/tom/.npm/_logs/2023-03-23T22_43_28_039Z-debug-0.log

[victorlin]

1660 should address all the concerns above.