Guidance on installing / building the development server

[JohnGrime]

Hi,

I'm trying to follow the instructions under "Development" to put a dev install onto an Ubuntu 20.04 Server machine for testing, but I'm having some problems.

Machine info:

jgrime@machine:~/viv_dev/viv$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04.6 LTS
Release:        20.04
Codename:       focal
jgrime@machine:~/viv_dev/viv$

Fetch/install of the required pnpm package manager:

jgrime@machine:~/viv_dev$ curl -fsSL https://get.pnpm.io/install.sh | sh -
==> Downloading pnpm binaries 8.0.0
 WARN  using --force I sure hope you know what you are doing
Copying pnpm CLI from /tmp/tmp.JNtuvo331u/pnpm to /home/jgrime/.local/share/pnpm/pnpm
Appended new lines to /home/jgrime/.bashrc

Next configuration changes were made:
export PNPM_HOME="/home/jgrime/.local/share/pnpm"
case ":$PATH:" in
  *":$PNPM_HOME:"*) ;;
  *) export PATH="$PNPM_HOME:$PATH" ;;
esac

To start using pnpm, run:
source /home/jgrime/.bashrc
jgrime@machine:~/viv_dev$

Viv repo fetch:

jgrime@machine:~/viv_dev$ git clone https://github.com/hms-dbmi/viv
Cloning into 'viv'...
remote: Enumerating objects: 9245, done.
remote: Counting objects: 100% (1018/1018), done.
remote: Compressing objects: 100% (552/552), done.
remote: Total 9245 (delta 598), reused 737 (delta 463), pack-reused 8227
Receiving objects: 100% (9245/9245), 18.89 MiB | 32.90 MiB/s, done.
Resolving deltas: 100% (6514/6514), done.
jgrime@machine:~/viv_dev$

Install via pnpm fails with a node dependency:

jgrime@machine:~/viv_dev/viv$ pnpm install
Scope: all 11 workspace projects
Downloading registry.npmjs.org/gl/5.0.3: 5.81 MB/5.81 MB, done
Downloading registry.npmjs.org/typescript/4.7.4: 11.7 MB/11.7 MB, done
 WARN  deprecated rollup-plugin-inject@3.0.2: This package has been deprecated and is no longer maintained. Please use @rollup/plugin-inject.
 WARN  deprecated ecstatic@4.1.4: This package is unmaintained and deprecated. See the GH Issue 259.nloading registry.npmjs.org/texture-compressor/1.0.2: 1.57 MB/14.2 MB
sites/docs                               |  WARN  deprecated source-map-resolve@0.5.3
sites/docs                               |  WARN  deprecated source-map-url@0.4.1
sites/docs                               |  WARN  deprecated urix@0.1.0
 ENOENT  not found: node

This error happened while installing the dependencies of @pnpm/meta-updater@0.0.6
 at @pnpm/find-workspace-packages@3.1.55
 at @pnpm/cli-utils@0.6.63
 at @pnpm/package-is-installable@5.0.14

pnpm: not found: node
    at getNotFoundError (/snapshot/dist/pnpm.cjs)
    at Function.whichSync [as sync] (/snapshot/dist/pnpm.cjs)
    at getCommandAbsolutePathSync (/snapshot/dist/pnpm.cjs)
    at Object.sync (/snapshot/dist/pnpm.cjs)
    at getSystemNodeVersionNonCached (/snapshot/dist/pnpm.cjs)
    at getSystemNodeVersionNonCached (/snapshot/dist/pnpm.cjs)
    at checkPackage (/snapshot/dist/pnpm.cjs)
    at packageIsInstallable (/snapshot/dist/pnpm.cjs)
    at resolveAndFetch (/snapshot/dist/pnpm.cjs)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
Progress: resolved 1110, reused 0, downloaded 1064, added 0
Downloading registry.npmjs.org/texture-compressor/1.0.2: 3.15 MB/14.2 MB
jgrime@machine:~/viv_dev/viv$

I then installed node with apt. Version info:

jgrime@machine:~/viv_dev/viv$ node -v
v10.19.0
jgrime@machine:~/viv_dev/viv$

Second attempt to pnpm install:

jgrime@machine:~/viv_dev/viv$ pnpm install
Scope: all 11 workspace projects
 WARN  deprecated rollup-plugin-inject@3.0.2: This package has been deprecated and is no longer maintained. Please use @rollup/plugin-inject.
 WARN  deprecated ecstatic@4.1.4: This package is unmaintained and deprecated. See the GH Issue 259.nloading registry.npmjs.org/texture-compressor/1.0.2: 1.28 MB/14.2 MB
sites/docs                               |  WARN  deprecated source-map-resolve@0.5.3
sites/docs                               |  WARN  deprecated resolve-url@0.2.1
sites/docs                               |  WARN  deprecated source-map-url@0.4.1
sites/docs                               |  WARN  deprecated urix@0.1.0
Downloading registry.npmjs.org/texture-compressor/1.0.2: 14.2 MB/14.2 MB, done
Packages: +1299
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Packages are hard linked from the content-addressable store to the virtual store.
  Content-addressable store is at: /home/jgrime/.local/share/pnpm/store/v3
  Virtual store is at:             node_modules/.pnpm
Progress: resolved 1342, reused 1067, downloaded 232, added 1299, done
node_modules/.pnpm/esbuild@0.14.54/node_modules/esbuild: Running postinstall script, done in 1.1s
node_modules/.pnpm/esbuild@0.15.5/node_modules/esbuild: Running postinstall script, done in 1s
node_modules/.pnpm/puppeteer@16.2.0/node_modules/puppeteer: Running install script...
node_modules/.pnpm/puppeteer@16.2.0/node_modules/puppeteer: Running install script, done in 9.9s8s
node_modules/.pnpm/gl@5.0.3/node_modules/gl: Running install script, failed in 1.7s
.../.pnpm/gl@5.0.3/node_modules/gl install$ prebuild-install || node-gyp rebuild
│ prebuild-install warn install No prebuilt binaries found (target=10.19.0 runtime=node arch=x6
│ gyp info it worked if it ends with ok
│ gyp info using node-gyp@9.1.0
│ gyp info using node@10.19.0 | linux | x64
│ gyp info find Python using Python version 3.8.10 found at "/usr/bin/python3"
│ gyp ERR! UNCAUGHT EXCEPTION
│ gyp ERR! stack TypeError: Object.fromEntries is not a function
│ gyp ERR! stack     at Object.<anonymous> (/home/jgrime/viv_dev/viv/node_modules/.pnpm/@npmcli
│ gyp ERR! stack     at Module._compile (internal/modules/cjs/loader.js:778:30)
│ gyp ERR! stack     at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)
│ gyp ERR! stack     at Module.load (internal/modules/cjs/loader.js:653:32)
│ gyp ERR! stack     at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
│ gyp ERR! stack     at Function.Module._load (internal/modules/cjs/loader.js:585:3)
│ gyp ERR! stack     at Module.require (internal/modules/cjs/loader.js:692:17)
│ gyp ERR! stack     at require (internal/modules/cjs/helpers.js:25:18)
│ gyp ERR! stack     at Object.<anonymous> (/home/jgrime/viv_dev/viv/node_modules/.pnpm/@npmcli
│ gyp ERR! stack     at Module._compile (internal/modules/cjs/loader.js:778:30)
│ gyp ERR! System Linux 5.4.0-144-generic
│ gyp ERR! command "/usr/bin/node" "/home/jgrime/viv_dev/viv/node_modules/.pnpm/node-gyp@9.1.0/
│ gyp ERR! cwd /home/jgrime/viv_dev/viv/node_modules/.pnpm/gl@5.0.3/node_modules/gl
│ gyp ERR! node -v v10.19.0
│ gyp ERR! node-gyp -v v9.1.0
│ gyp ERR! Node-gyp failed to build your package.
│ gyp ERR! Try to update npm and/or node-gyp and if it does not help file an issue with the pac
└─ Failed in 1.7s at /home/jgrime/viv_dev/viv/node_modules/.pnpm/gl@5.0.3/node_modules/gl
 ELIFECYCLE  Command failed with exit code 7.
jgrime@machine:~/viv_dev/viv$

Can anyone let me know what I'm doing wrong here? I'm trying to follow the instructions on the GitHub repo, which specify "The package manager used to install and link dependencies must be pnpm" - so the earlier instructions re. installing dependencies via npm ("Install" section of the readme) shouldn't be followed?

[ilan-gold]

Your node version is quite low - I think we test against 16.

[JohnGrime]

I've tried with node v16 via nvm and it seems to avoid that error, thanks!

Is anyone interested in me posting a minimal Dockerfile that allows the Viv dev install to be built and run? I ran the resultant Docker image on a local development server, and I could connect to port 3000 on the server and get the Avivator web interface from my desktop machine so I guess the Dockerfile is more-or-less functional!

[ilan-gold]

Hmmm, I guess you could post it here. @manzt any thoughts where we could put this? or nowhere?

[manzt]

Hmm. We could pin this issue with the included Dockerfile if desired, however if someone searches our issues with "Docker" it should still be discoverable.

It's not very common to use Docker for development of JS packages, and my worry would be that we have mixed messaging around a recommended setup for development. Including an engine field in the root package.json would likely be sufficient to avoid the issue here. Node v10 has been EOL since March 2021, and Node v14 will be EOL soon. I think it is reasonable that we only support the officially supported Node versions for development.

[ilan-gold]

This makes sense. @JohnGrime you can post the Dockerfile here if you'd like. Thanks!

[hkthirano]

I started the app with wsl-ubuntu. The procedure is as follows.

# start wsl
wsl --install -d Ubuntu

# setup ubuntu
sudo apt update -y

sudo apt install nodejs npm -y
sudo npm -g install n -y
sudo n stable -y
sudo apt purge nodejs npm -y
sudo apt autoremove -y

sudo apt install build-essential libffi-dev libssl-dev zlib1g-dev liblzma-dev libbz2-dev \
  libreadline-dev libsqlite3-dev libopencv-dev tk-dev git -y
git clone https://github.com/pyenv/pyenv.git ~/.pyenv
echo '' >> ~/.bashrc
echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.bashrc
echo 'export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.bashrc
echo 'eval "$(pyenv init --path)"' >> ~/.bashrc
source ~/.bashrc
pyenv install 3.10.8
pyenv global 3.10.8

sudo apt install xserver-xorg-dev libxi-dev libxext-dev -y

curl -fsSL https://get.pnpm.io/install.sh | sh -

# build & run viv
git clone https://github.com/hms-dbmi/viv.git
cd viv
pnpm install 
pnpm dev