wrtc installation issue on Mac M1

[adamConnerSax]

I'm trying to install the monitor on a Mac M1. When I do /usr/local/bin/npm install mcmc-monitor@latest

I get

code 1
npm ERR! path /Users/adam/node_modules/wrtc
npm ERR! command failed
npm ERR! command sh -c node scripts/download-prebuilt.js
npm ERR! response status 404 Not Found on https://node-webrtc.s3.amazonaws.com/wrtc/v0.4.7/Release/darwin-arm64.tar.gz
npm ERR! node-pre-gyp info it worked if it ends with ok
npm ERR! node-pre-gyp info using node-pre-gyp@1.0.10
npm ERR! node-pre-gyp info using node@18.16.0 | darwin | arm64
npm ERR! node-pre-gyp info check checked for "/Users/adam/node_modules/wrtc/build/Release/wrtc.node" (not found)
npm ERR! node-pre-gyp http GET https://node-webrtc.s3.amazonaws.com/wrtc/v0.4.7/Release/darwin-arm64.tar.gz
npm ERR! node-pre-gyp ERR! install response status 404 Not Found on https://node-webrtc.s3.amazonaws.com/wrtc/v0.4.7/Release/darwin-arm64.tar.gz 
npm ERR! node-pre-gyp ERR! install error 
npm ERR! node-pre-gyp ERR! stack Error: response status 404 Not Found on https://node-webrtc.s3.amazonaws.com/wrtc/v0.4.7/Release/darwin-arm64.tar.gz
npm ERR! node-pre-gyp ERR! stack     at /Users/adam/node_modules/@mapbox/node-pre-gyp/lib/install.js:67:15
npm ERR! node-pre-gyp ERR! stack     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
npm ERR! node-pre-gyp ERR! System Darwin 22.4.0
npm ERR! node-pre-gyp ERR! command "/usr/local/bin/node" "/Users/adam/node_modules/.bin/node-pre-gyp" "install"
npm ERR! node-pre-gyp ERR! cwd /Users/adam/node_modules/wrtc
npm ERR! node-pre-gyp ERR! node -v v18.16.0
npm ERR! node-pre-gyp ERR! node-pre-gyp -v v1.0.10
npm ERR! node-pre-gyp ERR! not ok

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/adam/.npm/_logs/2023-04-27T23_43_41_258Z-debug-0.log

I've tried installing things locally and globally. I've tried to install wrtc directly. I get the same error.

It seems like a known issue--that there's no arm-darwin version of wrtc--but none of the suggested workarounds were much help. See, e.g.,
https://github.com/node-webrtc/node-webrtc/issues/698

[magland]

@adamConnerSax thanks for this report! We hadn't tried it on a mac. We will look into possible alternative ways of doing this.

Out of curiosity, are you planning to use the remote monitoring features, or will this be monitoring things locally or on the local network?

[adamConnerSax]

I was planning to monitor locally (on the same machine). I have some models that take a few hours to run and in the development stage it would be pretty helpful to see what's happening with divergences and treedepths early on so as to catch the need to reparameterize (or just figure out where there might be some non-identification) early.

[magland]

I think we can solve this by making wrtc an optional dependency. Will follow up soon.

[jsoules]

Hi Adam, Thanks again for bringing this to our attention. We've just published some changes that make the wrtc dependency optional, as it likely won't be necessary for many of our users. Would you mind trying your install again with the new v. 0.1.14, and letting us know if you're able to install successfully?

Thanks!

[adamConnerSax]

Thanks for working on this! Sorry for the delay. Now I get a new error, this time upon running the monitor (NB: I just followed the readme directions so it's possible I'm not pulling the correct version somehow. But it does install without error this time and fail in a different place...)

node:internal/modules/cjs/loader:1078
  throw err;
  ^

Error: Cannot find module 'wrtc'
Require stack:
- /Users/adam/node_modules/mcmc-monitor/dist/RemotePeer.js
- /Users/adam/node_modules/mcmc-monitor/dist/Server.js
- /Users/adam/node_modules/mcmc-monitor/dist/index.js
- /Users/adam/node_modules/mcmc-monitor/bin/mcmc-monitor
    at Module._resolveFilename (node:internal/modules/cjs/loader:1075:15)
    at Module._load (node:internal/modules/cjs/loader:920:27)
    at Module.require (node:internal/modules/cjs/loader:1141:19)
    at require (node:internal/modules/cjs/helpers:110:18)
    at Object.<anonymous> (/Users/adam/node_modules/mcmc-monitor/dist/RemotePeer.js:7:32)
    at Module._compile (node:internal/modules/cjs/loader:1254:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1308:10)
    at Module.load (node:internal/modules/cjs/loader:1117:32)
    at Module._load (node:internal/modules/cjs/loader:958:12)
    at Module.require (node:internal/modules/cjs/loader:1141:19) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/Users/adam/node_modules/mcmc-monitor/dist/RemotePeer.js',
    '/Users/adam/node_modules/mcmc-monitor/dist/Server.js',
    '/Users/adam/node_modules/mcmc-monitor/dist/index.js',
    '/Users/adam/node_modules/mcmc-monitor/bin/mcmc-monitor'
  ]
}

Node.js v18.16.0

[jsoules]

Hi Adam, It seems like something didn't get updated properly in the build process for our last release, so you were getting the old version of the code. We've made some updates and confirmed that it's building properly now, so please give it another try.

Thanks for your help in making sure this is usable by as many people as possible!

[adamConnerSax]

Ah! Now the monitor starts on my end. Great! Web-viewer works as well, though fairly slow when loading a full run (4 chains, 1000 warmup, 1000 samples, 70 parameters). Which I know is not the purpose, but it's all I've got to check right now. Thanks!

[magland]

Ah! Now the monitor starts on my end. Great! Web-viewer works as well, though fairly slow when loading a full run (4 chains, 1000 warmup, 1000 samples, 70 parameters). Which I know is not the purpose, but it's all I've got to check right now. Thanks!

Glad it's working!

Is that dataset something you would be willing to share with us? If you zip the output folder and get it to us somehow, we can try it on our end and see what the bottleneck is. I'm guessing we can speed it up.

[adamConnerSax]

Let me wait until I have a slow model (slow to sample) to run and use the monitor for its designed purpose and see how it goes. I think my issue is that I place output files for several runs in the same folder. My workflow is driven via some Haskell libraries and they do the work of running stansummary on the right files to get the estimates/predictions as well as writing R scripts so I can use shinystan and loo for diagnostics. So I've never needed to worry about isolating the output files per run. I can change that. Would it also be possible to add a filter argument to the monitor (or via the web-app) to allow loading only a subset of stan output files?

[magland]

Could you clarify what you mean by "loading only a subset of stan output files"? Do you mean a subset of output directories? Or a subset of chains? Or something else?

[adamConnerSax]

Sorry! I mean that my "output" directory has dozens of files, 1 per chain but from several different model runs, usually different parameterizations or the like. They are named differently (modelA_1.csv, modelA_2.csv, modelA_3.csv, modelA_4.csv, modelB_1.csv, modelB_2.csv, ...). So rather than point at a directory, I would like to point at a directory and only monitor files matching, say, a regexp. But maybe I am the only one working this way? It wouldn't be hard to change on my end so that each stan run has its own output directory.

[magland]

We provide an option for the user to provide a mcmc-run.yaml file as described here:

https://github.com/flatironinstitute/mcmc-monitor#including-and-excluding-parameters

If you come up with a proposal for other information this file could contain, we can consider supporting that. I assume you would want to be able to select from the GUI whether to monitor modelA, modelB, ... in your example. So it would be more complex that a single regex.

If we come up with a spec, I assume you would create mcmc-run.yaml programatically as part of your workflow.

[adamConnerSax]

I don't think the yaml helps because there's only one for the directory. If the yaml allowed me to specify the file regex and the command-line allowed me to specify a yaml, I could make that work. I'm fine without being able to choose from the web-app. I would likely only monitor one run at a time. But it might just be simpler for me to change my workflow to one directory per run. There are ways that would simplify other things as well and shorten some currently unwieldy filenames...

[magland]

Okay sounds good. We can keep thinking about this.