Explain "nix-shell" etc.

kamahen commented 5 years ago

The README talks about "nix-shell in repo root". Please provide a pointer for those of us who aren't familiar with this tool.

robinp commented 5 years ago

Hi Peter!

On the phone now, so briefly in the mean time if you are eager to get started:

Nix is a package manager for reproducible environments. It comes as a fuller Linux OS as well, but nixpkgs itself can be used on any (?) Linux distro. See the one-paste curl command to get it installed: https://nixos.org/nix/download.html

After that, the nix-shell command will give you a shell, with the packages installed as defined in the default.nix or shell.nix file in the local dir.

Underhood uses a pinned nixpkgs version, so the binaries you would get from the nix-shell should be the "same". Hopefully no surprises.

Note, the first time it will fetch many packages. Please tell if you encounter any hurdle.

BR Robin

kamahen commented 5 years ago

Sorry for not being clearer in my question, and for probably missing something very obvious.

I'm sure I could figure out how to use nix-shell from the docs, but I'm not interested in doing that, wonderful though nix-shell might be. Rather, I just want to use UnderHood, so I'm requesting you to provide a "cookbook" of instructions. For example, I got to this part in your README: "Get a nix-shell in repo root" and despite some googling around, I couldn't figure out how to do that. (OK, I spent less than 5 minutes on it, but ...) Do you mean just executing curl https://nixos.org/nix/install | sh, or is there something more to do? Similarly, what does "Get a nix-shell in treetide/underhood/ui" mean?

So, I would very much appreciate a cookbook of what to type into my terminal window, to clear up my confusion.

robinp commented 5 years ago

Ah, I see ;) I did a speedrun out of curiosity on a (somewhat underpowered) fresh VPS box. Can be done in 30mins with just a few commands. Most of the time was fetching binary artifacts from the nix build cache (let me see if cachix can help there).

I'll post some detailed steps, but TLDR is:

yes, curl | sh the stuff, but not as root, rather a user with sudo access.
source the file it tells you to source
git clone underhood
cd underhood && nix-shell
now go and grab some coffee (15 mins?)
when you get the prompt (inside nix-shell), bazel build -c opt //treetide/underhood/.... This will be surprisingly fast.
bazel run -c opt //treetide/underhood/frontend_server
In an other window, cd treetide/underhood/ui, nix-shell, npm install, npm run start:dev

At this point, if you already have a Kythe http_server running at localhost:8080, you can just browse to localhost:9000 and see the CuH UI (the file nav on the left can take a while to load on large repo, see #2 ).

kamahen commented 5 years ago

Thank-you ... those instructions work. I'll prepare a pull request that updates the README with the details.

robinp commented 5 years ago

Thank you!

kamahen commented 5 years ago

I started http_server with this command:

/opt/kythe/tools/http_server -serving_table=/tmp/test/tables \
  -public_resources=/opt/kythe/web/ui \
  -listen=localhost:8081

and started underhood using npm run start:dev, but localhost:9000 showed this trivial program:

// Foobar
// Baz
int main() {}

I confirmed that the Kythe server was working by going to localhost:8081 ... any suggestions on what I've done wrong?

[nix-shell:~/src/underhood/treetide/underhood/ui]$ npm run start:dev

> underhood-ui@1.0.0 start:dev /home/peter/src/underhood/treetide/underhood/ui
> webpack-dev-server --mode 'development' --hot

ℹ ｢wds｣: Project is running at http://0.0.0.0:9000/
ℹ ｢wds｣: webpack output is served from /
ℹ ｢wds｣: Content not from webpack is served from /home/peter/src/underhood/treetide/underhood/ui/static
ℹ ｢wdm｣: Hash: c9e9e335451d0bb8debc
Version: webpack 4.35.3
Time: 1560ms
Built at: 2019-07-16 15:20:59
                     Asset       Size   Chunks             Chunk Names
                 bundle.js    138 KiB     main  [emitted]  main
         fonts/devicon.eot    203 KiB           [emitted]  
         fonts/devicon.svg    792 KiB           [emitted]  
         fonts/devicon.ttf    203 KiB           [emitted]  
        fonts/devicon.woff    203 KiB           [emitted]  
fonts/haskell-original.svg  833 bytes           [emitted]  
                index.html  448 bytes           [emitted]  
         vendors.bundle.js   1.89 MiB  vendors  [emitted]  vendors
Entrypoint main = vendors.bundle.js bundle.js
[0] multi (webpack)-dev-server/client?http://0.0.0.0:9000 (webpack)/hot/dev-server.js ./src/main.js 52 bytes {main} [built]
[./node_modules/normalize.css/normalize.css] 1.02 KiB {vendors} [built]
[./node_modules/vue-router/dist/vue-router.esm.js] 66.5 KiB {vendors} [built]
[./node_modules/vue/dist/vue.esm.js] 318 KiB {vendors} [built]
[./node_modules/webpack-dev-server/client/index.js?http://0.0.0.0:9000] (webpack)-dev-server/client?http://0.0.0.0:9000 4.29 KiB {vendors} [built]
[./node_modules/webpack-dev-server/client/overlay.js] (webpack)-dev-server/client/overlay.js 3.51 KiB {vendors} [built]
[./node_modules/webpack-dev-server/client/socket.js] (webpack)-dev-server/client/socket.js 1.53 KiB {vendors} [built]
[./node_modules/webpack-dev-server/client/utils/createSocketUrl.js] (webpack)-dev-server/client/utils/createSocketUrl.js 2.77 KiB {vendors} [built]
[./node_modules/webpack-dev-server/client/utils/log.js] (webpack)-dev-server/client/utils/log.js 964 bytes {vendors} [built]
[./node_modules/webpack-dev-server/client/utils/reloadApp.js] (webpack)-dev-server/client/utils/reloadApp.js 1.63 KiB {vendors} [built]
[./node_modules/webpack-dev-server/client/utils/sendMessage.js] (webpack)-dev-server/client/utils/sendMessage.js 402 bytes {vendors} [built]
[./node_modules/webpack-dev-server/node_modules/strip-ansi/index.js] (webpack)-dev-server/node_modules/strip-ansi/index.js 161 bytes {vendors} [built]
[./node_modules/webpack/hot sync ^\.\/log$] (webpack)/hot sync nonrecursive ^\.\/log$ 170 bytes {main} [built]
[./node_modules/webpack/hot/dev-server.js] (webpack)/hot/dev-server.js 1.59 KiB {vendors} [built]
[./src/main.js] 703 bytes {main} [built]
    + 118 hidden modules
Child html-webpack-plugin for "index.html":
     1 asset
    Entrypoint undefined = index.html
    [./node_modules/html-webpack-plugin/lib/loader.js!./src/index.html] 536 bytes {0} [built]
    [./node_modules/lodash/lodash.js] 528 KiB {0} [built]
    [./node_modules/webpack/buildin/global.js] (webpack)/buildin/global.js 472 bytes {0} [built]
    [./node_modules/webpack/buildin/module.js] (webpack)/buildin/module.js 497 bytes {0} [built]
ℹ ｢wdm｣: Compiled successfully.

When the Kythe server isn't running, I get this error message:

[HPM] Error occurred while trying to proxy request /api/filetree from localhost:9000 to http://localhost:8081 (ECONNREFUSED) (https://nodejs.org/api/errors.html#errors_common_system_errors)

robinp commented 5 years ago

@kamahen Something sounds amiss.

Note: Displaying the simple program is a red herring, it is the default the UI shows (hardcoded in the UI). What you should look for is the presence of the filetree root on the left side (takes a bit to load).

So port 8081 should be used by the frontend_server, and 8080 by the Kythe http_server.

Since you don't mention starting the frontend_server, maybe you didn't start it?

Note: The webpack dev server is set up in https://github.com/TreeTide/underhood/blob/develop/treetide/underhood/ui/webpack.config.js to proxy the /api requests to the frontend_server on 8081. In production, the line https://github.com/TreeTide/underhood/blob/develop/production/underhood/underhood_image.nix#L40 sets the port arguments (and nginx proxies at https://github.com/TreeTide/underhood/blob/develop/production/underhood/underhood_image.nix#L72).

TreeTide / underhood

Explain "nix-shell" etc. #8