Augment setup documentation

VictoriaVasys commented 7 years ago

R server

The instruction "Create a new empty folder on your host OS and clone this project & the R server" is a bit confusing; should I also be downloading an "R server"? How do I go about doing this?

Cloning syntax

git clone git@github.com/rubyforgood/ghgvc.git && cd ghgvc should be git clone git@github.com:rubyforgood/ghgvc.git && cd ghgvc

Creating databses

When trying to create dbs, I got the following errors:

#<Mysql2::Error: Can't connect to MySQL server on 'database' (111)>
Couldn't create database for {"adapter"=>"mysql2", "encoding"=>"utf8", "pool"=>5, "host"=>"database", "username"=>"root", "password"=>"root_password", "port"=>3306, "timeout"=>5000, "database"=>"climate_calculator_dev"}, {:charset=>"utf8"}
(If you set the charset manually, make sure you have a matching collation)
Created database 'climate_calculator_dev'
#<Mysql2::Error: Can't connect to MySQL server on 'database' (111)>
Couldn't create database for {"adapter"=>"mysql2", "encoding"=>"utf8", "pool"=>5, "host"=>"database", "username"=>"root", "password"=>"root_password", "port"=>3306, "timeout"=>5000, "database"=>"climate_calculator_test"}, {:charset=>"utf8"}
(If you set the charset manually, make sure you have a matching collation)
Created database 'climate_calculator_test'

After running docker-compose run --rm app bundle exec rails db:create again, it seemed create dbs fine (I got the following return)

Starting ghgvc_database_1 ...
Starting ghgvc_r_1 ... done
Created database 'climate_calculator_dev'
Created database 'climate_calculator_test'

Additional Docker commands would be helpful

I suggest adding

docker-compose run --rm app bin/rake routes
docker-compose run --rm web bin/rails c
docker-compose run --rm app rspec

Issues with testing

When I ran docker-compose run --rm app rspec, I got 3/5 failing tests; 2 of the errors were due to a problem locating chrome webdriver

Failure/Error: before { visit root_path }

     Selenium::WebDriver::Error::WebDriverError:
        Unable to find chromedriver. Please download the server from http://chromedriver.storage.googleapis.com/index.html and place it somewhere on your PATH. More info at https://github.com/SeleniumHQ/selenium/wiki/ChromeDriver.

& I suspect the third is related.

Biome
     Failure/Error: response = connection.eval("ghgvcr::get_biome(latitude, longitude)")

     Rserve::Connection::EvalError:
       eval failed: Packet[cmd=2130771970,len=<nil>, con='<nil>', status=error:'Unknown variable/method'(127)]

Failure to get biomes

When I click on the map in the browser (both on localhost:3000 and 127.0.0.1:3000), I get the following errors on the server:

Started GET "/get_biome?lng=-94&lat=38" for 172.18.0.1 at 2017-07-24 14:04:37 +0000
app_1       | Cannot render console from 172.18.0.1! Allowed networks: 127.0.0.1, ::1, 127.0.0.0/127.255.255.255
app_1       | Processing by WorkflowsController#get_biome as */*
app_1       |   Parameters: {"lng"=>"-94", "lat"=>"38"}
app_1       | Completed 500 Internal Server Error in 1668ms (ActiveRecord: 0.0ms)
app_1       |
app_1       |
app_1       |
app_1       | Rserve::Connection::EvalError (eval failed: Packet[cmd=2130771970,len=<nil>, con='<nil>', status=error:'Unknown variable/method'(127)]):
app_1       |
app_1       | app/services/biome.rb:11:in `list'
app_1       | app/controllers/workflows_controller.rb:62:in `get_biome'

The root page looks like this below the map:

I'm guessing all these errors are due to not having the R server, but I've never used it and not sure what I need to do for full setup.

jaydorsey commented 7 years ago

The instruction "Create a new empty folder on your host OS and clone this project & the R server" is a bit confusing; should I also be downloading an "R server"? How do I go about doing this?

I need to clean this language up. We switched the process for this up semi-recently. At one point, you had to clone the Rails app, the R server, and the data into separate folders. Now that the R server code is stable I've pushed it up as a docker image so you don't need to do anything w/ the R server other than run the docker-compose commands in the README to download the image (build/up), download the datasets.

Cloning syntax

Saw this in your PR. That's good to merge in and will do so shortly

Creating databses

The issue w/ the database is a docker piece I need to create a ticket for & resolve. The rake task doesn't wait for the mysql server to download & start, so sometimes when you run the command the first time it errors out because it's not connected. There's a script/pattern for this. Will create an issue for this (created issue #69)

Issues with testing

Good catch. I'll add "integrate chrome driver w/ Rails docker image" to the list. (created issue #68)

I'm guessing all these errors are due to not having the R server, but I've never used it and not sure what I need to do for full setup.

The R debugging is one of the harder things to troubleshoot. The most common cause of that error that I've seen so far is when the data volume doesn't mount properly, or doesn't download & unzip the netcdf files correctly (for any variety of reasons).

If you can run docker-compose run r /bin/bash, then once you're inside bash in the container, cd data. Do you see a file name_indexed_ecosystems.json, and two folders (maps and netcdf)?

jaydorsey commented 7 years ago

Also, you can confirm if the R server is running by typing docker-compose ps. You should see a line similar to

      Name                    Command               State             Ports
-------------------------------------------------------------------------------------
ghgvc_r_1          /bin/sh -c Rscript start.R       Up       127.0.0.1:6311->6311/tcp

If you see Exit for the State of ghgvc_r_1, something broke 😬 You can try docker-compose down and then docker-compose up app and see if that fixes it.

VictoriaVasys commented 7 years ago

@jaydorsey thanks for the thorough review & comments! docker-compose ps looks all good:

But when I ran docker-compose run r /bin/bash & cd data, the directory was empty:

I'm guessing that's the issue; I thought I was thorough in walking through the setup but since the data was empty, I tried to run docker-compose up get_data again and got the following:

Let me know if you have thoughts on how to troubleshoot this (fyi I don't have any other servers running re: the error 127.0.0.1:6311 failed: port is already allocated).

jaydorsey commented 7 years ago

I'm not sure why you might see that error, but I think I can help you step thru it.

Try docker-compose down, then docker-compose ps again. It should show all containers stopped/exited. Then docker-compose up get_data again. You generally want to run this container as up, rather than run; it will do it's thing and then exit gracefully.

If that doesn't work try:

docker-compose down # stop everything
docker-compose volume rm ghgvc_netcdf-data # removes the volume
docker-compose up get_data # should redownload & unzip the data

The ghgvc_netcdf-data name should be the name of the volume. If that command fails, try docker volume ls and look for one that matches on the netcdf-data name (could have a number prefix I suppose)

It should take a little bit to download the data, and the curl output doesn't update correctly, so if it looks like it hangs just let it keep going.

It should show ghgvc_get_data_1 exited with code 0 when it's done running. Then your docker-compose run r /bin/bash followed by cd data should make the data appear.

If none of those work, I usually restart either docker or my machine (sometimes both!) because it usually means I put a container into a state that it shouldn't be and that will generally fix it if you can't force it to stop with docker-compose or docker commands. (I'm running edge version of docker, don't know if that's the reason why) That should be a last resort though.

Side note: I'll be out on vacation for a week and won't have laptop access. Will be back on August 5th. Feel free to file any issues though and I'll get thru them when I get back 👍

VictoriaVasys commented 7 years ago

Excellent, thanks for the walk-through! docker-compose down & docker-compose up get_data worked for me. I'll make another PR for the README to include these trouble-shooting tips, clarify the R server instructions & include docker commands for testing & entering the Rails console.

Enjoy your vacation!! 🌞

rubyforgood / ghgvc