First working version of the Dockerfile after major refactoring

[asreimer]

This pull request merges in a bunch of refactoring. Namely: 1) All python environment setup is performed with helper scripts 2) Custom python package installation scripts are used instead of coding the installation process into the Dockerfile 3) All compilation that is needed to install any python package is performed using conda compilers, not OS provided compilers.

Testing of this pull request can be performed by building the docker image with:

cd /path/to/resen-core/repo/resen-core
docker build -t resenlab/testing .

After a successful build, start the docker container and jump into it:

docker run -itp 8000:8000 -v "$(pwd)"/testing:/home/jovyan/testing:ro,Z --name testing resenlab/testing bash

within the container, you can now cd ~/testing/tests/ and then run each of the test_*_imports.py scripts. Before running each of the tests, be sure to source the appropriate conda enviroment. There are currently 2 available, py27 and py36. For example:

conda activate py27
python test_py27_imports.py

will run the python 2.7 import tests.

[ljlamarche]

The py27 environment has the following error related to importing davitpy.

ERROR:root:Could not connect to remote DB: mongodb://sd_dbread:5d@sd-work9.ece.vt.edu:27017/radarInfo
WARNING:root:Reading hdw.dat info from local files in None
ERROR:root:radarRead: cannot read None
Traceback (most recent call last):
  File "/opt/conda/envs/py27/lib/python2.7/site-packages/davitpy/pydarn/radar/radInfoIo.py", line 67, in radarRead
    file_net = open(pathOpen, 'r')
TypeError: coercing to Unicode: need string or buffer, NoneType found
You may be getting this error because your computer cannot contact an appropriate internet server to get the latest radar.dat information.  You can use a local file instead by setting the SD_RADAR environment variable to the location of a local copy of radar.dat.

Example, you might add a similar line to your .bashrc:
export SD_RADAR=/home/username/tables/radar.dat

Also, make sure your SD_HDWPATH also points to the location of your hdw.dat files.
You can get the latest hdw.dat files from  https://github.com/vtsuperdarn/hdw.dat
Example, you might add a similar line to your .bashrc:
export SD_HDWPATH=/home/username/tables/hdw.dat/

ERROR:root:Could not update .radars.sqlite file with                               hdw.dat info
.

[ljlamarche]

The py36 environment had a DeprecationWarning attached to pytest.

/opt/conda/envs/py36/lib/python3.6/site-packages/_pytest/assertion/rewrite.py:8: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp

It also has an ImportWarning attached to pandas and cartopy that's related to Cython #7 .

/opt/conda/envs/py36/lib/python3.6/importlib/_bootstrap.py:219: ImportWarning: can't resolve package from __spec__ or __package__, falling back on __name__ and __path__
  return f(*args, **kwds)

We may be able to ignore both of these warnings.

[ljlamarche]

After discussion with @asreimer , we believe the davitpy issue(https://github.com/EarthCubeInGeo/resen-core/pull/5#issuecomment-481013478) can be resolved by including the hdw.dat repo may be solved doing the following:

1. Clone the hdw.dat repo into the cashe directory.

   cd cashe
   git clone https://github.com/vtsuperdarn/hdw.dat.git

2. Append the .bashrc file to set the SD_RADARS or SD_HDWPATH environment variables.

   echo "export SD_RADARS=$HOME/cache/hdw.dat" >>  ~/.bashrc

This should probably be done at the end of the setup_davitpy.sh helper script.

[pmreyes2]

@asreimer , I have done a pull request from enh_usepipenv branch of a fork to the main enh_usepipenv branch that fixes cartopy. The fix was to install shapely also from sources.

[ljlamarche]

Still need to add MadrigalWeb!

[asreimer]

Only need to add madrigalweb, then this can be merged. Any other problems seen in testing?

[asreimer]

I have moved the shapely installation stuff out of the setup_cartopy.py file. We do not need to clone the shapely repository to install it. Instead, we just need to build the shapely package ourselves from the source on pypi. This can be done with the command: LDFLAGS="-shared" pip install -UI --no-binary :all: shapely==1.6.4.post2

I have also added madrigalweb 3.1.10.

I'm now going to merge this pull request and pull in the tutorial in the first_tutorials branch so we can get a release candidate out.