setup.R - trying to install "datetime" package causes setup() to fail

[MikeCIDFG]

Running setup() in R (in an Ubuntu VM) attempts to create a new r-reticulate conda environment and install the necessary dependencies for MLWIC.

(Installed & validated Tensorflow separately in another conda env per instructions, but maybe that's redundant?).

As part of this process it tries to install the package datetime (also ref'd in eval.py). This doesn't seem to be a valid external package (datetime module is built in to Python: Lib/datetime.py).

This generates an error that the package wasn't found in any of several conda repositories. This error causes setup to fail immediately.

Suggested fix: either remove datetime from the list of dependencies to download in setup.R if it only needs the built-in Python datetime module, or if it really is some other required package, fix the case-sensitive name & download location.

R console msg:

Solving environment: ...working... failed

PackagesNotFoundError: The following packages are not available from current channels:

  - datetime

Current channels:

  - https://conda.anaconda.org/conda-forge/linux-64
  - https://conda.anaconda.org/conda-forge/noarch
  - https://repo.anaconda.com/pkgs/main/linux-64
  - https://repo.anaconda.com/pkgs/main/noarch
  - https://repo.anaconda.com/pkgs/free/linux-64
  - https://repo.anaconda.com/pkgs/free/noarch
  - https://repo.anaconda.com/pkgs/r/linux-64
  - https://repo.anaconda.com/pkgs/r/noarch
  - https://repo.anaconda.com/pkgs/pro/linux-64
  - https://repo.anaconda.com/pkgs/pro/noarch

To search for alternate channels that may provide the conda package you're
looking for, navigate to

    https://anaconda.org

and use the search bar at the top of the page.

Error: Error 1 occurred installing packages into conda environment r-reticulate

[mikeyEcology]

I removed datetime, but if you ran the function previously and received the error message it is not a problem. All of the other packages should have installed properly, and you would be able to run the other functions. You do not need to run setup again.

[MikeCIDFG]

Thanks - other packages didn't install & classify() error'd w/"no module Tensorflow" (it is installed via pip & in other conda env, but didn't install in r-reticulate env).

The 'datetime' error is gone & it successfully removes the previous environment...

Reinstalled MLWIC. New error says "Unsatisfiable Error", "-argparse" and "-tensorflow" are in conflict. Arparse is apparently also built in to Python >=2.7.

Also, despite setting setup's 'python_loc' arg to point to Python 2.7, reticulate conda create tries to install Python 3.6 anyway. Need to set python=2.7?

Thanks~

[mikeyEcology]

I'm sorry but I can't replicate this error. Are you running on a Windows machine? If you are on a Mac have you tried updating to the most recent OS?

[MikeCIDFG]

No, Ubuntu VM, installed & updated last week.

## Package Plan ##

  environment location: /anaconda2/envs/r-reticulate

  added / updated specs: 
    - python

   The following NEW packages will be INSTALLED:

    ... #removed some for space
    python:          3.6.5-1           conda-forge
    ... #removed some for space

Solving environment: ...working... failed

UnsatisfiableError: The following specifications were found to be in conflict:
  - argparse
  - tensorflow
Use "conda info <package>" to see the dependencies for each package.

Error: Error 1 occurred installing packages into conda environment r-reticulate

Was able to solve by manually editing the conda environment generated by setup().

source activate r-reticulate
conda install python=2.7 #this downgraded the auto-installed python 3.6.5
pip install tensorflow #adding it from inside the env works.

After that, runningclassify() on the example images seems to be working fine! Thanks~

[mikeyEcology]

Are you using a GPU or CPU on the Ubuntu machine?

[mikeyEcology]

Also, what would happen if you just ignored the error message and tried running classify? Our tensorflow function installs TensorFlow separately, because it does require a little bit more interaction with the terminal.

[MikeCIDFG]

Just CPU (might add GPUs to server & change this later). Installed the CPU-only version of TF.

The tensorflow() function installs TF globally via pip, but this doesn't make it available inside R in my case. I think it's related to this issue - conda env can only see/use what's explicitly installed for that environment: https://stackoverflow.com/questions/41640056/do-conda-environments-have-access-to-root-environment-system-packages

(Note: this is on a server, and I'm trying to get this set up for multiple users ... Not installing in individual 'home' directories (~/anaconda2/bin) but actual root directory (/anaconda2/bin) so all users share the same consistent r-reticulate conda environment. Maybe if I was doing this for individual users in home directories conda would find that version of TF? Not sure) ...

Just running classify() after setup() errors resulted in saying that it "could not find any module 'tensorflow'".

Fixing datetime allowed setup() to complete properly & install TF in the environment.

But, setup() also installs Python 3.6 (for some reason), and ignored the 'python_loc' parameter pointing to 2.7.

Running classify() at this point resulted in the argparse error reported above. Going into the r-reticulate conda environment and manually installing Python2.7 fixed that issue, which allowed classify() to run as expected.

In any case, it's working for now! Was able to run the sample project & get results back. Good work!

[JepsonNomad]

@MikeCIDFG Can you explain how you edited the conda environment? I can edit the setup() function using trace(setup, edit = TRUE) but the function code just looks like this for me:

function (python_loc = "/anaconda2/bin/python", conda_loc = "/anaconda2/bin/conda", r_reticulate = FALSE) { reticulate::use_python(python_loc) packs <- c("numpy", "cycler", "matplotlib", "tornado", "argparse", "six", "scipy", "tensorflow") if (!r_reticulate) { reticulate::conda_remove("r-reticulate") reticulate::conda_create("r-reticulate", conda = conda_loc) } reticulate::py_install(packs, conda = conda_loc) }

So I'm wondering where the (presumably bash?) code ends up. I think the issue you're describing above must relate pretty closely to mine here.

Edit: I overcame the problem by upgrading my OSX and installing tensorflow using pip.