pip install is failing

[znat]

Running python setup.py sdist as advised in #296 raises the following error (on current master 672020e):

error in deep_qa setup command: 'install_requires' must be a string or list of strings containing valid project/version requirement specifiers; Invalid requirement, parse error at "'+git://g'"

[matt-gardner]

Looks like you can probably just change this to keras==2.0.3. Want to give it a try, and submit a PR if it works?

[znat]

Thanks @matt-gardner , it works and I'm happy to submit a PR. Before, I had another error while installing: numpy not found when installing scikit-learn. The install goes on and it seems (according to the log) that scikit-learn is installed. I'm not a python expert, but if you have a quick idea I am happy to try it and to add the fix to the PR

Running setup.py bdist_wheel for scikit-learn ... error
  Complete output from command /Users/nzylber1/anaconda/envs/deep_qa/bin/python -u -c "import setuptools, tokenize;__file__='/private/var/folders/h_/5d7byhjs1z3d42s46gp7d1g40000gs/T/pip-build-ts_kx2j9/scikit-learn/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /var/folders/h_/5d7byhjs1z3d42s46gp7d1g40000gs/T/tmpmbmpci6ipip-wheel- --python-tag cp36:
  Partial import of sklearn during the build process.
  Traceback (most recent call last):
    File "/private/var/folders/h_/5d7byhjs1z3d42s46gp7d1g40000gs/T/pip-build-ts_kx2j9/scikit-learn/setup.py", line 169, in get_numpy_status
      import numpy
  ModuleNotFoundError: No module named 'numpy'
  Traceback (most recent call last):
    File "/private/var/folders/h_/5d7byhjs1z3d42s46gp7d1g40000gs/T/pip-build-ts_kx2j9/scikit-learn/setup.py", line 149, in get_scipy_status
      import scipy
  ModuleNotFoundError: No module named 'scipy'
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/private/var/folders/h_/5d7byhjs1z3d42s46gp7d1g40000gs/T/pip-build-ts_kx2j9/scikit-learn/setup.py", line 270, in <module>
      setup_package()
    File "/private/var/folders/h_/5d7byhjs1z3d42s46gp7d1g40000gs/T/pip-build-ts_kx2j9/scikit-learn/setup.py", line 250, in setup_package
      .format(numpy_req_str, instructions))
  ImportError: Numerical Python (NumPy) is not installed.
  scikit-learn requires NumPy >= 1.6.1.
  Installation instructions are available on the scikit-learn website: http://scikit-learn.org/stable/install.html

[matt-gardner]

I'm not sure what's up with that error. Try putting numpy higher in the list of install_requires? To solve the issue for yourself locally, I imagine running pip install -r requirements.txt will solve it.

[nelson-liu]

this is a known "problem" with scikit-learn, and numpy isn't in scikit-learns install_requires since it might need to be linked to a specific BLAS. There's a fix here (https://github.com/scikit-learn/scikit-learn/pull/6990), but I think that moving numpy above scikit-learn should be an ok workaround (I'm assuming that people who need to link their own BLAS will then uninstall numpy and do it properly themselves)

[nelson-liu]

Is it possible to get this tested in CI?

[matt-gardner]

@nelson-liu, this would just mean running python setup.py sdist in one of our scripts, right? Either on CircleCI or on TravisCI. Are you volunteering to look into it? =)

[znat]

I did try to move numpy above scikit-learn but have the same error

[matt-gardner]

Did you run pip install -r requirements.txt?

[nelson-liu]

In the ideal scenario, should one have to run pip install -r requirements.txt? Or is python setup.py sdist supposed to entail that

[matt-gardner]

I have no idea. @DeNeutoy set up the setup.py script, and knows a bit more about pypi stuff than I do.

[DeNeutoy]

@znat For me when you run the setup, pip is very clever and retries installing scikit-learn after it initially fails and after installing all other packages. The install is successful (I changed the keras version to 2.0.3 only, I did not move the install position of sklearn).

Please follow these steps:

git clone https://github.com/allenai/deep_qa.git
cd deep_qa 
conda env create --name deep_qa_env python=3.5
source activate deep_qa_env
python setup.py sdist
pip install dist/deep_qa-0.1.1.tar.gz

python
import sklearn
import deep_qa

[znat]

@DeNeutoy Yes, the install seems indeed successful. I was just reporting the error. As @nelson-liu mentioned it's independent of deep_qa so I'll just submit the PR with the keras=2.0.3as requested and close the issue.