Adding complicated dependencies

[jcpayne]

Is there documentation somewhere of how to add complicated dependencies? My nbdev builds are failing when nbdev_test_nbs hits imports that aren't in plain Python 3. There is one line in the nbdev Tutorial that suggests adding packages to the requirements = line in settings.ini, but fastai itself has 4 different kinds of requirements listed in settings.ini:

requirements = fastcore>=1.3.8...
pip_requirements = torch>=1.7.0,<1.8...
conda_requirements = pytorch>=1.7.0,<1.8
dev_requirements = nbdev>=1.0.10...

Also, I'm working with Detectron2 which requires a couple of complex package installation commands that I don't know how to add to a simple requirements template:

pip install detectron2 -f https://dl.fbaipublicfiles.com/detectron2/wheels/cu101/torch1.5/index.html
pip install -U 'git+https://github.com/cocodataset/cocoapi.git#subdirectory=PythonAPI' --ignore-installed certifi

I'd appreciate any advice. If there is no way to add the dependencies, I guess I could prevent all tests from running, but I would like to do the CI tests if possible since it's good programming practice to include them.

Thanks

John

[muellerzr]

You should do git+git:// on your link and add it to whatever requirements you ay have like so:

requirements = fastcore>=1.3.8.... git+git://github.com/cocodataset/cocoapi.git#subdirectory=PythonAPI

@hamelsmu you may know more on the pip installing custom wheels?

[jcpayne]

Thanks @muellerzr. I'll try that. Is there any reason to prefer requirements=... over the other variants (pip_requirements etc.) that I listed?

[muellerzr]

No, it was just an example

[hamelsmu]

@jcpayne shall I close this issue?

[jcpayne]

Not yet, please. Several questions are still unanswered; 1) How to write the detectron2 import line; 2) Zach's suggestion doesn't include the flags -- can I include them? (I haven't tried yet) 2) What is the difference, if any, between the 4 kinds of requirement statements. Why are all 4 present in fastai?

[hamelsmu]

@jph00 any recommendations on the best way to handle this?

[jph00]

Summary:

• requirements: Passed to both pip and conda setup
• pip_requirements: Passed to pip setup only
• conda_requirements: Passed to conda setup only
• dev_requirements: Passed to pip setup as a development requirement

For the format of each, see the pypi and conda docs on creating specifications in setup.py and meta.yaml.

@hamelsmu any chance you could add this to the docs somewhere?

@jcpayne since the lines you're showing aren't just setup.py specs, I'm not sure it's possible to specify them as a requirement in pip. Since nbdev just passes the string directly to pip, there's nothing we can do to support requirements that pip doesn't support.

[jcpayne]

Thanks @jph00 and @hamelsmu for your replies - I appreciate them. I'll see if I can get it to work but it looks like I'll have to learn something about setup.py and meta.yaml first!

John

[jph00]

FYI if you don't need conda support, you can ignore meta.yaml

[hamelsmu]

@jcpayne @jph00 I corrected the summary above to avoid future confusion

[hamelsmu]

@hamelsmu any chance you could add this to the docs somewhere?

Added in https://github.com/fastai/nbdev/pull/399

[cvillagrasa]

Hi! I've been searching for a while and I'm unable to find the following:

In nbdev, if I want to use conda specifications in accordance to meta.yaml, should I place them under conda_requirements in settings.ini or should I create a meta.yaml?

Thanks in advance,