search

delphi-suite / delphi

small language models training made easy
Apache License 2.0
10 stars 1 forks source link

Add requirements to setup.py #58

Closed jaidhyani closed 7 months ago

jaidhyani commented 8 months ago

Right now pip-installing delphi is broken because it doesn't pull in dependencies. We need to specify runtime dependencies in setup.py (this will probably be mostly copying from requirements.txt)

jettjaniak commented 7 months ago

assigned to Victor, let's try to make sure we're not duplicating the dependencies in both setup.py and requirements, maybe setup.py can just parse requirements.txt? not sure how to deal with them being split into cuda and non-cuda?

jaidhyani commented 7 months ago

There are ways for setup.py to reference requirements.py, or vice versa, but there are also reasons to not do this. First among them being "development requirements (requirements.txt) and different from runtime requirements (setup.py)". For example, you don't need black or isort to run delphi.

https://stackoverflow.com/a/49684835

The cuda/nocuda split introduces an additional complication. I think the best way to handle this in setup.py is probably with extras_required, though I don't know if there's a clean way to get this to integrate with a specific requirements file.