Open melange396 opened 3 months ago
We could even put some of the esoteric usage and/or larger installation dependencies behind package "extras".
If we combine this with migrating over to pyproject.toml
, we could use the optional-dependencies
interface to separate out the dev
dependencies. See here for an example of migrating over to pyproject.toml
. Then, we could change Makefile
in _delphi_utils_python
to do pip install -e .[dev]
and all the rest of the indicators Makefiles can stay the same.
migrating to using pyproject.toml
helps unlock this issue: https://github.com/cmu-delphi/covidcast-indicators/issues/1839
Gonna reopen this issue. Also, gonna edit your OP @melange396 so the subtasks are clearer because #1984 addressed the first of the subtasks and we could probably tackle the second in the Jenkins-fix follow-up too. The third subtask can be closed when we finish up #1972.
Thinking about the unused package subtask, I found deptry
and ran it on _delphi_utils and it reported this:
(env) ➜ _delphi_utils_python git:(1972-replace-covidcast) ✗ deptry .
Scanning 31 files...
data_proc/geomap/geo_data_proc.py:13:8: DEP003 'requests' imported but it is a transitive dependency
data_proc/geomap/source-file-sanity-check.ipynb:23:8: DEP003 'matplotlib' imported but it is a transitive dependency
delphi_utils/flash_eval/eval_day.py:6:1: DEP003 'scipy' imported but it is a transitive dependency
delphi_utils/validator/datafetcher.py:13:8: DEP003 'requests' imported but it is a transitive dependency
requirements.txt: DEP002 'scs' defined as a dependency but not used in the codebase
requirements.txt: DEP002 'darker' defined as a dependency but not used in the codebase
requirements.txt: DEP002 'freezegun' defined as a dependency but not used in the codebase
requirements.txt: DEP002 'mock' defined as a dependency but not used in the codebase
requirements.txt: DEP002 'moto' defined as a dependency but not used in the codebase
requirements.txt: DEP002 'pydocstyle' defined as a dependency but not used in the codebase
requirements.txt: DEP002 'pylint' defined as a dependency but not used in the codebase
requirements.txt: DEP002 'pytest-cov' defined as a dependency but not used in the codebase
requirements.txt: DEP002 'pytest' defined as a dependency but not used in the codebase
requirements.txt: DEP002 'requests-mock' defined as a dependency but not used in the codebase
requirements.txt: DEP002 'xlrd' defined as a dependency but not used in the codebase
Found 16 dependency issues.
For more information, see the documentation: https://deptry.com/
The following will be handled by #2014, as they get moved to dev.
requirements.txt: DEP002 'darker' defined as a dependency but not used in the codebase
requirements.txt: DEP002 'freezegun' defined as a dependency but not used in the codebase
requirements.txt: DEP002 'mock' defined as a dependency but not used in the codebase
requirements.txt: DEP002 'moto' defined as a dependency but not used in the codebase
requirements.txt: DEP002 'pydocstyle' defined as a dependency but not used in the codebase
requirements.txt: DEP002 'pylint' defined as a dependency but not used in the codebase
requirements.txt: DEP002 'pytest-cov' defined as a dependency but not used in the codebase
requirements.txt: DEP002 'pytest' defined as a dependency but not used in the codebase
requirements.txt: DEP002 'requests-mock' defined as a dependency but not used in the codebase
xlrd
is needed by Pandas (maybe we should write pandas[excel]
so this dependence is explicit? EDIT: pandas==1.5.3 does not support that extra). scs
is explicitly version pinned for cvxpy
because of upstream issuesrequests
as a dependecy (done in follow up)
delphi_utils
is a big hodge-podge of a package with a lot of dependencies, thus it has a relatively large net footprint and takes a while to install. Lets see if we can lessen that.darker
,freezegun
,mock
,moto
,pydocstyle
,pylint
,pytest-cov
,pytest
,requests-mock
. They can be put into something like a "requirements.test.txt
" file that gets installed via the CI and/or Makefile.setup.py
any unused packages, asgitpython
appears to be.covidcast
package has its own wacky imports, which includes a bunch of imaging/graphics and geospatial libraries. It is only used in thedelphi_utils.validator
module for pulling data from the API and can be replaced with the lighter-weightdelphi_epidata
client instead. Doing that is not quite as easy as moving requirements entries around, and it is covered in in #1972.