Closed juzen2003 closed 7 months ago
I'm going to suggest we don't use pip-compile
and requirements.in
but just use normal requirements.txt
. The reason is two-fold:
1) With requirements.in
you end up freezing particular versions of every module, and having to manually update to newer versions. This shouldn't be necessary for a package like this.
2) rms-pdslogger
on Linux and MacOS depends on xattr
but on Windows it doesn't. Since the requirements.in
file was processed on MacOS, xattr
is in requirements.txt
. Unfortunately that module can't be installed at all on Windows, which means you can't actually do pip install -r requirements.txt
. This problem doesn't happen if we stick to the old way of doing things.
Logs for reference:
yu-jenchang make_rms_pdsfile_standalone rms-pdsfile $ rm -rf venv
yu-jenchang make_rms_pdsfile_standalone rms-pdsfile $ python -m venv venv
yu-jenchang make_rms_pdsfile_standalone rms-pdsfile $ source venv/bin/activate
(venv) yu-jenchang make_rms_pdsfile_standalone rms-pdsfile $ ./run_tests_coverage.sh
Clean up previous coverage record
Run with use shelves
/opt/homebrew/lib/python3.10/site-packages/coverage/inorout.py:460: CoverageWarning: --include is ignored because --source is set (include-ignored)
self.warn("--include is ignored because --source is set", slug="include-ignored")
ImportError while loading conftest '/Users/yu-jenchang/Desktop/seti/workspace/rms-pdsfile/conftest.py'.
conftest.py:8: in <module>
from pdsfile import (Pds3File,
pdsfile/__init__.py:6: in <module>
from .pds3file import *
pdsfile/pds3file/__init__.py:6: in <module>
import pdslogger
E ModuleNotFoundError: No module named 'pdslogger'
Run with no shelves
/opt/homebrew/lib/python3.10/site-packages/coverage/inorout.py:460: CoverageWarning: --include is ignored because --source is set (include-ignored)
self.warn("--include is ignored because --source is set", slug="include-ignored")
ImportError while loading conftest '/Users/yu-jenchang/Desktop/seti/workspace/rms-pdsfile/conftest.py'.
conftest.py:8: in <module>
from pdsfile import (Pds3File,
pdsfile/__init__.py:6: in <module>
from .pds3file import *
pdsfile/pds3file/__init__.py:6: in <module>
import pdslogger
E ModuleNotFoundError: No module named 'pdslogger'
Combine results from all modes
Combined data file .coverage.YuJensMBP2023M2.72748.065746
Skipping duplicate data .coverage.YuJensMBP2023M2.72745.419978
Generate html
Wrote HTML report to htmlcov/index.html
Report coverage
Name Stmts Miss Branch BrPart Cover
-------------------------------------------------------------------------------------------
pdsfile/__init__.py 3 1 0 0 67%
pdsfile/general_helper.py 86 86 46 0 0%
pdsfile/pds3file/__init__.py 87 86 6 0 1%
pdsfile/pds3file/rules/ASTROM_xxxx.py 8 8 0 0 0%
pdsfile/pds3file/rules/COCIRS_xxxx.py 99 99 38 0 0%
pdsfile/pds3file/rules/COISS_xxxx.py 73 73 26 0 0%
pdsfile/pds3file/rules/CORSS_8xxx.py 126 126 60 0 0%
pdsfile/pds3file/rules/COSP_xxxx.py 13 13 0 0 0%
pdsfile/pds3file/rules/COUVIS_0xxx.py 100 100 40 0 0%
pdsfile/pds3file/rules/COUVIS_8xxx.py 71 71 24 0 0%
pdsfile/pds3file/rules/COVIMS_0xxx.py 98 98 44 0 0%
pdsfile/pds3file/rules/COVIMS_8xxx.py 69 69 26 0 0%
pdsfile/pds3file/rules/EBROCC_xxxx.py 73 73 30 0 0%
pdsfile/pds3file/rules/GO_0xxx.py 128 128 46 0 0%
pdsfile/pds3file/rules/HSTxx_xxxx.py 80 80 30 0 0%
pdsfile/pds3file/rules/JNOJIR_xxxx.py 23 23 0 0 0%
pdsfile/pds3file/rules/JNOJNC_xxxx.py 17 17 0 0 0%
pdsfile/pds3file/rules/JNOSP_xxxx.py 13 13 0 0 0%
pdsfile/pds3file/rules/NHSP_xxxx.py 13 13 0 0 0%
pdsfile/pds3file/rules/NHxxxx_xxxx.py 120 120 52 0 0%
pdsfile/pds3file/rules/RES_xxxx.py 6 6 0 0 0%
pdsfile/pds3file/rules/RPX_xxxx.py 29 29 2 0 0%
pdsfile/pds3file/rules/VGIRIS_xxxx.py 10 10 0 0 0%
pdsfile/pds3file/rules/VGISS_xxxx.py 71 71 24 0 0%
pdsfile/pds3file/rules/VG_0xxx.py 26 26 0 0 0%
pdsfile/pds3file/rules/VG_20xx.py 10 10 0 0 0%
pdsfile/pds3file/rules/VG_28xx.py 55 55 0 0 0%
pdsfile/pds3file/rules/__init__.py 25 25 0 0 0%
pdsfile/pds3file/rules/pytest_support.py 19 19 0 0 0%
pdsfile/pds3file/tests/__init__.py 0 0 0 0 100%
pdsfile/pds3file/tests/helper.py 9 9 0 0 0%
pdsfile/pds3file/tests/test_pds3file_blackbox.py 593 593 46 0 0%
pdsfile/pds3file/tests/test_pds3file_blackbox_cached.py 275 275 18 0 0%
pdsfile/pds3file/tests/test_pds3file_whitebox.py 301 301 36 0 0%
pdsfile/pds3file/tests/test_pdsviewable_blackbox.py 24 24 0 0 0%
pdsfile/pds4file/__init__.py 68 68 6 0 0%
pdsfile/pds4file/rules/__init__.py 25 25 0 0 0%
pdsfile/pds4file/rules/cassini_iss.py 75 75 26 0 0%
pdsfile/pds4file/rules/cassini_vims.py 75 75 26 0 0%
pdsfile/pds4file/rules/pytest_support.py 19 19 0 0 0%
pdsfile/pds4file/rules/uranus_occs_earthbased.py 75 75 26 0 0%
pdsfile/pds4file/tests/__init__.py 0 0 0 0 100%
pdsfile/pds4file/tests/helper.py 5 5 0 0 0%
pdsfile/pds4file/tests/test_pds4file_blackbox.py 84 84 4 0 0%
pdsfile/pdsfile.py 3003 3003 1474 0 0%
pdsfile/pdsviewable.py 269 269 124 0 0%
pdsfile/preload_and_cache.py 23 23 10 0 0%
-------------------------------------------------------------------------------------------
TOTAL 6474 6471 2290 0 1%
(venv) yu-jenchang make_rms_pdsfile_standalone rms-pdsfile $ pip install -r requirements.txt
Collecting coverage
Downloading coverage-7.4.0-cp39-cp39-macosx_11_0_arm64.whl (205 kB)
|████████████████████████████████| 205 kB 2.8 MB/s
Collecting pillow
Downloading pillow-10.2.0-cp39-cp39-macosx_11_0_arm64.whl (3.3 MB)
|████████████████████████████████| 3.3 MB 13.0 MB/s
Collecting pylibmc
Using cached pylibmc-1.6.3-cp39-cp39-macosx_11_0_arm64.whl (205 kB)
Collecting pytest
Downloading pytest-7.4.4-py3-none-any.whl (325 kB)
|████████████████████████████████| 325 kB 9.0 MB/s
Collecting rms-pdslogger
Downloading rms_pdslogger-0.0.5-py3-none-any.whl (14 kB)
Collecting rms-pdstable
Downloading rms_pdstable-0.0.5-py3-none-any.whl (17 kB)
Collecting rms-translator
Downloading rms_translator-0.0.1-py3-none-any.whl (9.5 kB)
Collecting exceptiongroup>=1.0.0rc8
Downloading exceptiongroup-1.2.0-py3-none-any.whl (16 kB)
Collecting iniconfig
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Collecting tomli>=1.0.0
Using cached tomli-2.0.1-py3-none-any.whl (12 kB)
Collecting pluggy<2.0,>=0.12
Downloading pluggy-1.3.0-py3-none-any.whl (18 kB)
Collecting packaging
Using cached packaging-23.2-py3-none-any.whl (53 kB)
Collecting numpy
Downloading numpy-1.26.3-cp39-cp39-macosx_11_0_arm64.whl (14.0 MB)
|████████████████████████████████| 14.0 MB 4.7 MB/s
Collecting pyparsing
Downloading pyparsing-3.1.1-py3-none-any.whl (103 kB)
|████████████████████████████████| 103 kB 10.8 MB/s
Collecting rms-textkernel
Downloading rms_textkernel-1.0.0-py3-none-any.whl (42 kB)
|████████████████████████████████| 42 kB 3.1 MB/s
Collecting xattr
Downloading xattr-1.0.0-cp39-cp39-macosx_11_0_arm64.whl (18 kB)
Collecting rms-julian
Downloading rms_julian-2.0.0-py3-none-any.whl (74 kB)
|████████████████████████████████| 74 kB 8.8 MB/s
Collecting rms-pdsparser
Downloading rms_pdsparser-0.0.4-py3-none-any.whl (16 kB)
Collecting cffi>=1.16.0
Downloading cffi-1.16.0-cp39-cp39-macosx_11_0_arm64.whl (176 kB)
|████████████████████████████████| 176 kB 10.9 MB/s
Collecting pycparser
Downloading pycparser-2.21-py2.py3-none-any.whl (118 kB)
|████████████████████████████████| 118 kB 7.6 MB/s
Installing collected packages: pyparsing, pycparser, numpy, rms-textkernel, rms-julian, cffi, xattr, tomli, rms-pdsparser, pluggy, packaging, iniconfig, exceptiongroup, rms-translator, rms-pdstable, rms-pdslogger, pytest, pylibmc, pillow, coverage
Successfully installed cffi-1.16.0 coverage-7.4.0 exceptiongroup-1.2.0 iniconfig-2.0.0 numpy-1.26.3 packaging-23.2 pillow-10.2.0 pluggy-1.3.0 pycparser-2.21 pylibmc-1.6.3 pyparsing-3.1.1 pytest-7.4.4 rms-julian-2.0.0 rms-pdslogger-0.0.5 rms-pdsparser-0.0.4 rms-pdstable-0.0.5 rms-textkernel-1.0.0 rms-translator-0.0.1 tomli-2.0.1 xattr-1.0.0
(venv) yu-jenchang make_rms_pdsfile_standalone rms-pdsfile $ ./run_tests_coverage.sh
Clean up previous coverage record
Run with use shelves
/Users/yu-jenchang/Desktop/seti/workspace/rms-pdsfile/venv/lib/python3.9/site-packages/coverage/inorout.py:456: CoverageWarning: --include is ignored because --source is set (include-ignored)
self.warn("--include is ignored because --source is set", slug="include-ignored")
============================================= test session starts =============================================
platform darwin -- Python 3.9.6, pytest-7.4.4, pluggy-1.3.0
rootdir: /Users/yu-jenchang/Desktop/seti/workspace/rms-pdsfile
collected 633 items
pdsfile/pds3file/tests/test_pds3file_blackbox.py ...................................................... [ 8%] ....................................................................................................... [ 24%] ....................................................................................................... [ 41%] ...................... [ 44%] pdsfile/pds3file/tests/test_pds3file_blackbox_cached.py ............................................... [ 51%] ................................................. [ 59%] pdsfile/pds3file/tests/test_pds3file_whitebox.py ...................................................... [ 68%] ..................................... [ 74%] pdsfile/pds3file/tests/test_pdsviewable_blackbox.py .... [ 74%] pdsfile/pds3file/rules/COCIRS_xxxx.py ..... [ 75%] pdsfile/pds3file/rules/COISS_xxxx.py .. [ 75%] pdsfile/pds3file/rules/CORSS_8xxx.py .... [ 76%] pdsfile/pds3file/rules/COUVIS_0xxx.py ........... [ 78%] pdsfile/pds3file/rules/COUVIS_8xxx.py ....... [ 79%] pdsfile/pds3file/rules/COVIMS_0xxx.py ... [ 79%] pdsfile/pds3file/rules/COVIMS_8xxx.py .. [ 80%] pdsfile/pds3file/rules/EBROCC_xxxx.py ... [ 80%] pdsfile/pds3file/rules/GO_0xxx.py ... [ 81%] pdsfile/pds3file/rules/HSTxx_xxxx.py ... [ 81%] pdsfile/pds3file/rules/NHxxxx_xxxx.py ... [ 81%] pdsfile/pds3file/rules/VGISS_xxxx.py .. [ 82%] pdsfile/pds3file/rules/VG_28xx.py ........ [ 83%] pdsfile/pds4file/tests/test_pds4file_blackbox.py ...................................................... [ 92%] .................................................. [100%]
======================================= 633 passed in 109.97s (0:01:49) ======================================= Run with no shelves /Users/yu-jenchang/Desktop/seti/workspace/rms-pdsfile/venv/lib/python3.9/site-packages/coverage/inorout.py:456: CoverageWarning: --include is ignored because --source is set (include-ignored) self.warn("--include is ignored because --source is set", slug="include-ignored") ============================================= test session starts ============================================= platform darwin -- Python 3.9.6, pytest-7.4.4, pluggy-1.3.0 rootdir: /Users/yu-jenchang/Desktop/seti/workspace/rms-pdsfile collected 633 items
pdsfile/pds3file/tests/test_pds3file_blackbox.py ...................................................... [ 8%] ....................................................................................................... [ 24%] ....................................................................................................... [ 41%] ...................... [ 44%] pdsfile/pds3file/tests/test_pds3file_blackbox_cached.py ............................................... [ 51%] ................................................. [ 59%] pdsfile/pds3file/tests/test_pds3file_whitebox.py ...................................................... [ 68%] ..................................... [ 74%] pdsfile/pds3file/tests/test_pdsviewable_blackbox.py .... [ 74%] pdsfile/pds3file/rules/COCIRS_xxxx.py ..... [ 75%] pdsfile/pds3file/rules/COISS_xxxx.py .. [ 75%] pdsfile/pds3file/rules/CORSS_8xxx.py .... [ 76%] pdsfile/pds3file/rules/COUVIS_0xxx.py ........... [ 78%] pdsfile/pds3file/rules/COUVIS_8xxx.py ....... [ 79%] pdsfile/pds3file/rules/COVIMS_0xxx.py ... [ 79%] pdsfile/pds3file/rules/COVIMS_8xxx.py .. [ 80%] pdsfile/pds3file/rules/EBROCC_xxxx.py ... [ 80%] pdsfile/pds3file/rules/GO_0xxx.py ... [ 81%] pdsfile/pds3file/rules/HSTxx_xxxx.py ... [ 81%] pdsfile/pds3file/rules/NHxxxx_xxxx.py ... [ 81%] pdsfile/pds3file/rules/VGISS_xxxx.py .. [ 82%] pdsfile/pds3file/rules/VG_28xx.py ........ [ 83%] pdsfile/pds4file/tests/test_pds4file_blackbox.py ...................................................... [ 92%] .................................................. [100%]
TOTAL 6473 622 2299 216 88% (venv) yu-jenchang make_rms_pdsfile_standalone rms-pdsfile $
Updates:
requirements.in
to includerms-pdstable
,rms-pdslogger
,rms-translator
andpillow
pdsviewable.py
underpdsfile
and updated the import correspondinglypdsgroup
related functions in the testsos.path.join
with_clean_join
inpdsfile.py
Setup notes:
PYTHONPATH
, norms-pdsfile
andrms-webtools
rms-pdsfile
directory and set up virtual environement by doingpython -m venv venv
(or name your preferred venv folder)pip install -r requirements.txt
, before doing this, if you try to doimport pdsfile
inipython
or do./run_tests_coverage.sh
in terminal it will fail because packages are not installed../run_tests_coverage.sh
will pass, and import will work fine (similar to these inipython
):In [3]: print(pdsfile.file) /Users/yu-jenchang/Desktop/seti/workspace/rms-pdsfile/pdsfile/init.py
In [4]: import pdslogger
In [5]: print(pdslogger.file) /Users/yu-jenchang/Desktop/seti/workspace/rms-pdsfile/venv/lib/python3.10/site-packages/pdslogger/init.py
In [6]: import pdstable
In [7]: print(pdstable.file) /Users/yu-jenchang/Desktop/seti/workspace/rms-pdsfile/venv/lib/python3.10/site-packages/pdstable/init.py
In [8]: import translator
In [9]: print(translator.file) /Users/yu-jenchang/Desktop/seti/workspace/rms-pdsfile/venv/lib/python3.10/site-packages/translator/init.py
In [11]: print(pdsfile.pdsviewable.file) /Users/yu-jenchang/Desktop/seti/workspace/rms-pdsfile/pdsfile/pdsviewable.py
In [12]: print(pdsfile.pdscache.file) /Users/yu-jenchang/Desktop/seti/workspace/rms-pdsfile/pdsfile/pdscache.py