hyperspy / rosettasciio

Python library for reading and writing scientific data format
https://hyperspy.org/rosettasciio
GNU General Public License v3.0
51 stars 28 forks source link

De MRC Auto get Metadata #311

Closed CSSFrancis closed 1 week ago

CSSFrancis commented 2 months ago

Description of the change

A few sentences and/or a bulleted list to describe and motivate the change:

Progress of the PR

Minimal example of the bug fix or the new feature

from rsciio.mrc import file_reader
file_reader("your_mrc_file.mrc") # new automatically finds metadata file
file_reader("your_mrc_file.mrc", metadata_file = "your_mrc_file_info.txt ") # old

Note that this example can be useful to update the user guide.

codecov[bot] commented 2 months ago

Codecov Report

Attention: Patch coverage is 93.68421% with 6 lines in your changes missing coverage. Please review.

Project coverage is 87.88%. Comparing base (88923bf) to head (d7569e7). Report is 25 commits behind head on main.

Files with missing lines Patch % Lines
rsciio/mrc/_api.py 93.68% 4 Missing and 2 partials :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #311 +/- ## ========================================== + Coverage 87.84% 87.88% +0.03% ========================================== Files 85 85 Lines 11180 11241 +61 Branches 2280 2294 +14 ========================================== + Hits 9821 9879 +58 - Misses 860 863 +3 Partials 499 499 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

jlaehne commented 1 month ago

Is it possible to downscale the size of the movie file by e.g. binning?

CSSFrancis commented 1 month ago

Is it possible to downscale the size of the movie file by e.g. binning?

Sure if that is helpful I can do that.

jlaehne commented 1 month ago

Is it possible to downscale the size of the movie file by e.g. binning?

Sure if that is helpful I can do that.

We download only for testing, but still 32 MB for one data format is quite heavy. So far we have 137 MB of data and the edax format is the only other one that exceeds 30 MB where it could not be reduced (usually up to 10). So wherever we can be more slim without compromising testing, we should do so: https://rosettasciio.readthedocs.io/en/latest/contributing.html#making-test-data-files

CSSFrancis commented 1 month ago

@CSSFrancis, when reviewing this PR, I start to wonder if the DE MRC metadata should be documented in the https://hyperspy.org/rosettasciio/file_specification/index.html section? What do you think?

I added references to where the .mrc file format is defined :)

ericpre commented 4 weeks ago

pre-commit.ci autofix

ericpre commented 4 weeks ago

One workflow fails, because some test fails which are present on main are missing from this branch - a rebase should fix it or it is merge to main.

ericpre commented 2 weeks ago

There is a genuine test failure due to the # character in the filename that doesn't play well with downloading the test files using pooch.

It is trying to download from the following url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc

while the actual path is https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20%230_sum.mrc

In the pooch registry, the file are saved as:

'mrc/20241021_00405_0_Virt #0_sum.mrc' 3f798cf43ff1f3c39750b379a22962a4c6d2d546f1298461fabc6bced0cc876f
'mrc/20241021_00405_1_Virt #1_sum.mrc' 3f798cf43ff1f3c39750b379a22962a4c6d2d546f1298461fabc6bced0cc876f
'mrc/20241021_00405_ext1_Ext #1.mrc' 4962c3ead6547d819e22447765e93e357911bacd2905191d97d339b5cc0dd1dc

I added a third to use the custom url functionality but I think that the correct fix is that pooch parse the url. The update of the registry will fails because of the third columns and pre-commit CI will fail.

pytest log ```python =========================== short test summary info ============================ ERROR tests/test_import.py::test_import_version - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_import.py::test_rsciio_dir - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_import.py::test_rsciio_utils - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_import.py::test_import_all - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_import.py::test_format_name_aliases - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_import.py::test_dir_plugins[plugin0] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_import.py::test_dir_plugins[plugin1] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_import.py::test_dir_plugins[plugin2] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_import.py::test_dir_plugins[plugin3] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_import.py::test_dir_plugins[plugin4] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_import.py::test_dir_plugins[plugin5] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_import.py::test_dir_plugins[plugin17] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_import.py::test_dir_plugins[plugin18] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_import.py::test_dir_plugins[plugin19] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_import.py::test_dir_plugins[plugin20] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_import.py::test_dir_plugins[plugin21] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_import.py::test_dir_plugins[plugin22] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_import.py::test_dir_plugins[plugin23] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_import.py::test_dir_plugins[plugin24] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_import.py::test_dir_plugins[plugin25] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_import.py::test_dir_plugins[plugin26] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_import.py::test_dir_plugins[plugin27] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_import.py::test_dir_plugins[plugin28] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_import.py::test_dir_plugins[plugin29] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_import.py::test_dir_plugins[plugin30] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_import.py::test_dir_plugins[plugin31] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_import.py::test_dir_plugins[plugin32] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_lazy_not_implemented.py::test_lazy_not_implemented[dens] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_lazy_not_implemented.py::test_lazy_not_implemented[digitalsurf] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_lazy_not_implemented.py::test_lazy_not_implemented[impulse] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_lazy_not_implemented.py::test_lazy_not_implemented[jobinyvon] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_lazy_not_implemented.py::test_lazy_not_implemented[msa] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_lazy_not_implemented.py::test_lazy_not_implemented[netcdf] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_lazy_not_implemented.py::test_lazy_not_implemented[pantarhei] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_lazy_not_implemented.py::test_lazy_not_implemented[phenom] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_lazy_not_implemented.py::test_lazy_not_implemented[protochips] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_lazy_not_implemented.py::test_lazy_not_implemented[renishaw] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_lazy_not_implemented.py::test_lazy_not_implemented[trivista] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_set_log_level.py::test_set_log_level[CRITICAL] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc Error: ests/test_set_log_level.py::test_set_log_level[ERROR] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc Warning: ts/test_set_log_level.py::test_set_log_level[WARNING] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_set_log_level.py::test_set_log_level[INFO] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_set_log_level.py::test_set_log_level[DEBUG] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_set_log_level.py::test_set_log_level[NOTSET] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/utils/test_rgbtools.py::TestRGBTools::test_rgbx2regular_array_corder_from_c - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/utils/test_rgbtools.py::TestRGBTools::test_rgbx2regular_array_corder_from_c_dask - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/utils/test_rgbtools.py::TestRGBTools::test_rgbx2regular_array_corder_from_f - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/utils/test_rgbtools.py::TestRGBTools::test_rgbx2regular_array_corder_from_c_slices - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/utils/test_rgbtools.py::TestRGBTools::test_rgbx2regular_array_cordermask_from_cmasked - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/utils/test_rgbtools.py::TestRGBTools::test_rgbx2regular_array_cordermask_from_cmasked_slices - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/utils/test_rgbtools.py::TestRGBTools::test_regular_array2rgbx_corder_from_c - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/utils/test_rgbtools.py::TestRGBTools::test_regular_array2rgbx_corder_from_f - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/utils/test_rgbtools.py::TestRGBTools::test_regular_array2rgbx_corder_from_c_slices - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/utils/test_rgbtools.py::TestRGBTools::test_regular_array2rgbx_cordermask_from_cmasked - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/utils/test_rgbtools.py::TestRGBTools::test_regular_array2rgbx_cordermask_from_cmasked_slices - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/utils/test_utils.py::test_msxml_sanitization - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/utils/test_utils.py::test_default_x2d - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/utils/test_utils.py::test_skip_interchild_text_flatten - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/utils/test_utils.py::test_concat_interchild_text_val_flatten - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/utils/test_utils.py::test_list_interchild_text_val_flatten - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/utils/test_utils.py::test_wrong_type_x2d_initiation - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/utils/test_utils.py::test_d2s_fail - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/utils/test_utils.py::test_d2s_dtype - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/utils/test_utils.py::test_d2s_extra_dict_ok - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/utils/test_utils.py::test_d2s_sarray - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/utils/test_utils.py::test_d2s_partial_sarray - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/utils/test_utils.py::test_d2s_type_cast_ok - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/utils/test_utils.py::test_d2s_type_cast_invalid - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/utils/test_utils.py::test_d2s_string_cut - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/utils/test_utils.py::test_d2s_array1 - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/utils/test_utils.py::test_d2s_array2 - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/utils/test_utils.py::test_d2s_arrayX - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/utils/test_utils.py::test_serial_date_to_ISO_format - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/utils/test_utils.py::test_ISO_format_to_serial_date - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/utils/test_utils.py::test_datetime_to_serial_date - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/utils/test_utils.py::test_get_date_time_from_metadata - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/utils/test_utils.py::test_get_chunk_slice[shape0] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/utils/test_utils.py::test_get_chunk_slice[shape1] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/utils/test_utils.py::test_get_chunk_slice[shape2] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/utils/test_utils.py::test_get_chunk_slice[shape3] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_import.py::test_dir_plugins[plugin6] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_import.py::test_dir_plugins[plugin7] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_import.py::test_dir_plugins[plugin8] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_import.py::test_dir_plugins[plugin9] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_import.py::test_dir_plugins[plugin10] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_import.py::test_dir_plugins[plugin11] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_import.py::test_dir_plugins[plugin12] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_import.py::test_dir_plugins[plugin13] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_import.py::test_dir_plugins[plugin14] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_import.py::test_dir_plugins[plugin15] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ERROR tests/test_import.py::test_dir_plugins[plugin16] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc ================== 37 skipped, 91 errors in 81.02s (0:01:21) =================== ```
CSSFrancis commented 1 week ago

@ericpre What is the action item here? I can remove the # from the file names. That is actually just a default for the virtual detector name so it shouldn't effect any of the auto laoding.

CSSFrancis commented 1 week ago

pre-commit.ci autofix

CSSFrancis commented 1 week ago

I'll avoid self merging but @ericpre feel free to merge if you think this is good.

ericpre commented 1 week ago

Renaming the files is the most simplest here and the issue with # in the filename should be handle in pooch.