Enhance visibility of missing dependencies

pnuu commented 1 month ago

Curently PyYAML hides import errors in instantiated classes, and these are only logged at DEBUG level. This PR changes the log level to ERROR for these cases.

[x] Tests added

pnuu commented 1 month ago

This was the original log message when defusedxml was not installed for avhrr_l1b_eps reader. Note that the crucial info - missing defusedxml library - is reported in a DEBUG level message.

[DEBUG: 2024-10-16 11:01:28 : satpy.readers.yaml_reader] Reading ('/home/lahtinep/Software/pytroll/pytroll_packages/satpy/satpy/etc/readers/avhrr_l1b_eps.yaml',)
[INFO: 2024-10-16 11:01:28 : satpy.readers] Cannot use ['/home/lahtinep/Software/pytroll/pytroll_packages/satpy/satpy/etc/readers/avhrr_l1b_eps.yaml']
[DEBUG: 2024-10-16 11:01:28 : satpy.readers] while constructing a Python object
cannot find module 'satpy.readers.eps_l1b' (No module named 'defusedxml')
  in "/home/lahtinep/Software/pytroll/pytroll_packages/satpy/satpy/etc/readers/avhrr_l1b_eps.yaml", line 144, column 22
[WARNING: 2024-10-16 11:01:28 : satpy.readers] Don't know how to open the following files: {'/home/lahtinep/data/satellite/polar/avhrr_l1b_nat/AVHR_xxx_1B_M01_20241015100703Z_20241015114603Z_N_O_20241015105547Z.nat'}
Traceback (most recent call last):
  File "/home/lahtinep/bin/test_avhrr_l1b_nat.py", line 16, in <module>
    main()
  File "/home/lahtinep/bin/test_avhrr_l1b_nat.py", line 12, in main
    scn = Scene(reader="avhrr_l1b_eps", filenames=[fname])
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/lahtinep/Software/pytroll/pytroll_packages/satpy/satpy/scene.py", line 155, in __init__
    self._readers = self._create_reader_instances(filenames=filenames,
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/lahtinep/Software/pytroll/pytroll_packages/satpy/satpy/scene.py", line 176, in _create_reader_instances
    return load_readers(filenames=filenames,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/lahtinep/Software/pytroll/pytroll_packages/satpy/satpy/readers/__init__.py", line 590, in load_readers
    _check_reader_instances(reader_instances)
  File "/home/lahtinep/Software/pytroll/pytroll_packages/satpy/satpy/readers/__init__.py", line 629, in _check_reader_instances
    raise ValueError("No supported files found")
ValueError: No supported files found

codecov[bot] commented 1 month ago

Codecov Report

Attention: Patch coverage is 93.33333% with 3 lines in your changes missing coverage. Please review.

Project coverage is 96.08%. Comparing base (b8991c0) to head (3db6547). Report is 112 commits behind head on main.

Files with missing lines	Patch %	Lines
satpy/readers/__init__.py	86.95%	3 Missing :warning:

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #2931 +/- ## ======================================== Coverage 96.07% 96.08% ======================================== Files 373 375 +2 Lines 54491 54612 +121 ======================================== + Hits 52352 52472 +120 - Misses 2139 2140 +1 ``` | [Flag](https://app.codecov.io/gh/pytroll/satpy/pull/2931/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pytroll) | Coverage Δ | | |---|---|---| | [behaviourtests](https://app.codecov.io/gh/pytroll/satpy/pull/2931/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pytroll) | `3.98% <6.66%> (-0.01%)` | :arrow_down: | | [unittests](https://app.codecov.io/gh/pytroll/satpy/pull/2931/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pytroll) | `96.17% <93.33%> (+<0.01%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pytroll#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

mraspaud commented 1 month ago

Is there a more fine-grained exception we could catch instead, for just building objects (which probably mean a missing import)?

pnuu commented 1 month ago

I don't know. I'll have a look after figuring out the CodeScene complaint.

coveralls commented 1 month ago

Pull Request Test Coverage Report for Build 11366063321

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

For more information on this, see Tracking coverage changes with pull request builds.
To avoid this issue with future PRs, see these Recommended CI Configurations.
For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

42 of 45 (93.33%) changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.007%) to 96.177%

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
satpy/readers/init.py	20	23	86.96%
<!--	Total:	42	45	93.33%	-->

Totals
Change from base Build 11324402105:	0.007%
Covered Lines:	52708
Relevant Lines:	54803

💛 - Coveralls

pnuu commented 1 month ago

https://github.com/pytroll/satpy/pull/2931/commits/ca987bc50a932b15192f483062f554562bf700bc should make CodeScene happier. Later today I'll look if there are more fine-grained failures we could check for, but I have a feeling there is none as YAMLError was there also originally.

mraspaud commented 1 month ago

https://github.com/yaml/pyyaml/blob/69c141adcf805c5ebdc9ba519927642ee5c7f639/lib/yaml/constructor.py#L552 -> ConstructorError?

pnuu commented 1 month ago

ConstructorError seems to work perfectly :+1:

pytroll / satpy