pytroll / pytroll-collectors

Collector modules for Pytroll
GNU General Public License v3.0
3 stars 18 forks source link

Cleanup dependencies #101

Closed pnuu closed 2 years ago

pnuu commented 2 years ago

This PR cleans up dependencies of the package in the following way:

The main motivation is to reduce the number of necessary packages installed, for example when creating a container for a single task.

The use of pyresample.parse_area_file means that the area definition file isn't automatically located based on SATPY_CONFIG_PATH environment variable, and instead area_definition_file needs to be defined in the configuration file of Geographic Gatherer.

In addition unused pytroll_collectors.file_notifiers module were removed.

codecov[bot] commented 2 years ago

Codecov Report

Merging #101 (96c2dd4) into main (d85bcf9) will increase coverage by 1.24%. The diff coverage is 93.87%.

@@            Coverage Diff             @@
##             main     #101      +/-   ##
==========================================
+ Coverage   85.82%   87.07%   +1.24%     
==========================================
  Files          23       22       -1     
  Lines        3210     3202       -8     
==========================================
+ Hits         2755     2788      +33     
+ Misses        455      414      -41     
Impacted Files Coverage Δ
pytroll_collectors/triggers/__init__.py 57.14% <40.00%> (-12.86%) :arrow_down:
pytroll_collectors/geographic_gatherer.py 87.59% <100.00%> (+2.09%) :arrow_up:
pytroll_collectors/helper_functions.py 67.30% <100.00%> (ø)
...troll_collectors/tests/test_geographic_gatherer.py 100.00% <100.00%> (ø)
pytroll_collectors/tests/test_region_collector.py 96.74% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update d85bcf9...96c2dd4. Read the comment docs.

pnuu commented 2 years ago

Although the tests pass, I'll still try to run a live test with geographic gatherer to see that it really works.

pnuu commented 2 years ago

Confirmed to work with area_definition_file configured in the config file, and also confirmed to crash (and hang!) if it is not given. I'll add a check for the missing option

pnuu commented 2 years ago

And the latest commits should handle and report the missing config option.

pnuu commented 2 years ago

I'm trying to test what was the effect on Docker image size for segment_gatherer. Seem to be getting timeouts when updating things, so might take a while.

pnuu commented 2 years ago

The docker image size for segment_gatherer dropped from 1.97 GB to ~784~ 648 MB, so quite nice improvement. The base image I'm using is already 449 MB.

edit: forgot that segment_gatherer doesn't require Pyresample, so updated the image size.

pnuu commented 2 years ago

Also the size of Docker image for geographic_gatherer dropped from the 1.97 GB down to 826 MB with the latest commit that makes geographic_gatherer.py to work solely on Posttroll messages.