pytroll / trollflow2

Next generation Trollflow. Trollflow is for batch-processing satellite data using Satpy
https://trollflow2.readthedocs.org/
GNU General Public License v3.0
10 stars 15 forks source link

Move import to be dask-dist friendly #208

Closed gerritholl closed 2 months ago

gerritholl commented 4 months ago

Make trollflow2 more multiprocessing / dask-distributed friendly. Move the launcher import in satpy_launcher.py within the main function, so that it only gets executed in the primary process. The call to Manager() in trollflow2.__init__.py was causing attempts to use trollflow2 with dask distributed to fail with a RuntimeError.

I don't know how to add a unit test for this.

codecov[bot] commented 4 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 95.78%. Comparing base (c337b9e) to head (cb4cab3). Report is 4 commits behind head on main.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #208 +/- ## ======================================= Coverage 95.78% 95.78% ======================================= Files 12 12 Lines 2989 2989 ======================================= Hits 2863 2863 Misses 126 126 ``` | [Flag](https://app.codecov.io/gh/pytroll/trollflow2/pull/208/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pytroll) | Coverage Δ | | |---|---|---| | [unittests](https://app.codecov.io/gh/pytroll/trollflow2/pull/208/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pytroll) | `95.78% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pytroll#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.