HEPCloud / decisionengine

HEPCloud Decision Engine framework
Apache License 2.0
6 stars 25 forks source link

changes to fix source logging #670

Closed goodenou closed 1 year ago

goodenou commented 1 year ago

Individual Source loggers were failing because of the addition of ChannelWorkers as different multiprocessing processes.

Each of the SourceWorker objects is a multiprocessing.Process that has its own logger based on the SOURCELOGGERNAME. Each SourceWorker also has a dict of Sources, each of which has its own logger which was based on the CHANNELLOGGERNAME, which is actually in another multiprocessing.Process. Any logging done by the Sources was thus not getting recorded.

I changed the design so that now the Sources get their logger from the SourceWorker; the logger is passed through as a parameter in the Source initialization.

lgtm-com[bot] commented 1 year ago

This pull request introduces 1 alert when merging bd863ccc4e29cf7ffc324a55137944f4d18a788f into e0749d8b83c65345da9e18ddaa04f922c2b3c162 - view on LGTM.com

new alerts:

codecov[bot] commented 1 year ago

Codecov Report

Base: 97.27% // Head: 97.28% // Increases project coverage by +0.00% :tada:

Coverage data is based on head (6431c64) compared to base (e0749d8). Patch coverage: 100.00% of modified lines in pull request are covered.

:exclamation: Current head 6431c64 differs from pull request most recent head f120e0f. Consider uploading reports for the commit f120e0f to get more accurate results

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #670 +/- ## ======================================= Coverage 97.27% 97.28% ======================================= Files 50 50 Lines 3123 3133 +10 Branches 561 563 +2 ======================================= + Hits 3038 3048 +10 Misses 54 54 Partials 31 31 ``` | Flag | Coverage Δ | | |---|---|---| | python-3.10 | `97.22% <100.00%> (+<0.01%)` | :arrow_up: | | python-3.6 | `96.85% <100.00%> (-0.12%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=HEPCloud#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/HEPCloud/decisionengine/pull/670?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=HEPCloud) | Coverage Δ | | |---|---|---| | [...c/decisionengine/framework/engine/SourceWorkers.py](https://codecov.io/gh/HEPCloud/decisionengine/pull/670/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=HEPCloud#diff-c3JjL2RlY2lzaW9uZW5naW5lL2ZyYW1ld29yay9lbmdpbmUvU291cmNlV29ya2Vycy5weQ==) | `97.76% <100.00%> (-0.02%)` | :arrow_down: | | [...rc/decisionengine/framework/modules/EmptySource.py](https://codecov.io/gh/HEPCloud/decisionengine/pull/670/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=HEPCloud#diff-c3JjL2RlY2lzaW9uZW5naW5lL2ZyYW1ld29yay9tb2R1bGVzL0VtcHR5U291cmNlLnB5) | `100.00% <100.00%> (ø)` | | | [src/decisionengine/framework/modules/Source.py](https://codecov.io/gh/HEPCloud/decisionengine/pull/670/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=HEPCloud#diff-c3JjL2RlY2lzaW9uZW5naW5lL2ZyYW1ld29yay9tb2R1bGVzL1NvdXJjZS5weQ==) | `100.00% <100.00%> (ø)` | | | [...cisionengine/framework/taskmanager/module\_graph.py](https://codecov.io/gh/HEPCloud/decisionengine/pull/670/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=HEPCloud#diff-c3JjL2RlY2lzaW9uZW5naW5lL2ZyYW1ld29yay90YXNrbWFuYWdlci9tb2R1bGVfZ3JhcGgucHk=) | `100.00% <100.00%> (ø)` | | Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=HEPCloud). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=HEPCloud)

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.