CrayLabs / SmartSim

SmartSim Infrastructure Library.
BSD 2-Clause "Simplified" License
228 stars 36 forks source link

ML Worker Manager MVP #608

Closed ankona closed 3 months ago

ankona commented 3 months ago

This PR contains an ML worker manager MVP. The worker manager executes a single-threaded version of the planned ML pipeline for a single worker instance.

codecov[bot] commented 3 months ago

Codecov Report

Attention: Patch coverage is 41.77215% with 138 lines in your changes missing coverage. Please review.

Please upload report for BASE (mli-feature@3c9915c). Learn more about missing BASE report.

Files with missing lines Patch % Lines
.../_core/mli/infrastructure/control/workermanager.py 18.58% 92 Missing :warning:
...e/mli/infrastructure/storage/dragonfeaturestore.py 0.00% 21 Missing :warning:
smartsim/_core/mli/infrastructure/worker/worker.py 78.20% 17 Missing :warning:
smartsim/_core/mli/comm/channel/channel.py 66.66% 4 Missing :warning:
smartsim/_core/mli/comm/channel/dragonchannel.py 63.63% 4 Missing :warning:
Additional details and impacted files [![Impacted file tree graph](https://app.codecov.io/gh/CrayLabs/SmartSim/pull/608/graphs/tree.svg?width=650&height=150&src=pr&token=96HFI2F45E&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=CrayLabs)](https://app.codecov.io/gh/CrayLabs/SmartSim/pull/608?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=CrayLabs) ```diff @@ Coverage Diff @@ ## mli-feature #608 +/- ## ============================================== Coverage ? 63.75% ============================================== Files ? 91 Lines ? 6425 Branches ? 0 ============================================== Hits ? 4096 Misses ? 2329 Partials ? 0 ``` | [Files with missing lines](https://app.codecov.io/gh/CrayLabs/SmartSim/pull/608?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=CrayLabs) | Coverage Δ | | |---|---|---| | [...m/\_core/mli/infrastructure/storage/featurestore.py](https://app.codecov.io/gh/CrayLabs/SmartSim/pull/608?src=pr&el=tree&filepath=smartsim%2F_core%2Fmli%2Finfrastructure%2Fstorage%2Ffeaturestore.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=CrayLabs#diff-c21hcnRzaW0vX2NvcmUvbWxpL2luZnJhc3RydWN0dXJlL3N0b3JhZ2UvZmVhdHVyZXN0b3JlLnB5) | `100.00% <100.00%> (ø)` | | | [smartsim/\_core/mli/comm/channel/channel.py](https://app.codecov.io/gh/CrayLabs/SmartSim/pull/608?src=pr&el=tree&filepath=smartsim%2F_core%2Fmli%2Fcomm%2Fchannel%2Fchannel.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=CrayLabs#diff-c21hcnRzaW0vX2NvcmUvbWxpL2NvbW0vY2hhbm5lbC9jaGFubmVsLnB5) | `66.66% <66.66%> (ø)` | | | [smartsim/\_core/mli/comm/channel/dragonchannel.py](https://app.codecov.io/gh/CrayLabs/SmartSim/pull/608?src=pr&el=tree&filepath=smartsim%2F_core%2Fmli%2Fcomm%2Fchannel%2Fdragonchannel.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=CrayLabs#diff-c21hcnRzaW0vX2NvcmUvbWxpL2NvbW0vY2hhbm5lbC9kcmFnb25jaGFubmVsLnB5) | `63.63% <63.63%> (ø)` | | | [smartsim/\_core/mli/infrastructure/worker/worker.py](https://app.codecov.io/gh/CrayLabs/SmartSim/pull/608?src=pr&el=tree&filepath=smartsim%2F_core%2Fmli%2Finfrastructure%2Fworker%2Fworker.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=CrayLabs#diff-c21hcnRzaW0vX2NvcmUvbWxpL2luZnJhc3RydWN0dXJlL3dvcmtlci93b3JrZXIucHk=) | `78.20% <78.20%> (ø)` | | | [...e/mli/infrastructure/storage/dragonfeaturestore.py](https://app.codecov.io/gh/CrayLabs/SmartSim/pull/608?src=pr&el=tree&filepath=smartsim%2F_core%2Fmli%2Finfrastructure%2Fstorage%2Fdragonfeaturestore.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=CrayLabs#diff-c21hcnRzaW0vX2NvcmUvbWxpL2luZnJhc3RydWN0dXJlL3N0b3JhZ2UvZHJhZ29uZmVhdHVyZXN0b3JlLnB5) | `0.00% <0.00%> (ø)` | | | [.../\_core/mli/infrastructure/control/workermanager.py](https://app.codecov.io/gh/CrayLabs/SmartSim/pull/608?src=pr&el=tree&filepath=smartsim%2F_core%2Fmli%2Finfrastructure%2Fcontrol%2Fworkermanager.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=CrayLabs#diff-c21hcnRzaW0vX2NvcmUvbWxpL2luZnJhc3RydWN0dXJlL2NvbnRyb2wvd29ya2VybWFuYWdlci5weQ==) | `18.58% <18.58%> (ø)` | |