search

Cloud-CV / EvalAI

:cloud: :rocket: :bar_chart: :chart_with_upwards_trend: Evaluating state of the art in AI
https://eval.ai
Other
1.77k stars 787 forks source link

[BugFix] Remove manage worker option #4416

Closed gchhablani closed 2 months ago

gchhablani commented 2 months ago

Scale worker is incorrect, and does not handle the resources/backend attributes gracefully. Removing it from frontend in this PR.

This feature can be re-added once the issues are fixed.

Similarly, start/stop worker is also problematic.

People start workers initially, and then switch to remote eval without stopping the workers. These workers have to be manually shut down which consumes credits, and is impractical.

We have auto-scaling, and these users should never need to start/stop/restart things as long as that script works fine. Also, we are working on auto-deciding the amount of resources to be allocated. Should be added soon, this will handle the scale workers issue.

codecov-commenter commented 2 months ago

:warning: Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 69.30%. Comparing base (96968d6) to head (f2b6141). Report is 1110 commits behind head on master.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #4416 +/- ## ========================================== - Coverage 72.93% 69.30% -3.63% ========================================== Files 83 20 -63 Lines 5368 3574 -1794 ========================================== - Hits 3915 2477 -1438 + Misses 1453 1097 -356 ``` [see 64 files with indirect coverage changes](https://app.codecov.io/gh/Cloud-CV/EvalAI/pull/4416/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) [see 64 files with indirect coverage changes](https://app.codecov.io/gh/Cloud-CV/EvalAI/pull/4416/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) ------ [Continue to review full report in Codecov by Sentry](https://app.codecov.io/gh/Cloud-CV/EvalAI/pull/4416?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://app.codecov.io/gh/Cloud-CV/EvalAI/pull/4416?dropdown=coverage&src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None). Last update [da7e068...f2b6141](https://app.codecov.io/gh/Cloud-CV/EvalAI/pull/4416?dropdown=coverage&src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None).