search

openshift / origin-server

OpenShift 2 (deprecated)
889 stars 516 forks source link

ensure that gears are moved to nodes with adequate free disk space #6382

Closed sallyom closed 8 years ago

sallyom commented 8 years ago

Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1122084 bz 1122084 If a gear is moved to a node and that node has no free space left after the move, the node becomes unusable. This PR ensures adequate free space + a buffer of free space on destination nodes when moving gears.

sallyom commented 8 years ago

WIP [test] please

sallyom commented 8 years ago

@Miciah, I've addressed your feedback. Thanks!!

sallyom commented 8 years ago

@a13m can you please review? Thanks.

Miciah commented 8 years ago

openshift-bot, please run the [extended:node] [test]s.

a13m commented 8 years ago

Thanks for working on this. I think I would have preferred a check like 

(DESTINATION_AVAILABLE_SPACE - SOURCE_GEAR_QUOTA) / DESTINATION_TOTAL_SPACE > 0.05

In other words, make sure that moving the gear to this node does not cause the disk usage to exceed 95% of its capacity (possibly even 90%). The way you have it written, it seems like we could have 1.03 GB free and then still write a 1 GB gear to the node. We don't like to cut things that close! I would be okay with making that threshold configurable in case OSE customers want something different.

Miciah commented 8 years ago

I see no need to make the threshold configurable unless a customer specifically requests it. 95% seems like a thoroughly reasonable threshold.

Miciah commented 8 years ago

@sallyom, the "Online Test Results: Waiting for stable build of 'libra_ami'" message means the image that we use for running tests is currently failing tests and needs to be rebuilt (possibly after fixing whatever problem is causing it to fail tests).

sallyom commented 8 years ago

@Miciah, awesome, thank you.

sallyom commented 8 years ago

[test] pleeeeeeese, Ms. openshift-bot

openshift-bot commented 8 years ago

Evaluated for online test up to bd8961c27c3d8d75dd0bf9caf614d320fdf64a86

openshift-bot commented 8 years ago

Online Test Results: SUCCESS (https://ci.dev.openshift.redhat.com/jenkins/job/test_pull_requests/9306/) (Extended Tests: node)

sallyom commented 8 years ago

@Miciah review please thanks

Miciah commented 8 years ago

Lovely jubbly! openshift-bot, please [merge]!

openshift-bot commented 8 years ago

Online Merge Results: SUCCESS (https://ci.dev.openshift.redhat.com/jenkins/job/merge_pull_requests/6731/) (Image: devenv_5792)

openshift-bot commented 8 years ago

Evaluated for online merge up to bd8961c27c3d8d75dd0bf9caf614d320fdf64a86