search

openshift / vagrant-openshift

Other
83 stars 117 forks source link

`git status` instead of `git fsck` for performance #537

Closed dobbymoodge closed 7 years ago

dobbymoodge commented 7 years ago

git fsck is very CPU-intense on large complex repos (like openshift/origin). This change uses git status to check repo viability instead, hopefully ameliorating the serious performance issues we've been seeing in Jenkins.

dobbymoodge commented 7 years ago

@stevekuznetsov @smarterclayton

stevekuznetsov commented 7 years ago

How many of these run these days anymore? Also, what was the point of this stanza? The two aren't equivalent in when they will expose errors.

dobbymoodge commented 7 years ago

@stevekuznetsov These tests all run vagrant origin-local-checkout:

base_ami
devenv_ami
fork_ami
jenkins
mongodb
mysql
openldap
postgresql
sti-nodejs
sti-perl
sti-php
sti-python
sti-ruby
build-and-release-latest-openshift-ansible-images
origin_extended
origin_coverage
networking_extended
build-and-release-latest-origin-metrics
gssapi_extended
ldap_groups_extended
openshift-pipeline-plugin
origin_extended_build_tests
origin_extended_image_tests
s2i-base
s2i-wildfly
s2i_verify_godeps
test-customer-diagnostics
test-openshift-client-plugin
test-openshift-pipeline-plugin
test-openshift-sync-plugin
test-origin-aggregated-logging
test-origin-metrics
test_pr_origin_extended
test_pull_requests_origin_check
test_pull_requests_origin_conformance
test_pull_requests_origin_gce
test_pull_requests_origin_integration
test_pull_requests_origin_networking
test_pull_requests_origin_web_console
test_pull_requests_s2i
test_pull_requests_s2i_windows
test_pull_requests_vagrant_openshift
vendor_origin_web_console
test_pull_requests_origin_future

I'm not sure how often, but when it does happen it definitely impacts jenkins' responsiveness.

The point of the stanza is to discover if it's safe to run git gc or if the repo needs to be rm -rfed and re-cloned. git status will fail for the kinds of problems we've seen (git gc gets interrupted and leaves corrupt index) and for whatever cases it doesn't catch, we can handle manually.

Right now running git fsck against openshift/origin pegs the CPUs for minutes at a time, and it is very noticeable in the UI. This change ought to help avoid that.

smarterclayton commented 7 years ago

SGTM

stevekuznetsov commented 7 years ago

LGTM

dobbymoodge commented 7 years ago

[test]

openshift-bot commented 7 years ago

Evaluated for vagrant openshift test up to a2e8efed61bbba36c29738cbe0c527d5531e69ab

openshift-bot commented 7 years ago

Vagrant OpenShift Test Results: Running (https://ci.openshift.redhat.com/jenkins/job/test_pull_requests_vagrant_openshift/100/) (Base Commit: 7b4be50102fb5cb583bcfe8272d2a21cc8593584)