Want to access one of the builder nodes

[mohit84]

Recently we have observed that multiple test cases are failing because of timed out, we want to analyze the issue. To debug it more we want to access one of the builder nodes for some days (1 or 2 days).

@mscherer Can you please share one builder node with us to find out RCA?

[mscherer]

Sure, but 1) to whom 2) isn't it what softserve is supposed to do ?

[mohit84]

Sure, but 1) to whom 2) isn't it what softserve is supposed to do ?

You can assign to me, Is builder server configuration similar to sofserve?. We have observed sometimes the test case is not failing on softserve VM in 5 to 10 attempts but in builder environment is failing on 1st or 2nd attempt. The other reason is the softserve VM is available only for 4hrs so sometimes it is difficult to debug an issue. We want to keep the environment for more than 1 day.

[mscherer]

I think softserve should be more than 4h, cause that's really short. But sure, I can get you a node, and reinstall after :)

[xhernandez]

@mscherer Mohit is on PTO today and it seems all regressions from last 3 days are failing on all nodes. Can you also give me access to try to see what's happening ?

Thanks

[xhernandez]

It seems that tests are starting to pass again and execution time of each test is roughly 1/4th of the time it was before. I'm not sure what changed.

[xhernandez]

@mscherer Failures happen again and the tests are running very slow. Please give me access to one of the builders when you can.

Thanks

[mscherer]

So I shared builder-c7-3.aws.gluster.org . While it should be fine and removed from rotation, tell me if there is anything weird happening (like if ansible revert some changes, etc).

Tell me when you finished, so I can reinstall it.