search

confluentinc / ducktape

System integration and performance tests
11 stars 93 forks source link

Upgrade paramiko version to 2.10.0 #312

Closed lbradstreet closed 2 years ago

lbradstreet commented 2 years ago

This fixes pem file issues with later ubuntu installs.

sonupillai commented 2 years ago

https://www.paramiko.org/changelog.html

Changelog looks safe. I dont think there is an easy way to test ducktape within jenkins. Locally you can swap out with your local ducktape version.

stan-is-hate commented 2 years ago

@lbradstreet have you tested this with our tests? A long time ago we had to lock paramiko version because it introduced failures, but that was many versions ago. I've already merged the changes that introduce requirements.txt so I'll fix merge commits myself, but I'd like to understand testing done before merging. Also if the testing was done on 0.7.x or 0.8.x/master. It's ok if you haven't tested it on all the branches yet, I'll take care of this

stan-is-hate commented 2 years ago

I've updated the PR to include all the latest changes and retargeted it to 0.8.x. Will kick off a couple of jenkins runs with our system tests just to be sure it's reliable.

lbradstreet commented 2 years ago

@stan-confluent do you remember what kinds of errors you were hitting? I've tested this fairly thoroughly with ubuntu 22, but not with the older AMIs.

stan-is-hate commented 2 years ago

@lbradstreet I don't remember the details at this point, can dig them up if needed, but the test runs I did were ok, some issues happened which don't seem to be related to this change. However, I'll do a couple of reruns just in case.

stan-is-hate commented 2 years ago

I've triggered a couple of reruns, one with old paramiko, one with a new one. If needed, I'll run more times. My test runs ran into some weird issue with SR not stopping, but that might just be a test issue and not paramiko issue, so I'll confirm with old paramiko.

lbradstreet commented 2 years ago

I've triggered a couple of reruns, one with old paramiko, one with a new one. If needed, I'll run more times. My test runs ran into some weird issue with SR not stopping, but that might just be a test issue and not paramiko issue, so I'll confirm with old paramiko.

Thanks

stan-is-hate commented 2 years ago

I did a couple test runs and they seem ok. I would suggest releasing this change as part of 0.9.x release and branch cut, @imcdo @sonupillai wdyt? I can release all the outstanding changes as a new 0.8.x release, and then release this one + ssh timeout as part of 0.9.x release

stan-is-hate commented 2 years ago

Unless we want to also roll out new paramiko version for older kafka versions that still use ducktape 0.7.x and python 2 - kafka 2.6 and below, confluent 6.0.x and below.

lbradstreet commented 2 years ago

Personally I'm fine with releasing for the python3 versions first and evaluating a backport later if we feel it's helpful. The current paramiko is mostly blocking my AMI upgrades and python2 testing would take a lot longer.

stan-is-hate commented 2 years ago

@sonupillai @imcdo if you don't have any objections, I'll release both this one and ssh connect timeout as 0.9.x later today, first this one, then ssh timeout, as separate releases.

stan-is-hate commented 2 years ago

Well, best laid plans! I still want to release this, but other things got in the way. Hope to do it before the end of the week though.