mtreinish / ciml

a machine learning pipeline for analyzing CI results.
Apache License 2.0
5 stars 4 forks source link

Connection time out #55

Closed jlousada315 closed 4 years ago

jlousada315 commented 4 years ago

Now, when I run the command I get this multiple times without stopping

Connection time out on https://openstack.fortnebula.com:13808/v1/AUTH_e8fd161dc34c421a979a9e6421f823e9/logs_15/677915/1/gate/tempest-full/be64459/controller/logs/dstat-csv_log.txt.gz Connection time out on https://openstack.fortnebula.com:13808/v1/AUTH_e8fd161dc34c421a979a9e6421f823e9/logs_15/677915/1/gate/tempest-full/be64459/controller/logs/dstat-csv_log.txt

Originally posted by @johnnylousas in https://github.com/mtreinish/ciml/issues/53#issuecomment-595231096

jlousada315 commented 4 years ago

and now after 20 minutes running I got : Check-point 350/2466, saved 66, skipped 284 raise ReadTimeout(e, request=request) requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='46449bb2bbb3addfe46a-8a2792774c357b262a61e0059b7e8a58.ssl.cf2.rackcdn.com', port=443): Read timed out. (read timeout=15)

kwulffert commented 4 years ago

@johnnylousas Some extra background. The data history of the OpenStack test result database goes back much further than the available log files. This is usually done by the OpenStack because because log files are too large to keep around for a long time. This is why you may encounter files that are not there. We handle several type of errors, from 404 to no connection (recently added for issue #53). The error found here is related to ReadTimeout. I added the handling of it too. When running the command after the latest pull request, the process will go faster as you have already cached several logs.