search

redhat-openstack / tripleo-quickstart

Ansible roles for setting up TripleO virtual environments and building images
16 stars 15 forks source link

improvement: use get_url instead of running curl #62

Closed EmilienM closed 8 years ago

EmilienM commented 8 years ago

When downloading undercloud image, and probably other images, getting this meesage:

FAILED - RETRYING: TASK: setup/undercloud : Get undercloud image (18 retries left). Result was: {u'changed': True, u'end': u'2016-03-24 00:41:16.746223', 'failed': True, u'stdout': u'', u'cmd': [u'curl', u'-sf', u'-C-', u'-o', u'_undercloud.qcow2', u'https://ci.centos.org/artifacts/rdo/images/mitaka/delorean/stable/undercloud.qcow2'], u'rc': 18, u'start': u'2016-03-24 00:38:33.771564', u'stderr': u'', u'delta': u'0:02:42.974659', 'invocation': {'module_name': u'command', u'module_args': {u'creates': None, u'executable': None, u'_uses_shell': False, u'_raw_params': u'curl -sf -C- -o _undercloud.qcow2 https://ci.centos.org/artifacts/rdo/images/mitaka/delorean/stable/undercloud.qcow2', u'removes': None, u'warn': True, u'chdir': u'/home/stack/oooq_cache'}}, 'stdout_lines': [], u'warnings': [u'Consider using get_url module rather than running curl']}

It would be a better practice to use get_curl, instead of curl.

trown commented 8 years ago

This is actually intentional. The get_url module does not support retry with automatic resume, and the centos ci artifacts server will randomly drop connections. So when using get_url, the image download just failed for anyone without a very good connection to the artifacts server. (And would have failed and started over in your example above.) We actually had a few iterations to land on the current implementation:

First was get_url without any validation via md5 which exploded in a very non-obvious way much later when the connection dropped.

So we added an md5 check: https://github.com/redhat-openstack/tripleo-quickstart/commit/e4d25a336d298414eb2c52182b8bf9a47012b514

But then when the connection dropped, it would fail in an obvious way, but still fail over and over. So we switched to wget since it has built-in retry w/ resume: https://github.com/redhat-openstack/tripleo-quickstart/commit/a1e6e4dca8a2f72760ebbb1ed43ceaccfb8a57dd

But wget doesn't work with file:/// urls, so larsks made the current implementation using some magic flags and magic error codes for curl: https://github.com/redhat-openstack/tripleo-quickstart/commit/8c2a00809817a047bf312b72f390b5cb50ef9819

EmilienM commented 8 years ago

ok, that's a good reason for now :-)