search

theforeman / puppet-pulp

Puppet module for setting up Pulp 2 as part of Katello installation
GNU General Public License v3.0
16 stars 66 forks source link

Massively speed up providers and fix timeouts #334

Closed alexjfisher closed 6 years ago

alexjfisher commented 6 years ago

In my environment, get_repo_info was failing with...

Error: Failed to apply catalog: https request threw exception Net::ReadTimeout. Connection details: url=https://pulp.example.com/pulp/api/v2/repositories/oel-6-x86_64-latest/?details=True,cert_path=/root/.pulp/user-cert.pem.

details=True can be very slow. It causes huge amounts of processing and we don't use the results. Using ?distributors=True&importers=True instead is much faster.

I've got repos where ?distributors=True&importers=True returns in around a second whilst details=True took > 4 minutes (testing the REST API manually with curl).

The returned json (for an RPM repo) is missing just 2 fields compared to details=True. These are:

See https://docs.pulpproject.org/dev-guide/integration/rest-api/repo/retrieval.html#retrieve-a-single-repository

This change has been tested with pulp_rpmrepo only, but I think should also be fine with the two other providers (pulp_isorepo and pulp_puppetrepo).

My puppet runs had started failing most times, and sucessful runs were taking around 10 minutes. After this fix, puppet runs are now less than a minute.

ekohl commented 6 years ago

Thanks!