(PE-37537) Add --rand-catalogs, --rand-facts to benchmark
These options will replace --rand-perc and provide for a flexible way to
specify the average frequency of different degrees of catalog and
factset mutations.
Factsets are difficult to work with as a CHANGE-COUNT because it's a
nested map that would take time to count and randomly select from. It's
easier for now to use the existing code that supplies a percentage
chance to change a leaf as it walks the factsets structure.
(PE-37537) Keep catalog mutation in balance by providing del of resources
Previously, catalog mutation was limited to adding or modifying
resources (and edges if include-edges) through catalog.utils functions
which would round trip through internal and wire format.
Over long running benchmark sessions, this causes catalogs to bloat
with added resources, slowing down benchmark itself and increasing load
on the system under test.
This patch changes the behavior to provide three cli.benchmark
functions, add-resource, mod-resource and del-resource so that deletions
can balance additions over the long term. Based on --rand-catalog
settings, a given catalog may have 0 or n resource changes, each of
which has an equal chance of being an add, mod or del call.
The intention is to maintain the overall shape and stability of catalog
sizes while still providing random changes over time.
To this end, additions clone an existing resource so as to introduce
resources of a size similar to existing resources. And we avoid adding
or deleting resources with blob parameters since this would distort
catalog shape. In these cases, we mod instead. A blob parameter may be
rebuilt to the same size, so they can still change over time in
resource_params.
The last catalog resource can't be deleted, a mod is performed instead.
Aside from the lower bound of 1, resource additions and removals are
equally weighted and should keep the total catalog sizes reasonably
balanced.
By default we don't include edges in catalogs anymore, but if
include-edges is true, then each added resource gets a single
containment edge with another random (non-cloned) resource, and each
deletion removes an added (cloned) resource if one exists along with
its single edge. This is done to avoid dealing with complexity in the
resource graph which must be valid when Puppetdb receives the catalog.
These options will replace --rand-perc and provide for a flexible way to specify the average frequency of different degrees of catalog and factset mutations.
Factsets are difficult to work with as a CHANGE-COUNT because it's a nested map that would take time to count and randomly select from. It's easier for now to use the existing code that supplies a percentage chance to change a leaf as it walks the factsets structure.
Previously, catalog mutation was limited to adding or modifying resources (and edges if include-edges) through catalog.utils functions which would round trip through internal and wire format.
Over long running benchmark sessions, this causes catalogs to bloat with added resources, slowing down benchmark itself and increasing load on the system under test.
This patch changes the behavior to provide three cli.benchmark functions, add-resource, mod-resource and del-resource so that deletions can balance additions over the long term. Based on --rand-catalog settings, a given catalog may have 0 or n resource changes, each of which has an equal chance of being an add, mod or del call.
The intention is to maintain the overall shape and stability of catalog sizes while still providing random changes over time.
To this end, additions clone an existing resource so as to introduce resources of a size similar to existing resources. And we avoid adding or deleting resources with blob parameters since this would distort catalog shape. In these cases, we mod instead. A blob parameter may be rebuilt to the same size, so they can still change over time in resource_params.
The last catalog resource can't be deleted, a mod is performed instead.
Aside from the lower bound of 1, resource additions and removals are equally weighted and should keep the total catalog sizes reasonably balanced.
By default we don't include edges in catalogs anymore, but if include-edges is true, then each added resource gets a single containment edge with another random (non-cloned) resource, and each deletion removes an added (cloned) resource if one exists along with its single edge. This is done to avoid dealing with complexity in the resource graph which must be valid when Puppetdb receives the catalog.