aliciaclark1066
commented
1 year ago @richard-russell Thanks for submitting a request! We've received it and will prioritize it when able. The team is focused on some high priority deliverables so it will take some time before we can get to this.
What problem would you like to address?
Hcdiag is unable to fully consume Terraform Enterprise APIs due to the maximum API page size of 100 items. Hcdiag offers a REST GET method but can only request one page at a time, with hard-coded page number and page size. Most medium to large customers will have more than 100 workspaces, so this prevents hcdiag from profiling the number of resources per workspace, for example. It would be useful if hcdiag had a method to pull and coalesce the API payload for all pages.
Who is the audience for this request?
This would be useful for Solution Architects and Solution Engineers, as it would unlock more detailed reporting on workspace and run metrics.
Which HashiCorp products does this affect?
This affects many (all?) v2 Terraform Enterprise APIs. I can't comment on whether other HashiCorp products also use API pagination.
What would the ideal solution look like?
Ideally hcdiag would have a way to handle the API pagination and return a payload consisting of the aggregate responses from pages. This could be in the form of:
What alternatives have you considered?
hcdiag-ext pulls a single page with page size = 1, which is fine as long as we only care about metadata / summary info, but does not give access to details. It would also be possible to hard code collection of multiple pages, but this is clunky and does not scale, nor solve the general problem where the number of pages is unknown, and different for every customer.
How does this impact you?
Without this, we cannot pull detailed information about any TFE resources with more than 100 instances, and can only report on summary numbers. This info would be useful for baselining what typical usage looks like, and for highlighting outliers.
What are the customer data privacy implications for this request? This would make no difference to the type of information gathered. It would simply allow it to be complete instead of for an arbitrary 100 instances.
Are there any deadlines? No deadlines.
Is there any additional context?
No further context