Content: Concourse - Sizing requirements

[LukeShortCloud]

Summary of content

Concourse requires a handful of sizing requirements for all of the resources (CPU, RAM, and storage). These need to be taken into account when first deploying Concourse.

Summary of audiences

Developers and administrators deploying and managing Concourse in a production environment where scale and performance are important.

Level of content

Level of content as it pertains to the topic proposed.

• [x] 101: "Awareness / Exploration"
• [x] 201: "Prototyping / Deploying"
• [x] 301: "Scaling
• [ ] Content Series: To include multiple levels of content (Note: Series pieces should either be able to be consumed independently, or very explicit in the title, and with links in the body of the content that they are part of a series)

Contributor Checklist

1. [x] I have added a card to the GitHub Content Backlog and/or moved it to the proper column. See Content Prewriting and Ideation
2. [ ] If contributing via Google Docs: I have added a link to my document to this issue, and set the document's Share settings to "Anyone on the internet with this link can comment".
3. [ ] My content is drafted in accordance with existing style guides and best practices. See Drafting Content
4. [ ] The front matter (metadata) is filled out completely (e.g. description, topics, tags, author, etc - as appropriate for the content type). See Drafting Content
5. [ ] My content has been reviewed by my team for technical accuracy. See Editing Content
6. [ ] My content has beed reviewed and edited by a copy editor. See Editing Content
7. [ ] If applicable: My content has gone through a legal review (VMware Employees Only). See Editing Content
8. [ ] If applicable: I have tested my content locally and everything passes. See Testing Content
9. [ ] Optional: I have filled out the Social Media request form (optional) (VMware Employees Only). See Next Steps
10. [ ] If applicable: I have created an author page and opened a pull request to get it added. See Next Steps

Reviewer Checklist

• [ ] Review if the content is a good fit for the Tanzu Developer Center. See Content Prewriting and Ideation
• [ ] There this GitHub issue has been added to the content backlog project
• [ ] Ensure with the author that the content has been reviewed for technical accuracy.
• [ ] Ensure with the author that the content has been through copy editing.
• [ ] If applicable: Ensure with the author that the content has been through Legal review.
• [ ] Review fit with the Tanzu Developer Center Style Guide and the VMware Style Guide (VMware Employees Only).
• [ ] Ensure the front matter is filled out correctly and completely.
• [ ] If applicable: Review additions to custom_dict.txt.
• [ ] Ensure with the author they have reviewed the staging site for correctness. The reviewer should review this too.
• [ ] If applicable: Review any new Team pages.
• [ ] Optional: Ensure the author has provided copy for Social Media distribution.\
• [ ] If contributing via Google Docs: Send the staging link to the author to review. The reviewer should review this too. Get the final 👍 from the author to push the content public.

[LukeShortCloud]

The following are recommendations straight from Concourse developers.

The two biggest areas of performance bottlenecks in Concourse: network and storage.

Network:

• Assuming the underlying network infrastructure is not the bottleneck, create more local worker hosts to spread out network traffic.
• Use Grafana or another telemetry tool to monitor the network traffic on all of the hosts (especially the worker hosts).

Storage:

• 1 TB is the suggested amount of storage space to allocate.
  • The majority of the space is used by Concourse cache.
  • If using 500 GB (or anything below 1 TB), it is recommended to clean up the worker hosts frequently by "re-rolling" them.
• Use Grafana or another telemetry tool to monitor the worker hosts for storage usage and capacity planning.

Database:

• PostgreSQL should have a sane retention policy for audit logs. Alternatively, setup and enable an "event nuzzle" to send the logs to an external source.
• Increase max connections. This is a common issue.
• A single database instance is fine even for large-scale deployments.

Remote workers:

• Not recommended for large-scale Concourse clusters. Keep everything local for the best performance.

[LukeShortCloud]

We can have a relative link to here for how to setup Grafana.

[LukeShortCloud]

Increase the maximum number of connections to PostgreSQL by setting max_connections. The default value is 100. Increase to 1000.

https://www.postgresql.org/docs/13/runtime-config-connection.html

[LukeShortCloud]

PostgreSQL data retention can be configured by (1) archiving and (2) setting up partitions in the tables.

1. https://www.postgresql.org/docs/13/continuous-archiving.html
2. https://www.postgresql.org/docs/13/ddl-partitioning.html