A canned emails should be sent when a storage increase request is approved

[jleach]

Describe the issue

When certain types of quota change requests are approved or rejected it would be very useful to send canned replies with key information for the team. For example, when team requests additional storage it would be very useful to email the following to the team:

Subject: Quota Request for ABC Forms Improvement Project.

Hi,

Your request for storage quota in the abc123-dev namespace has been reviewed. It seems like a reasonable
request so it has been approved. Please remember that on OpenShift 4.x you can now **dynamically grow PVCs**.
This means you should be setting up monitoring to alert you when a PVC is approaching 80% capacity and
then grow it in size to last into the medium term.

Thanks in Advance,

Platform Services Registry.

Additional context Jason will provide 3 examples of the email text for Zhanna to implement.

n/a

Definition of done

As a registry administrator, when I approve a storage quota increase request I would like a canned email sent to outline expectations and best practices.

[funtigr]

For now we have a Quota Changed e-mail. Not sure if that's what was meant.

And after rejection we get this one:

[Iryna-Kaplun]

Hey team! Please add your planning poker estimate with Zenhub @okanji @Kolezhanchik @funtigr

[jleach]

@Kolezhanchik I was thinking, for a crawl-walk-run approach we could just update the message that is sent out when an edit request is approved with a message that includes all three strategies, one for each of CPU, memory, and disk. Maybe requiring different messages in different circumstances is overkill. I've written the sample below:

[jleach]

Hi,

We've reviewed, evaluated, and approved your quota change request. Our goal is to assist platform teams in becoming more responsible stewards of shared platform resources. By using resources efficiently, we can support more projects on the platform without needing additional hardware and licenses. Plus, improved hardware utilization helps reduce our cluster's carbon footprint, aligning with our climate action objectives.

Pro-tip 🤓

To maximize your project's cluster resource usage, we've compiled some best practices to follow. If you'd like a one-on-one meeting with a platform resource specialist, please use this calendar link to book a session: https://calendly.com/jason-leach/30min

CPU Utilization

Paying attention to CPU resource allocation on the cluster is crucial. When adjusting your application's CPU resources, consider the following strategies for effective management:

1. Aim to set the requested cpu (request.cpu) at around 80% of what your application uses, plus an additional 20%. For instance, if your application uses 50m from Monday to Friday, 9-5, set the "request.cpu" value to 60m.
2. Avoid setting the requested cpu below 10m or 20m, as there is limited value in going too low, and it may have some drawbacks.
3. Be mindful of multithreaded applications or multi-process containers, as threads and processes can consume quota more quickly, resulting in application throttling.
4. Keep in mind that the requested cpu (request.cpu) does not impact your application's performance; only the CPU limit (limit.cpu) can do this.

Disk Storage

With Openshift 4.x, you can now dynamically expand Persistent Volume Claims (PVC), allowing you to start with limited disk storage and increase it as needed. To monitor PVCs effectively, use SysDig to keep track of capacity and alert your team when it reaches around 80%. Then, you can adjust the size to accommodate future demands. If you need help setting up SysDig alerts for your PVCs, please reach out to Billy Li at CITZ:EX for assistance.

Memory

Applications usually require a specific amount of memory, and it's often challenging to reduce this requirement. However, there are essential strategies to manage memory resources effectively:

Once your application has been running for a while and you understand its memory usage pattern, adjust the "request.memory" value to be approximately 20% more than what your application needs. Use the "limit.memory" setting as a tool for detecting memory leaks. Avoid running your application consistently above the "request.memory" value, as it could lead to the application being moved between nodes when they adjust their workloads.

[Kolezhanchik]

hi @jleach thank you for the wording, I'm going to include e.g. "CPU Utilization..." if CPU for any of namespaces was change, "Disk Storage..." if storage for any of namespace was changed, I'm going to discuss with @alexandralloyd where on email layout it should be added. Not sure about "Pro-tip" and "hi" parts, should "pro-tip" part be included to any quota changed confirmation email?

[funtigr]

After Quota Change is approved and provisioned the e-mail is received by contacts: