Allow Annotations in Helm chart

[daniel-spray]

What does this PR do?

Adds optional annotations to the helm chart deployment

Where should the reviewer start?

helm chart configs

How should this be manually tested?

Deployed on a test cluster where Values.yaml podAnnotations was empty. Also deployed where podAnnotations contained an annotation

Any background context you want to provide?

Shout out to @Gyuncho for the pull request! https://github.com/cloudability/metrics-agent/pull/260

What picture best describes this PR (optional but encouraged)?

What are the relevant Github Issues?

Developer Done List

• [ ] Tests Added/Updated
• [ ] Updated README.md
• [x] Verified backward compatible
• [ ] Verified database migrations will not be catastrophic
• [x] Considered Security, Availability and Confidentiality

For the Reviewer:

By approving this PR, the reviewer acknowledges that they have checked all items in this done list.

Reviewer/Approval Done List

• [ ] Tests Pass Locally
• [ ] CI Build Passes
• [ ] Verified README.md is updated
• [ ] Verified changes are backward compatible
• [ ] Reviewed impact to Security, Availability and Confidentiality (if issue found, add comments and request changes)

[GyunCho]

@daniel-spray Oh thanks Daniel for opening this PR! I definitely missed making changes to some other files like Chart.yaml.

Btw, can you also include this change please? readOnlyRootFilesystem: {{ .Values.readOnlyRootFilesystem }}

I will need to set readOnlyRootFilesystem: true to prevent root filesystem having more than read permission. After this, one will need to set emptyDir, but I thought that's up to a user.

More detail: https://stackoverflow.com/questions/57274253/how-to-exempt-a-directory-when-using-readonlyrootfilesystem-in-kubernetes

I also tested this works without any issue