Support OpenShift

[SimonGolms]

Original Report

Hi, I am currently working on an OpenShift solution. Before I publish anything I would like to know if PRs be welcome? Otherwise I would create a fork for other developers / users who are also interested in OpenShift support.

---

Update

Upstream Limitations

1. Clickhouse known to require "privileged"

TODO for Sentry Components

• [x] https://github.com/getsentry/snuba/pull/2696

[BYK]

@SimonGolms what does it entail? It's hard to speculate before seeing the patch but I can say for sure that our aim is to keep this repo focused on a single path and that is with docker-compose for now.

[BYK]

@SimonGolms, still interested?

[SimonGolms]

Hi @BYK, yes, still interested. I am currently working on the OpenShift support under my repo: https://github.com/SimonGolms/sentry-onpremise-openshift

• The templates can be proceed in OpenShift and will create a new Sentry instance.
• Due to some incompatibilities, I could not refer to the ./sentry/Dockerfile and added a new one.
• Automated tests with TravisCI are missing.

For me it doesn't feel like that the repo is in a mergeable state right now. But you and everyone else who is interested is welcome to take a look at the repo and leave a comment/commit/issue/PR so it could be merged and supported soon.

Feel free to close the issue.

[BYK]

Due to some incompatibilities, I could not refer to the ./sentry/Dockerfile and added a new one.

How can we help to remove these incompatibilities?

[BYK]

For me it doesn't feel like that the repo is in a mergeable state right now. But you and everyone else who is interested is welcome to take a look at the repo and leave a comment/commit/issue/PR so it could be merged and supported soon.

Seems like you're trying to mimic everything here including migrations from older versions. I don't think they add much value as we don't have a pre-existing OpenShift setup so I only expect fresh installs from this repo.

[SimonGolms]

How can we help to remove these incompatibilities?

I thought there was a problem with the ARG and fallback, as OpenShift doesn't support it https://github.com/getsentry/onpremise/blob/6538f472888bf0f8796faea649a9b32ca80ba959/sentry/Dockerfile#L1-L2

I just saw that the issue with autocommit got fixed yesterday: https://github.com/getsentry/sentry/pull/16816 and can now use getsentry/sentry:latest which will make the upper problem obsolete - can test it next week at the earliest!

[SimonGolms]

Seems like you're trying to mimic everything here including migrations from older versions. I don't think they add much value as we don't have a pre-existing OpenShift setup so I only expect fresh installs from this repo.

I also only expect fresh installs. However and as far as I know the migration step for postgres is still necessary with sentry upgrade --noinput.

At this step I was unsure: https://github.com/getsentry/onpremise/blob/6538f472888bf0f8796faea649a9b32ca80ba959/install.sh#L156-L163 At least Sentry had finally permissions to the uploaded SourceMaps after implementing this step. Agree this probably could have been fixed with chown -R sentry:sentry /data only.

[github-actions[bot]]

This issue has gone three weeks without activity. In another week, I will close it.

But! If you comment or otherwise update it, I will reset the clock, and if you label it Status: Accepted, I will leave it alone ... forever!

---

"A weed is but an unloved flower." ― Ella Wheeler Wilcox 🥀

[kprasadpn]

Hi @SimonGolms ,

I am trying to deploy sentry onprimise application on openshift , can i use your solution for this , my real problem while working with your descriptors is , my environment is managed by administrator on shared manner so root privilege is not possible . In this situation i am trying to clarify two things : 1) Do you have a working solution for this , if yes please share me the github repo path . 2) Is the root privilege on openshift container is mandatory ? do we have any other solution ?

It will be great if you can provide some answer on my queries .

I was opened a ticket for this , from there i got this link . Thank you @BYK for this .

[SimonGolms]

Hi @kprasadpn

can i use your solution for this

Sure, feel free to fork my code and start on your own solution

1. Do you have a working solution for this , if yes please share me the github repo path .
2. Is the root privilege on openshift container is mandatory ? do we have any other solution ?

The Sentry solution consists of several microservices, which are based on other containers that are developed and maintained by other developers/teams. You could submit suggestions for improvement via issues or PR for adjusted user permissions. Or you could create and use your own images for OpenShift with the appropriate user permissions based on their Docker images. However, both of these might be time consuming tasks.

[BYK]

Or you could create and use your own images for OpenShift with the appropriate user permissions based on their Docker images.

@SimonGolms we have recently removed the need to locally build Sentry images. We are also hoping to remove the need to build cron images locally. Is this helping or hurting your OpenShift support? What can we do to make this easier for you folks?

[APCG]

Is there a solution available that does not require root previleges to deploy Sentry on Openshit?

[i5okie]

Would be amazing to get rid of root privileges on pods. Need to deploy to Openshift. Root privilege requirement is an absolute deal-breaker.

[bittner]

What the is technical reason for root permissions? Sentry doesn't have to deal with privileged ports, in privileged parts of the network layer or the kernel, does it? :thinking:

[chadwhitacre]

Is the root requirement something baked in on the Sentry side? Are there changes required in Sentry repos to support this?

[joeltraber]

It's spread in different containers.. can't remember all the details but clickhouse definitely makes problems:, also see https://github.com/sentry-kubernetes/charts/issues/67

Here is a list of what may cause problems in OpenShift: https://docs.openshift.com/container-platform/4.10/authentication/managing-security-context-constraints.html#default-sccs_configuring-internal-oauth

To be secure in OpenShift we should not need to set any of these security-context except "restricted" which is the default. To get sentry running I had to use "privileged".

[chadwhitacre]

Clickhouse is upstream. Anyone know of Sentry-owned components that can't run "restricted"? If not I think we can close this out with a note about upstream limitations in the ticket description.

[Iridescens]

As per https://github.com/sentry-kubernetes/charts/issues/67 i have almost working Helm chart with only nonroot scc applied to default, sentry-sentry-redis, sentry-rabbitmq, sentry-kafka service accounts. Not even anyuid scc is needed. Default SA can also be stripped of nonroot when/if all elements could use dedicated SA. That being said, running this modified chart, deployment sentry-snuba-api produces error: failed switching to "snuba": operation not permitted which is a blocking issue since gosu is not permitted in OKD/OpenShift without extended permissions.. So the answer to question

Anyone know of Sentry-owned components that can't run "restricted"?

is "snuba" - it explicitly trying to switch users with gosu.

[chadwhitacre]

@getsentry/snuba-maintainers Can you speak to this? Seems that snuba uses gosu to switch users which makes it difficult (impossible?) to deploy Sentry in certain environments (OpenShift here). Is the gosu usage a hard requirement?

[evanh]

That usage is 2 years old. I suspect we could replace it. I can create a task for that internally.

[asottile-sentry]

after https://github.com/getsentry/snuba/pull/2696 -- snuba should no longer require gosu -- does this get things further?