My experience on setting up self-host codecov

LeoQuote commented 1 year ago

First thanks to you guys in codecov, providing such an amazing product.

There are a few tips I found may save your time

1. Get a let's encrypt certificate or commercial certificate

At first, I tried to set up an HTTP site for codecov, but what I experienced is login would fail.(maybe I'm wrong about that)

So an HTTPS site is a must, and codecov uploader does not support self-signed certificate, so a let's encrypt certificate would help you.

2. Make sure to add sslmode=require when using timescaledb or pg(if ssl is enabled)

Codecov require a timescaledb to have full feature, if your timescaledb instance is provided by SaaS or a helm chart, there's high probability that it provides service with SSL enabled, if sslmode=require is not added, you would not start codecov properly.

3. Inject RUN_ENV=ENTERPRISE in worker container

If not set, links in pr comment would use app.codecov.io instead of your instance. See https://github.com/codecov/self-hosted/pull/14 for details

4. Override default comment layout

The default pr comment layout would be :

        "comment": {
            "layout": "reach,diff,flags,tree,reach",
            "behavior": "default",
            "show_carryforward_flags": False,
        },

https://github.com/codecov/shared/blob/5e3ba08c6e526846339d467dd8984c40281733c0/shared/config/__init__.py#L46C1-L50

Notice the duplicated reach, I personally advise you to use reach,diff,flags,tree, or there would be two coverage Grid pictures, which would take a lot of space in your pull request.

5. Build your own frontend/gazebo

There are plenty of configs that can only be configured during Build time, such as GHE_URL -> https://github.com/codecov/gazebo/blob/5e7147306dc0b297603fe97f2d9fece63d9dd8fa/src/shared/utils/provider.js#L85C1-L87

So if you want to have a normal frontend that can link you back to pull request, the only way to achieve that is build your own front-end.

you can refer to the official dockerfile when building your own image

# for GHE_URL, setting env would be sufficient
ENV REACT_APP_ENV=enterprise
ENV REACT_APP_GHE_URL ="https://your-ghe.example.com"

6. Try the cloud?

I tried to set up codecov instance and make it usable in a week, but I'm not confident to say that this is in perfect condition, maintaining a service could be tiring, codecov contains 4 components and 4 dependencies, they all require basic monitoring, or backup if they are database. And once it went wrong, all the thing you can do is to check the code, or preying that there's someone nice and know this well in the community.

I'm not an employee neither sentry nor codecov, but I do believe for some/most of the people, cloud service would be cheaper and easier.

koltyakov commented 1 year ago

Regarding the SSL part. When trying with SSL, I managed to get auth (GitHub) work but could not get it working without. While app settings are identical in both cases. So I'm also tending to believe that SSL is a must-have to make auth work.

LeoQuote commented 1 year ago

https://github.com/douban/charts/tree/master/charts/codecov Just released the first version of helm chart for codecov, you should try it if you're using k8s. PRs are welcomed!

rohan-at-sentry commented 10 months ago

@LeoQuote - this is great. thanks for taking the time to provide this feedback.

I'll review this and make changes to our docs to better call out some of the points you've raised.

codecov / self-hosted