Postgres support

[tomlawesome]

⚠️ Please verify that this feature request has NOT been suggested before.

• [X] I checked and didn't find similar feature request

🏷️ Feature Request Type

Other

🔖 Feature description

Allow users to use an external Postgres database to store all configuration/info for Uptime Kuma

✔️ Solution

Add support for Postgred databases

❓ Alternatives

Add support for MySQL though personally, I think Postgres is better.

📝 Additional Context

I think currently, data is store with sqlite? I would rather store my data on my existing database server because I have backup routines, etc, which will protect the data. I like to keep the dat a spearate from the app so I can just reinstall the app, point it at the correct database

[deefdragon]

going to cross reference #894. I don't know if its Louis saying he wont support other databases, or just that they arent supported RN tho.

also gonna cross reference #953

[trogper]

I have made branch with mysql support in my fork. See my comment in #953. Adding postgres support should be just a few lines of code in knexfile.js. Please read the comment and try it out if you can

[ondrejsika]

Postgres support will be great

[trogper]

Postgres should be possible in my fork, you can try it out. Just edit knexfile.js

[gc-ss]

Postgres should be possible in my fork, you can try it out. Just edit knexfile.js

There are people who would love to use uptime-kuma if it had Postgres support. It's great to hear your fork supports Postgres - can you/someone explain to me why your fork can't be merged back into this uptime-kuma repo?

[trogper]

@gc-ss It's still work in progress, or more of a proof of concept. There are a few things to be decided upon and I dont think I am the one who should do it. Read my comment, where I explaned it a bit or my PR #1139

[gc-ss]

I dont think I am the one who should do it.

The usecase here is to allow multiple sources to write/update uptime-kuma concurrently.

I imagine the reason why a lot of interest in Postgres exists is to allow concurrent writes and people are ready to employ Postgres instead of trying to scale SQLite.

Is that why you too added Postgres support?

I believe there's a way where we can just continue to use the current SQLite integration and also allow concurrent writes. Here's how:

There is a copy of SQLite that has fantastic concurrent write support.

This is a great article by itself but feel free to jump to the last paragraph

https://blog.expensify.com/2018/01/08/scaling-sqlite-to-4m-qps-on-a-single-server/amp/

1. If I can chase this copy of SQLite with concurrent write support down
2. Will you be able to help test it with me and ensure it works with uptime Kuma without breaking existing uptime Kuma functionality?

Again - Thank You for your contributions and time!

[trogper]

If by concurrent writes you mean running multiple instances of kuma on a single database, then I think that will break a lot of things in kuma. I added primarily MySQL support because I wanted to deploy kuma to Google Cloud Run (docker on google cloud), which does not support writing to local files. Additionally I prefer having data stored in a traditional database, as having it in a SQLite file insde a continer seems like a black box to me. I have not finished/worked on it as I am busy with school and work currently.

I am afraid kuma will not opt to use this copy (fork) of SQLite by default, as it seems to solve only one peculiar requirment, which might collide with another requirment of another user. I personally am wary of using smaller forks of software, as many times they lack support and maintenance of the original.

Why do you need concurrent writes to kuma database? What is your "business case"?

[AtjonTV]

Yesterday I started to consolidate my PostgreSQL Servers into a hosted Database Service to remove the burden of managing multiple server instances. I wanted to include the Kuma database too so that I have all data and backups in one place, but sadly Kuma currently only supports SQLite.

[Maffsie]

Any hope of this, or MySQL, being merged in any time soon? I'm running Uptime Kuma locally with a database size of approx. 700MB (around 50 separate checks so far), and I'm finding the performance worsens over time at about the same rate I experienced with Grafana before I moved that service to using a separate database. I'd really like to move Uptime Kuma to using my Postgres or MySQL/MariaDB servers, for better interactive performance.

[Freezepop]

Guys, please add support for PostgreSQL or other DBMS with the possibility of a normal HA architecture! In production, SQLite looks like the wrong solution.

[Maven35]

is it possible to add this, would love to have an HA solution for my Kubernetes uptime-kuma

[trogper]

@Freezepop @Maven35 Just having postgres support is not sufficient for it to be HA. It requires more changes than just database backend.

[Maven35]

@trogper you are correct, but getting off of SQLite is at least a step toward that. I'm aware that there would have to be other serious architectural changes to the application to support multiple instances of the application running concurrently.

[Milo123459]

How could I go about implementing this? I would like to have a go.

[trogper]

Edit (@CommanderStorm): Updated this comment to reflect the current release planning

@Milo123459 if you mean Postgres support, people have already done it, but it is not merged. While mariaDB-support is included in the 2.0 release, Postgres-support was pushed off for another major release due to the reasons listed here https://github.com/louislam/uptime-kuma/pull/3748#issuecomment-1871465660

You can subscribe to https://github.com/louislam/uptime-kuma/pull/3748 or to this issue to get notified once this is the case

[Milo123459]

Oh, my bad. Do you know when v2 will release?

[trogper]

No, I don't. Louislam is working on it, there's a branch and a pull request for tracking the progress

[Milo123459]

Thanks! Can you send the PR link over please?

[cypa]

wouldn't SQLite -> postgres replication be enough?

[lucyrose39]

I had been running Kuma for some time now and like it. Yesterday I started to consolidate my PostgreSQL Servers into a hosted Database Service to remove the burden of managing multiple server instances. I wanted to include the Kuma database too so that I have all data and backups in one place, but sadly Kuma only supports SQLite!? It would be awesome if Kuma could support PostgreSQL!

wouldn't SQLite -> postgres replication be enough?

sqlite is notoriously bad for performance and causes monitoring to fail with several monitoring instance

[brooksvb]

I think different database options are very much needed. With about 160 monitors on our Uptime Kuma instance, I believe SQLite is heavily bottlenecking performance for every single page load. We plan to split our monitors between a second host and the original, but I think we could squeeze a lot more out if we had support for a real database.

[CommanderStorm]

Not really a database issue, more a "how do we use the db"- issue. v2 will be able to support way more monitors even using SQLite aggregated storage and other improvements like https://github.com/louislam/uptime-kuma/pull/3515 Benchmarking of the newer limit is currently outstanding, as waiting for a few bugfixes. See https://github.com/louislam/uptime-kuma/pull/4500 for further context.

If you are running into problems with 160 monitors, something fishy is going on. Normally, users report problems after 500-800 monitors.

[brooksvb]

Thank you for the additional context.

If you are running into problems with 160 monitors, something fishy is going on. Normally users report probelms after 500-800 monitors.

We are running Uptime Kuma on a thin client, so it does have limited resources, but we didn't expect it to start getting choked up on page loads this quickly. I will be eager to test v2 and see the improvements.

[fabry006]

Any update about integrating any DBMS?

[CommanderStorm]

@fabry006 Please see my post from a month ago: https://github.com/louislam/uptime-kuma/issues/959#issuecomment-1894730411

[lucyrose39]

Just going to add onto this saying I've experienced issues with 50 monitors @ 5 sec intervals, after about a week the UI fails to load by timeout

[CommanderStorm]

@lucyrose39 You actually don't want the issue you are posting in

there is a good reason why the 20s minimum exists. Sending a ping of every 100ms fills the database rapidly. Please refer to our plans for the v2.0 release https://github.com/louislam/uptime-kuma/issues/4500 Meanwhile, reduce the retention, reduce the time between heartbeats, use faster storage or shard via using a different instance for some monitors.

As stated in said pinned issue: The v2.0 comes with better performance through external MariaDB, storing heartbeats in an aggregated form, full server-side pagination for important events => even though benchmarking is still out, we are confident that this pushes the prominent "limits" (highly hardware-dependent, not a limit imposed by us) ~500 Monitor or ~1.5GB DB-size

[Aj7Ay]

How can we attach PostgreSQL to Uptime with Compose file Any reference?

[CommanderStorm]

What do you mean by attach? Do you mean monitor? (if yes: just use the url to said instance) Do you mean as a database: see https://github.com/louislam/uptime-kuma/issues/959#issuecomment-1894730411

[Aj7Ay]

No can i use postgre sql to store data instead of SQL

[CommanderStorm]

PostgreSQL also uses SQL.

Assuming you mean SQLite3: As written in above comment, v2.0 will come with the ability to use maridb/mysql. Postgres is not on the table (see linked PR) for v2.0 as the addition to be able to choose another DB (See pinned issue) is enough support effort/work for the time being. => The needed testing and verification effort is postponed to a future major/minor release

[Aj7Ay]

Thanks