Improve Postgres documentation

[ngosang]

Postgres documentation is hard to find => https://github.com/boramalper/magnetico/tree/master/pkg

• Could you add some info or links in the readme?
• Is it possible to create a docker-compose-postgres.yml file with Magnetico + Postgres (with required config)?
• Is there a tool/script/documentation to migrate from SQLite to Postgres?
• Write some recommendations about use cases when Postgres will improve performance

Ping @boramalper @skobkin

[skobkin]

Could you add some info or links in the readme?

I'll try to make those changes if I'd not forget.

Is it possible to create a docker-compose-postgres.yml file with Magnetico + Postgres (with required config)?

You don't need separate docker-compose.yml files to be able to switch database back-end. I'll check if we need co change anything to be able to choose back-end using environment variables for example.

Is there a tool/script/documentation to migrate from SQLite to Postgres?

Yes, I've implemented simple tool which uses Magnetico's own database back-ends to add torrents from SQLite to the PostgreSQL or other supported database. But consider that this implementation will lose torrent add date because it uses AddTorrent() method from the database interface.

https://gitlab.com/skobkin/magnetico-go-migrator

Write some recommendations about use cases when Postgres will improve performance

Almost any cases when you plan to index a lot of torrents, use multiple crawlers or multiple database clients (for example public search service).

[ngosang]

You don't need separate docker-compose.yml files to be able to switch database back-end. I'll check if we need co change anything to be able to choose back-end using environment variables for example.

I mean a specific docker-compose with magneticow + magneticod + postgres all of them preconfigured. The idea is running postgres in Docker too. You can find an example docker-compose for postgres in https://hub.docker.com/_/postgres

But consider that this implementation will lose torrent add date because it uses AddTorrent() method from the database interface.

Something is better than nothing. At least one user is interested => https://github.com/boramalper/magnetico/issues/218#issuecomment-734961622

[skobkin]

I mean a specific docker-compose with magneticow + magneticod + postgres all of them preconfigured. The idea is running postgres in Docker too. You can find an example docker-compose for postgres in https://hub.docker.com/_/postgres

It's possible... But not sure if needed. It still could be implemented in the same docker-compose.yml. Maybe if I (or Bora) have a time.

Something is better than nothing

Sorry, I forgot to paste the link. Previous comment is fixed now.

[sigiltenebrae]

I've been tearing my hair out a bit trying to get postgres to work, maybe this is a total noob question but I can't figure out where to implement "magneticod --database=postgres://username:password@127.0.0.1:5432/database?schema=custom_schema_name&sslmode=disable" I assume these are not enviroment variables for the docker container, but where do I pass it in?

[skobkin]

@sigiltenebrae

I can't figure out where to implement

These are command line arguments.

I assume these are not enviroment variables for the docker container, but where do I pass it in?

Correct. But with proper docker-compose.yml you can also use environment variables. Check this for example.

[sigiltenebrae]

awesome, working perfectly.

[sigiltenebrae]

on a related note, I've set up and installed your Postgres magnetico-web, but I can't appear to connect over LAN. It definitely has a db connection and created the user table. Are there special settings required to enable this? Sorry if this is the wrong place to ask.

[skobkin]

@sigiltenebrae

awesome, working perfectly

Glad to hear!

I've set up and installed your Postgres magnetico-web

You need to think if you really need it. Magnetico has built-in web interface which should work just fine (only with SQLite database for now). If you're not trying to run search instance on some public server and just running it inside your local network then most likely you don't need neither PostgreSQL nor magnetico-web.

but I can't appear to connect over LAN

Firstly you should understand that it's not intended for local usage. So most likely it should be on the same server as magneticod daemon and PostgreSQL server. I don't mean that it will not work over the network, just that it's not a usual case which can indicate that you're doing something strange.

Secondly, I can't help you if you don't provide installation details, configuration and everything that you did to set it up.

If you're concerned about the config data, you can send config to me using my email from the profile. But there shouldn't be anything too important.

And finally:

I can't appear to connect over LAN

What makes you think that? If you read something from the logs, provide the log too or describe what's wrong from your point of view. I can also recommend to check out points 2-6 from here.

Sorry if this is the wrong place to ask

You're right. I'd recommend you to move this conversation to #170 or to the issue in my repository. If you're going to proceed with some information to help you, please do it in one of places I mentioned above.