boramalper / magnetico

Autonomous (self-hosted) BitTorrent DHT search engine suite.
http://labs.boramalper.org/magnetico/
GNU Affero General Public License v3.0
3.06k stars 344 forks source link

magneticod-go can't migrate magneticod-python's database #195

Open skobkin opened 5 years ago

skobkin commented 5 years ago

When I'm trying to use magneticod (Golang version) with database from Python version I'm getting following error every time:

~/tmp/magnetico# file database.sqlite3
database.sqlite3: SQLite 3.x database, last written using SQLite version 3022000

~/tmp/magnetico# du -sh database.sqlite3
20G     database.sqlite3

~/tmp/magnetico# docker run --name=magneticod --rm -v /root/tmp/magnetico:/root/.local/share/magneticod boramalper/magnetico:d_v0.7.1 -v --trawler-ml-addr=0.0.0.0:64879
2019-04-10T10:52:04.031Z        INFO    magneticod v0.7.0-beta2 has been started.
2019-04-10T10:52:04.031Z        INFO    Copyright (C) 2018  Mert Bora ALPER <bora@boramalper.org>.
2019-04-10T10:52:04.031Z        INFO    Dedicated to Cemile Binay, in whose hands I thrived.
2019-04-10T10:52:04.031Z        INFO    Compiled on 2018-12-30T16:45:44Z
2019-04-10T10:52:04.032Z        WARN    Updating database schema from 0 to 1... (this might take a while)
2019-04-10T10:52:04.032Z        FATAL   Could not open the database at `sqlite3:///root/.local/share/magneticod/database.sqlite3?_journal_mode=WAL&_busy_timeout=3000&_foreign_keys=true`%!(EXTRA zapcore.Field={error 25 0  setupDatabase: sql.Tx.Exec (v0 -> v1): database table is locked})

It works all right if there is no database from previous version.

jangrewe commented 5 years ago

I'm pretty sure the Go rewrite doesn't know anything about the schema of the Python version, the migrations it makes are only from older Go versions.

skobkin commented 5 years ago

It makes no sense because many people collected a lot of data using Python version.

If not in magneticod itself I think migration from the old Python version must be implemented at least in small separate tool to save that data.

jangrewe commented 5 years ago

If you think it must be implemented, feel free to go ahead and do it. In the meantime everybody else just moved on...

skobkin commented 5 years ago

@jangrewe

If you think it must be implemented, feel free to go ahead and do it.

If I think about anything I can say about it too. Nothing more.

Probably I'll make migration tool for my database. But it'll likely not be a part of magneticod.

In the meantime everybody else just moved on...

I like how you're speaking for all of the users. Sure it's a good habit.

gnouts commented 5 years ago

I don't follow closely the project and I was surprised of this breaking change... I mean, what's the point of having an offline torrent database if you have to start from scratch every two years ^^' Nevermind, I still like this project, I'll start from scratch again, and I hope you'll handle migration when changes happen in the future :)