Implement MusicBrainz-like replication

acoustid / acoustid-server

AcoustID's web site and API

MIT License

64 stars 20 forks source link

Implement MusicBrainz-like replication #9

Closed lalinsky closed 10 years ago

lalinsky commented 13 years ago

We should use dbmirror to produce replication packets in the same format as MusicBrainz. The mbslave project can be then generalized to be able to import them into the Acoustid database.

dceballos commented 12 years ago

Any update to this? Is there currently any way to replicate? :)

lalinsky commented 12 years ago

The server exports replication packets, but there is nothing to apply them on a slave database. If you are interested in writing that script, please let me know. :)

dceballos commented 12 years ago

Sure, I'll look into it when I get some free time. Are the packets still generated using dbmirror?

lalinsky commented 12 years ago

The replication uses a little different format from what MB uses, but the workflow is the same.

You basically have to do:

Read replication_control from the database
N = current_replication_sequence + 1
Fetch http://acoustid.org/data/replication/acoustid-update-$N.xml.bz2
Parse the XML and for each "event" run either insert, update or delete on the database in one large transaction.

I've implemented it for MusicBrainz some time ago, so you could use this for some ideas: https://github.com/lalinsky/musicbrainz-server/blob/acoustid/admin/FetchAcoustidUpdates

dceballos commented 12 years ago

Great, thanks so much Lukas. I'll give it a shot soon.

adamansky commented 12 years ago

Please test the #23 pull request, it can solve this problem