segler-alex / radiobrowser-api-rust

radio-browser API implementation in rust
GNU Affero General Public License v3.0
230 stars 96 forks source link

Server sync problem #129

Closed egorikftp closed 2 years ago

egorikftp commented 2 years ago

Hey, I'm developing a native Android application based on API and faced some unexpected crashes. I made a research and found huge resynchronization across servers.

In the app, I randomly choose the server according to documentation. For example, fr1.api.radio-browser.info Then if the user plays and likes some station he presses a favorite button and I store in the database stationuuid (UUID). I expect, that this ID will be unique.

Then, the user opens the app again and the I choose server again. In this case de1.api.radio-browser.info Now, to load actual info about this radio station I will execute

http://de1.api.radio-browser.info/json/stations/byuuid?uuids=10f7b21c-74ed-4527-a59f-958f4d2a62c2

And the response will be empty image

The same call for the previous fr1 server: http://fr1.api.radio-browser.info/json/stations/byuuid?uuids=10f7b21c-74ed-4527-a59f-958f4d2a62c2

image

Also, to confirm the existing problem you can check the statistics page: https://stats.api.radio-browser.info/d/BTQ7l8sZk/radiobrowser?orgId=1&refresh=5m

FR server has more stations them others image

Could you please take a look? Thanks a lot for a great radio database 🙂

segler-alex commented 2 years ago

Hi, thank you very much for your interest in this problem! I know about this for some time, the stats.api.radio-browser.info i created for exactly this reason to make errors more easily visible, also for myself. This is a problem I am fighting for some time now, some months ago, i think the start of this yes i thought i found a solution and results really did finally get almost identical on all 3 of my servers. BUT: some time ago i also tried to fix another problem i have: a lot of streams that have the same streaming url, some of them were in the database more than 10 times. so i added another feature that allows for each server to delete the stream if it is a clone. then problems with sync started rising again. you might ask why: the main reason is, that i want to have servers that stand on their own, which means,

BUT: of course this facts make it harder to implement. how does a server know which stream to delete? what if another server deletes the other stream of 2 .... so i tried to add some rules to it that should apply on all servers equally so they get to the same conclusion on which one to delete. this then got me into more problems. i do not want to get into more details, i also forgot some details maybe already, as said, only my hobby project.

But you are perfectly right, this is a bad problem and I still do not have a good solution, that will not break something else :( :( :( if you are interested please look at the code, i think i have split it in a good way. of course, it could always be more structured. the "pull" directory is the important one. and the cleanup thing that deletes duplicates. and both together break each other basically. a quickfix would be to disable duplicate deletion for now, this would get me back to the state before, and the graphs will get almost equal again over 24 hours period after the change.

sorry to not be more helpfull for now. i will try to find a better solution, and it is getting cold outside pretty fast here in vienna, so i will have more time in the future again to do more programming. i will accept pull requests if you have any ideas. i also like to discuss ideas on how to do sync better if you have experience in this.the bad part is, currently i am the only programmer of the backend part of radio-browser.info

i hope i could at least give you some insights about the current state of things.

segler-alex commented 2 years ago

i deactivated duplicate removal algorithm for now. please check again now. i will try to find a better way.