search

segler-alex / radiobrowser-api-rust

radio-browser API implementation in rust
GNU Affero General Public License v3.0
227 stars 97 forks source link

Several hundred duplicates in database, and how to deal with them. #165

Closed JazzfanRS closed 1 year ago

JazzfanRS commented 2 years ago

Finding several hundred duplicate listings (based on exact copy of individual lines.) Why aren't they culled before creating the daily .zip?

Separate issue related to duplicate listing: Allow for multiple formats instead of just one per submission. Dozens of stations have Mp3 and AAC. Allow those to be placed in one listing (submission).

pvagner commented 2 years ago

I've just submitted a station which has already been added I have just searched wrong. Perhaps when adding stream url which is already in the database, it should refuse to add it again.

JazzfanRS commented 2 years ago

To my understanding there was script in place to remove duplicates, but it's been awhile since I looked that closely at the SQL. I just decided to remove them from the download with my own script along with other problems in their database. What format do you use the database in. I have it set up to download daily, delete everything but Name, Country, Language, and Genre, and source URL and format in a tab separated format (plaintext) for use by RadioSure PC app.