segler-alex / radiobrowser-api-rust

radio-browser API implementation in rust
GNU Affero General Public License v3.0
227 stars 97 forks source link

Allow station edits somehow again. #49

Closed Wikinaut closed 10 months ago

Wikinaut commented 4 years ago

Some stations need edits, for example, the icons and title of "Kulturradio". How can I update that? http://www.radio-browser.info/gui/#!/history/960c1c0e-0601-11e8-ae97-52543be04c81

segler-alex commented 3 years ago

@conradfr i deployed a new version 0.7.13 today which does try to detect duplicates automatically. a few 1000 have been deleted automatically already. This check is now a default and i will try to improve this logic to detect even more duplicates.

keinstein commented 3 years ago

Hi, As far as I understand, the main problem is trust. You want to keep control over the content for quality reasons. But on the other hand this control lowers the quality. There are different approaches to delegate the responsibility. We have a hierarchical system for the Linux Kernel with several descrete levels. Wikipedia is similar. Here you have at least 3 levels: anonymous, normal users, and moderators. Stack Exchange uses a reputation system that provides a continuous spectrum of levels. Some more or less arbitrary hurdles have been set up to give access to people to different editing features. When you behave too badly, your account is disabled and you loose all your reputation and power.

A reputation system could be something like that:

Everyone can view the data. Logged in users can suggest changes. For every accepted suggestion you get 5 points. When a suggestion gets rejected you lose 1 point. When you got 100 points you are allowed to vote for others suggetions. After 5 votes a suggestion is accepted and the old state is logged (diff feature for verification). Reverting a change is also a suggestion. The winning voters get 5 points and the losing ones lose 1 point, each. With 5000 points you are allowed to accept changes from other users directly. With 10000 you are allowed to reject changes from other users directly. In both cases the voters and suggesters get points as described above. With 100000 points your are allowed accept your own changes and can apply for the status of an administrator. Administrators handle the things that must be done immediately (remove illegal content if the get informed, block bad users etc.). The site owner keeps control over administrators by accepting applications (usually) and blocking their accounts when the abuse their power.

The implementation could start with a login process, then with hand-picked administrators, before opening registration to the public. Then the reputation system without achievements. Each achievement could be implemented as the user base matures and gets educated. Until the achievement is in place the administrators have to fulfil that role.

One big advantage of this system is: when it does not work, you can adjust the numbers without the need to code anything. So fine tuning is easier than in a hard coded system. And it is extensible.

getcom commented 3 years ago

@segler-alex How ist this duplicate check working? For example I created the entry for "Cavo Paradiso" initially, which were using the SHOUTcast stream URL http://s5.onweb.gr:8488 in the past. Now this URL has changed to https://neos.win:48488/1. The previous stream URL is still active but is now used by "NRG. NEXT RADIO GENERATON - SMOOTH". So the current stream URL in the database is outdated.

What is happening if I would create a new entry? Is it a duplicate entry or would be a new name necessary? Which one will then be deleted if the name is identically? The old one or the new one?

It is not really possible to automatically detect if a working stream is the correct one if you do not analyze the metadata of the stream. A possible solution for is https://github.com/JamesHeinrich/getID3. In this case the stream name would be different to the database name. If detected this entry should the be marked/tagged as outdated/wrong and should not longer be published. It would be possible to update such a database entry if you analyze the website live streaming URL if a website is existing. This should be possible for the most internet streaming services/radio stations. The most stations could then be automatically updated. You have to parse the site links and check if this are streams or not. If yes: From the metadata entries it would the also be possible, if the sender has also new genres/substreams which then automatically could be added to the database. If done in this way, the "destroyers" have no chance.

The underground stations without a site URL has to be done manually. This entries are then marked as "unchecked". This small piece can only be done by the community which are using your database. This could be the developers or also the consumers or at least the initial contributor. Just my five cents....

The intention for me is that Im using some Marantz devices which have vTuner implemented which is a charged service in the meanwhile. As replacement Im using Ycast, with is using your database, running on a raspberry and some DNS overrides to get my favorite internet radio stations available on this devices. Hopefully there will be a solution for the update process in the future.

Cheers Ralf

jooola commented 2 years ago

Hello there, didn't read the whole thread carefully so I might suggest something that has already been proposed.

A good way of allowing modification could be to check for the domain name ownership. Using a TXT record with some kind of session based key, pretty much like lets encrypt and its DNS verification works.

Maybe this gives some idea to move this ticket forward.

phranck commented 2 years ago

This thread is so old with so many requests and support offers. I don't understand the maintainer in this case. I think I will not pursue this project any further and possibly set up something of my own.

silvernode commented 2 years ago

I think things should be kept simple. When you create a station, there could be an Email entry text field which is used to send a randomly generated key. This key can then be entered into a text field that is presented to the user when the edit button is clicked.

Bulwagga commented 2 years ago

Nice. -------- Original message --------From: silvernode @.> Date: 11/7/21 4:02 PM (GMT-05:00) To: segler-alex/radiobrowser-api-rust @.> Cc: Bulwagga @.>, Mention @.> Subject: Re: [segler-alex/radiobrowser-api-rust] Allow station edits somehow again. (#49) I think things should be kept simple. When you create a station, there could be an Email entry text field which is used to send a randomly generated key. This key can then be entered into a text field that is presented to the user when the edit button is clicked.

—You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or unsubscribe.Triage notifications on the go with GitHub Mobile for iOS or Android.

ghost commented 2 years ago

c417b50 does this mean we will soon be able to edit stations? 👀

Spielmops commented 2 years ago

I offered a long time ago, to keep the german public stations up_to_date, Alex has only to trust me and give me the right, to edit the datas. That's a very simple solution. And for trusting me I can give him my website-address spielmops.org. I think, there are more users like me and it does not need some thousand users to keep the whole database uptodate, a dozen will do for the beginning ...

Spielmops

bentolor commented 2 years ago

I created this station. It seems now, that the provider deliberately changed its URLs but keeps the old one active delivering a russian message that you should visit the stations original website.

I understand the difficulties to defend against malicious contributors, but I don't think that creating the new stream-meta approach is sufficien, as it only allows the provider to edit entries.

What's the proposed solution in this example case? Creating new duplicate entries?

PanderMusubi commented 2 years ago

See also https://github.com/tunerapp/stations/issues/3

JazzfanRS commented 2 years ago

At the now disbanded RadioSure website access was granted to registered users to propose changes to the database. From there it waited for approval by another registered user (typically one with longer standing within the hierarchy of volunteers). With one user with absolute authority to remove access to editor privileges of anyone. And I suspect didn't have access to edit stations themselves but just visually tracked changes made.

That led a few users to radio-browser and a way to convert the database to one used by RadioSure. No station paid to be on the database. The app was not a streaming service users paid for.

The vandalism I recently witnessed in a recent backup for radio-browser included anti(politicial name) in the streaming URL and the same directed at specific streaming services with no message. But that was maybe 10 out of 31000. Why keep a listing for an entire week before it is dropped? A streaming service shouldn't need 6 days to find out their stations aren't connecting.

bombelman commented 2 years ago
  1. Many apps use this database, which has unreliable information that cannot be edited.
  2. You are making an issue of something that has been solved many years ago already.
magic-ian commented 2 years ago

Why not make database edits available via Github/Gitlab pull requests only, and designate a set bunch of trusted users to merge them in. Perhaps a front-end can be written to feed in edit requests from a webpage front-end to a git-bot for non-programmers too.

For example, like @Spielmops, I just finished getting the streams & metadata of all radio stations in Canada, but there are some already there that don't have all the metadata that I would love to be able to edit/replace before I commit adding all the new ones to the database. I wouldn't want to be the one maintaining them, but I'd do an initial data population.

TommyRIB commented 2 years ago

Not all of us are into sabotaging peoples accounts. I created an account and all that I want to do is to edit my tags and fix my listen to link and delete a duplicate account.. People that create accounts must be allowed to make changes. You must let people make changes.

dreisafer commented 2 years ago

For 1 year I try only to add a logo to the transmitter, answer you get none if you write directly to the mail to segler and as I see I can still edit nothing myself, only the stupid logo must change, that is new ( 1 year). The old is also no longer displayed, so no logo at all, a broken link picture, looks beautiful 😒. Why complicated when it is also easy to protect against vandalism

goliv04053 commented 2 years ago

I have begun to use RadioBrowser and I think that with a collective moderation system and a version method (like Wikipedia), it should resolve the problem. Also, why not implement a login system? It should resolve or help resolve vandalism.

STPKITT commented 2 years ago

Judging by all the ideas which were posted there are plenty of ways in which this problem could be solved, even uncomplicated ones. It's about time to apply such a solution since 2 1/2 years is a long time considering the average lifespan of a radio streaming link.

GothamGlobe commented 2 years ago

1 version 0.84 will import stations with random order, you'll have to unistall and reinstall it again each time to restore fav orders. 2 lastfm api won't pop up cover art 40% of the times, can you add spotify api support insetead in next update 3 now playing in lower end, wont display cover art thumbnail, only radio station logo, can you fix that as well 4 many radio stations have wrong icons or names. can you add edit mode at least for certain users if you have concerns about vandalism tx

goliv04053 commented 1 year ago

I don't know but I believe this can help in some way. (https://blog.ajay.app/voting-and-pseudo-randomness-or-sponsorblock-or-youtube-sponsorship-segment-blocker) This is an algorithm implemented by SponsorBlock to prevent spam on crowd-sourcing. Give a look

andersan commented 1 year ago

has this been solved?

if not, is there an active fork of this project where someone is working on this solution?

numeropi commented 1 year ago

I think the best solution is the proposed by https://github.com/segler-alex/radiobrowser-api-rust/issues/49#issuecomment-1156854138 and I think most people liked it.

For editing/deleting spam or duplicate stations... a git issue tracker (ex. here) and an small team of trusted Mergers and you won't have any extra work load. I found the database great but then this issue stopped me from willing to collaborate...

jcorporation commented 1 year ago

I created with WebradioDB a database that works only with issues and GitHub workflows to create a curated database. Everyone is welcome to contribute.

diyfr commented 1 year ago

https://www.radio-browser.info/history/960bf492-0601-11e8-ae97-52543be04c81

Change http://185.52.127.132/fr/30601/mp3_128.mp3?origine=fluxradios by https://scdn.nrjaudio.fm/fr/30601/mp3_128.mp3?origine=playernostalgie&cdn_path=audio_lbs10

Knusper commented 1 year ago

Are you aware many duplicate entries in the database? I came here, because this was not touched in the FAQ.... At least dupes should be easy to fix without human intervention?

rgctoronto commented 1 year ago

The Geomaps feature is a nice one. It brings the "Radio Garden" idea to the radio-browser site.

However, there is "defacto" (intentional or unintentional) vandalism on the map with stations "pinned" on to various "blank" spots on the map i.e. in the middle of the ocean, or on parts of the world with large land masses.

It might help if there was a check box or something where people could check off "location of stream unknown" instead of just pinning it "anywhere" on the map. That would at least help with the "unintentional" vandalism.

I don't think there is any "perfect" solution to the editing problem, but there are some that have been outlined over several years that are much better than the "no edits" at all. I think it would be a good idea to start implementing some of them and working out the "bugs" as they come about.

I'm just coming back to the site after a number of years absence and adding/updating radio streams that probably broke a long time ago. I have my own very large lists that I've maintained on a local computer in "RadioTray-NG". So checking and adding a little bit at a time.

magic-ian commented 1 year ago

I recommend contributing to jcorporation's fork, and opening issues for any project that rely on this unmaintained database to switch to the new one that is. That is where I'll be contributing my addition of stations moving forwards.

rgctoronto commented 1 year ago

I recommend contributing to jcorporation's fork, and opening issues for any project that rely on this unmaintained database to switch to the new one that is. That is where I'll be contributing my addition of stations moving forwards.

That's something I'll take a look at.

It's sad because Alex has done some amazing excellent work that is appreciated by so many people. One batch of bad experiences has unfortunately made him extremely cautious about moving forward on this with a different approach. It's obvious that there are plenty of people of goodwill with ideas who would like to find ways to help, but it appears that the bad experience is getting in the way.

rgctoronto commented 1 year ago

@magic-ian I had a look at the other database. Sadly there are only a little over 400 entries there vs. nearly 40,000 on radio-browser.

Building another database from nothing is a huge undertaking. If it were possible to fork this database into another one that was editable (with sufficient moderation/editing controls of some kind), then one wouldn't be building from scratch. Then, if the new database was of better quality than the old one, various software projects would pick it up and use it instead.

bombelman commented 1 year ago

this will go on forever without a solution. I'm willing to set up a new database, which can grow to at least 50K, taking most or all good suggestions mentioned here. What's stopping anyone really ?

numeropi commented 1 year ago

I belive what makes a big difference is the usability of radiobrowser, it is a opensource app that it is easily installed in Android and can search radio stations, etc...

@jcorporation if your webradiodb could easily have an app to just download and install and be able to pin radiostations from your database(that is easier to send you new feedback about stations...) then it will probably success.

jcorporation commented 1 year ago

My webradiodb project is designed to be used with myMPD, but this is an other topic. We can discuss about webradiodb in an issue in the webradiodb repository.

terr72 commented 1 year ago

Alex' Github projects seem to be orphaned since the beginning of 2023. An e-mail I wrote to him a few weeks ago regarding a software with a pretty large userbase (foobar2000) that recently added his API and could be added to the supported apps list also remained unanswered.

I don't get it. Why not implement the suggestion that magic-ian posted over a year ago and got a massive thumbs-up? A few commited people that maintain the database through Github issues/requests. That's how almost all large ad blocking lists handle it and their evaluation and filter creation process requires a bit more brainpower. In the radio browser list it's basically only about "is the logo missing or better/more current than the previous one", "OK, the URL changed" or "yeah, that's a typo".

terr72 commented 1 year ago

Are you aware many duplicate entries in the database? I came here, because this was not touched in the FAQ.... At least dupes should be easy to fix without human intervention?

That's actually a massive side effect of the current "we do not touch it!" policy: people constantly submit new entries for existing stations, because they lack logos, have outdated URLs or typos. Because that's basically the only way to have a correct entry in their players since years.

I don't know - this project started out so good and then Alex seem to have developed trust issues when some idiots vandalized entries. But that's the nature of collaborative open source efforts: either raise the levels to participate for everyone (like Wikipedia did, massive effort) or find yourself a few trustworthy people as maintainers (easier and a lot of people showed interest in this thread).

The idea to get station owners to maintain their entries is perfectly fine in theory, but fails in reality: I'd say that about 98% of stream owners don't even know about this list and if they knew, probably 5% would care about it or have capable personnel to do so.

Vrihub commented 1 year ago

Alex' Github projects seem to be orphaned since the beginning of 2023.

Fine, then we should just set up a new community-managed station database following the suggestions in this thread, then fork radiodroid and make the fork use the new database.

terr72 commented 1 year ago

Fine, then we should just set up a new community-managed station database following the suggestions in this thread, then fork radiodroid and make the fork use the new database.

You won't get the apps to switch their hardcoded API server entries very fast. It's not just about RadioDroid. They typically don't change horses unless the current server shuts down or a fork turns out to be a superior winner with advantages, which can take a long time even afterwards. There's currently no need for them and most users - it's just the power users striving for a better database quality.

It would be nice if @segler-alex could say some words here: if he has basically given up on the projects or if he's only short on time and plans to resume and resolve things.

conradfr commented 1 year ago

One additional solution could be to generate a link with a unique token to allow the person who submitted a new radio to edit it later.

JensKorte commented 1 year ago

Fork it with another name, ask F-Droid to include the fork. That should be normal way imho.

terr72 commented 1 year ago

One additional solution could be to generate a link with a unique token to allow the person who submitted a new radio to edit it later.

How does that help with the existings 40.000 entries? And how do you ensure someone who submitted an entry will keep it updated? The problem isn't a lack of entries, it's existing entries that are incomplete, outdated or duplicates.

Fork it with another name, ask F-Droid to include the fork. That should be normal way imho.

Erm, RadioDroid and the radio-browser API/database are two separate projects. Have you recently checked what other software is using the database and the API? RadioDroid is only a very small portion of the user base.

JensKorte commented 1 year ago

Fork it with another name, ask F-Droid to include the fork. That should be normal way imho.

Erm, RadioDroid and the radio-browser API/database are two separate projects. Have you recently checked what other software is using the database and the API? RadioDroid is only a very small portion of the user base.

I didn't think of that.

rgctoronto commented 1 year ago

Fine, then we should just set up a new community-managed station database following the suggestions in this thread, then fork radiodroid and make the fork use the new database.

You won't get the apps to switch their hardcoded API server entries very fast. It's not just about RadioDroid. They typically don't change horses unless the current server shuts down or a fork turns out to be a superior winner with advantages, which can take a long time even afterwards. There's currently no need for them and most users - it's just the power users striving for a better database quality.

It would be nice if @segler-alex could say some words here: if he has basically given up on the projects or if he's only short on time and plans to resume and resolve things.

The task at hand is to fork the database and build a better one. If the database is better then the many apps that use the current database will gradually, slowly, switch over to it.

Even with its many faults, the existing database is the best there is and the many apps that use it have no reason to change.

terr72 commented 1 year ago

The task at hand is to fork the database and build a better one. If the database is better then the many apps that use the current database will gradually, slowly, switch over to it.

Even with its many faults, the existing database is the best there is and the many apps that use it have no reason to change.

That's what I was trying to point out. It's not just about forking the database. You need to replicate the API server infrastructure capable of handling the loads and with DDoS protection (4MB+ JSON responses for a single search are not uncommon). Currently this is sponsored - this ain't cheap. Then you need to find people commited to manually maintain the entries for a long period with no guarantee that the current apps will ever make the switch to the alternative DB.

terr72 commented 1 year ago

Has anyone reading here had contact with Alex via e-mail this year? I've also tried his second e-mail address with no response.

Chartwalker commented 1 year ago

no way at the moment. sorry. to get at this problem, we have to find solutions for multiple subproblems first.

The basic idea would be to introduce a karma level of 100.

This means that those who make reasonably correct entries will receive 10 karma points for each valuable contribution.

Those who obviously and maliciously make wrong entries or level up users although they intentionally make wrong entries will have their previous karma deleted and -1000 karma points per process, so they have to produce 110 correct new entries to be back.

Small mistakes should not be punished immediately. Those who make new entries but need significant additions will receive a karma point plus the karma points of other helpful users who contribute corrections and get a karma point themselves for doing so.

If the Karme level is 100 then you will be unlocked, otherwise you need other advocates who will level you up by 1+ or level you down by -1 for small mistakes.

rgctoronto commented 1 year ago

The basic idea would be to introduce a karma level of 100.

There have been a number of worthwhile proposals and ideas put forward for editing/moderation of editing.

But, we have the problem of the current owner of this project being unresponsive for a very long time. Until the project owner becomes responsive or else those with the skill set and resources to fork the database project come forward, we are stuck at an impasse, with a database of multiple entries of varying quality.

terr72 commented 1 year ago

But, we have the problem of the current owner of this project being unresponsive for a very long time.

It's unclear if it's "simple" ghosting for unknown reasons (none of our business) or if something seriously has happened. I mean you typically don't abandon two successful projects and don't reply to any e-mails - that's why I was asking if someone had contact by e-mail since the last Github activity back in Dec. 2022.

segler-alex commented 1 year ago

Hello, I want to add my 50cent to this discussion, because I was asked from multiple people. I will make it short: IMHO:

to wrap this up: thank you for reading my thoughts, on the theme. I think both paths are valid, but time is limited. This is my hobby, and it will always stay that way. It will always stay free. I want to do what I believe is the right path. Also internet indexes ended at some point and searching machines took over. in my dream radio-browser will become as automatic as possible and will try to get to even lower maintenance levels needed. AND of course as stable as possible. I hope some of you like this path and do want to support it. Thank you all for your support.

Ok that was not short :)

diyfr commented 1 year ago

Idea for radiodroid : make radiobrowser responsive design and set it as Progressive Web App.

terr72 commented 1 year ago

@segler-alex, I'm fine with everything as long as the database quality improves.

What you describe is more of a classic crawler, that's been around since 20+ years, except for the "listening" part. Throwing out outdated entries, finding logos by looking for the closest image to the embedded player or the embedded meta information about it in the web page is basic crawler work.

Instead of making an AI listen to the actual stream (very resource intensive and like cracking a nut with a sledgehammer) you could only connect for the shortest possible time and extract the artist & title tags every x minutes over a period of y hours. Then you could query the genre(s) of those and have matching genre tags. That's WAY more effective than to train an AI to determine the genre by listening - but if the integration of AI helps you to make it more "sexy" for you to get back on the project, do so.

keinstein commented 1 year ago

@segler-alex and all others Many good ideas have been proposed, here. And I think the goals can be achieved only by different projects. Users have different needs. I for myself am interested in local radio stations from my city and those which have a Germany-wide program. I don't need hundreds of stations that I can't hear simultaneously. So a good hand-picked list would help me. And I would like to mark the list as favourite, so that a new radio station in my city appears as soon as it is added to the main list. This way I'm also motivated in keeping the list right. There are some corner cases where my opinion of „right” clashes with the opinions of others. This is why different community models should be tried out.

There are many factors that can make a community model successful or fail. One important factor is the community itself. Different people need different models. A model that works with one project can fail with a competing project just by chance or because other people are involved.

The different projects may even use the same software.

At least they should agree on a common API that the end user equipment can use. So the Apps/Firmware developers can easily different directories just by adding another url or allowing the end users to add their own URL. Here, the distinction between must, should and can is very helpful, as people tend to implement things differently. Also proprietary extensions are very helpful to implement features that could be added to the common standard as soon as they work.

Also a common interface for content providers is helpful. However, you must always keep in mind that providing metadata about their streams is not their main concern. Some people run successful radio stations on their own, but may horribly fail in choosing 3 descriptive keywords about their streams. Some stations are listened to just for some side effects like their irony or because of an erotic voice. When you produce a stream you cannot have all parameters in mind.

AI cannot solve this problem. The difference between AI and classical empirical methods is mainly the number of parameters. AI adds lots of just-in-case parameters that are used or not used depending on the training data. So AI has the same problems as all statistical methods including human experience: Data inside its domain may be classified sufficiently well, but it may fail horribly in cases outside of its domain. So an AI will probably perform well on mainstream stations, but it can't classify each corner case.