pupeno / repeater_world

Ham radio repeater website and database (open source, free data).
https://repeater.world
GNU Affero General Public License v3.0
22 stars 2 forks source link

API #600

Open ajorg opened 1 month ago

ajorg commented 1 month ago

I think having the service regularly import data from various sources is probably a good idea, but why not also support an API that coordinators can use to upload their data to you?

I wanted to do that for RepeaterBook, but they don't have an API and I'm not going to manually add data for even a few repeaters when I have dozens to add.

By the way, how do you plan to deal with duplicate data when records don't match perfectly?

pupeno commented 1 month ago

I'd like to learn more that use case. What would have been the program or software talking to that API? Where is that data now, in your case, that you wouldn't have had to type it if there was an API?

For now, I'm using the call sign and the transmit frequency as the unique identifiers of a repeater. If it's the same for both, I consider the repeater to be the same so as not to create duplicates.

ajorg commented 1 month ago

I'd be writing a script to compare the data in RepeaterBook with the data from WWARA. You can see the script here: https://github.com/ajorg/WWARA/blob/main/wwara-repeaterbook

I would have liked to have an API to use to submit changes, so either an add record API that knows how to avoid duplicates, or an update record API to send specific changes to. Search and read record APIs are also needed of course.

When trying to compare WWARA with RepeaterBook I found that sometimes either frequency could be wrong, probably because someone made a mistake manually entering it. I found that I also needed to check if the location was close, to get a better guess of what matched or didn't. Actual duplicates were also present, where two repeaters use the same callsign and the same frequencies, but are on opposite sides of a mountain, and usually with different input tones.

A really good primary key would be the coordinator + the coordinator's own ID, but crowdsourced data is almost always going to lack that, sadly.

Data can be wrong from the coordinators too. Another script I have is https://github.com/ajorg/WWARA/blob/main/wwara-qc which checks records against the band plans and for a variety of other errors and oddities. I've caught dozens of errors in the data (all corrected now).

pupeno commented 1 month ago

So WWARA is the source of truth here (plus the corrections you mentioned)?

In Repeater World, the approach to tackle this would be an importer from WWARA. They get run every week, so there's no need for someone to have to run them periodically to update the data. That's not to say there wouldn't be a write API at some point.

And I'm aware of the bad data. I think I wrote hundreds of lines of code to deal with it. If you open any file here you'll find all the little workarounds: https://github.com/pupeno/repeater_world/tree/main/lib/importers