search

fluids-ai / weather_radio_locator

A web based app that uses Python to provide lists of AM, FM, Weather Band, and Shortwave radio stations.
Creative Commons Zero v1.0 Universal
0 stars 0 forks source link

Update etl.py for ETL from radio station sources #1

Open hammad93 opened 1 month ago

hammad93 commented 1 month ago

This can be closed when the Python file has the ability to return structured data from the unstructured data sources below,

https://www.weather.gov/nwr/station_search
https://radio-locator.com/
https://short-wave.info/index.php
hammad93 commented 1 month ago

https://github.com/fluids-ai/weather_radio_locator/tree/scrape This branch was created to track changes associated with the issue.

hammad93 commented 1 month ago

Currently, all NOAA stations according to the station_search are in this JSON file,

https://www.weather.gov/source/nwr/JS/xmitter.js

Using lat and lon, we can create a function that finds the nearest station ourselves

hammad93 commented 1 month ago

https://radio-locator.com/cgi-bin/locate?select=lonlat&geo=y&latd=40.5536768&lond=-74.30144

Replace lat and lon here

hammad93 commented 1 month ago

POST request to get all English stations, https://www.short-wave.info/index.php

station=Any Station language=English now=03 now2=43 GMT=true submit=GO mobile=false

hammad93 commented 1 month ago

Currently, all NOAA stations according to the station_search are in this JSON file,

https://www.weather.gov/source/nwr/JS/xmitter.js

Using lat and lon, we can create a function that finds the nearest station ourselves

This does not include the band information. It is available from further webscraping here, https://www.weather.gov/nwr/sites?site=KZZ31 by replacing the site variable with the index from xmitter.js. This link can be included in the metadata.

hammad93 commented 1 month ago

POST request to get all English stations, https://www.short-wave.info/index.php

station=Any Station language=English now=03 now2=43 GMT=true submit=GO mobile=false

Latitude and longitude are included in the data for each station. The default API does not search by location but we can use the same method that we use for the NOAA weather bands.