freifunk-gluon / census-exporter

Scraping communities for census data
https://stats.darmstadt.freifunk.net/d/CA5PRFmMz/gluon-census?orgId=1
MIT License
10 stars 11 forks source link

communities.json: update URLs 2023-03 #28

Closed herbetom closed 1 year ago

herbetom commented 1 year ago

I've deployed it. Seems to still work:

Mar 05 02:20:00 elsa systemd[1]: Started Gluon Census Textfile Exporter.
Mar 05 02:20:07 elsa census-exporter.py[1622627]: meshviewer        https://data.aachen.freifunk.net/meshviewer.json
Mar 05 02:20:07 elsa census-exporter.py[1622627]: meshviewer        https://map.tecff.de/yanic/meshviewer.json
Mar 05 02:20:07 elsa census-exporter.py[1622627]: nodes.json v2        https://karte.freifunk-ansbach.de/data/nodes.json
Mar 05 02:20:07 elsa census-exporter.py[1622627]: meshviewer (old)        https://map.freifunk-bielefeld.de/meshviewer/data/meshviewer.json
Mar 05 02:20:07 elsa census-exporter.py[1622627]: nodes.json v2        http://map.freifunk-bochum.de:4000/nodes.json
Mar 05 02:20:07 elsa census-exporter.py[1622627]: meshviewer        https://freifunk-bs.de/data/meshviewer.json
Mar 05 02:20:07 elsa census-exporter.py[1622627]: meshviewer        https://downloads.bremen.freifunk.net/data/meshviewer.json
Mar 05 02:20:07 elsa census-exporter.py[1622627]: nodes.json v2        https://descartes.chemnitz.freifunk.net/nodes.json
Mar 05 02:20:07 elsa census-exporter.py[1622627]: meshviewer        https://meshviewer.darmstadt.freifunk.net/data/meshviewer.json
Mar 05 02:20:07 elsa census-exporter.py[1622627]: nodes.json v1        http://karte.ffdon.de/data/nodes.json
Mar 05 02:20:07 elsa census-exporter.py[1622627]: 2023-03-05 02:20.07 Exception caught while fetching url ex=ReadTimeout(ReadTimeoutError("HTTPSConnectionPool(host='map-ng.ffdo.de', port=443): Read timed out. (read timeout=5)"))
Mar 05 02:20:07 elsa census-exporter.py[1622627]: meshviewer        https://map.freifunk-3laendereck.net/data/meshviewer.json
Mar 05 02:20:07 elsa census-exporter.py[1622627]: nodes.json v2        https://meshviewer.freifunk-dresden.de/data/nodes.json
Mar 05 02:20:07 elsa census-exporter.py[1622627]: meshviewer        https://map.freifunk-duesseldorf.de/data/meshviewer.json
Mar 05 02:20:07 elsa census-exporter.py[1622627]: nodes.json v2        https://map.eulenfunk.de/data/nodes.json
Mar 05 02:20:07 elsa census-exporter.py[1622627]: meshviewer        http://vps643489.ovh.net/meshviewer/data/meshviewer.json
Mar 05 02:20:07 elsa census-exporter.py[1622627]: nodes.json v2        https://karte.freifunk-emskirchen.de/data/nodes.json
Mar 05 02:20:07 elsa census-exporter.py[1622627]: meshviewer        https://map.erfurt.freifunk.net/meshviewer/data/meshviewer.json
Mar 05 02:20:08 elsa census-exporter.py[1622627]: meshviewer        https://map.freifunk-essen.de/data/meshviewer.json
Mar 05 02:20:08 elsa census-exporter.py[1622627]: meshviewer        https://karte.ff-en.de/data/meshviewer.json
Mar 05 02:20:08 elsa census-exporter.py[1622627]: meshviewer        https://yanic.batman15.ffffm.net/meshviewer.json
Mar 05 02:20:09 elsa census-exporter.py[1622627]: meshviewer        https://meshviewer.fulda.freifunk.net/data/meshviewer.json
Mar 05 02:20:09 elsa census-exporter.py[1622627]: meshviewer        https://www.freifunk-gera-greiz.de/meshviewer/data/meshviewer.json
Mar 05 02:20:09 elsa census-exporter.py[1622627]: meshviewer        https://map.cccgoe.de/data/meshviewer.json
Mar 05 02:20:09 elsa census-exporter.py[1622627]: meshviewer        https://map.ffhgw.de/data/meshviewer.json
Mar 05 02:20:10 elsa census-exporter.py[1622627]: meshviewer        https://unifi.greifswald.freifunk.net/data/meshviewer.json
Mar 05 02:20:13 elsa census-exporter.py[1622627]: 2023-03-05 02:20.13 Unexpected HTTP status code    status_code=502 url=https://gg.indie-freifunk.net/data/meshviewer.json
Mar 05 02:20:14 elsa census-exporter.py[1622627]: nodes.json v2        https://map03.4830.org/data/nodes.json
Mar 05 02:20:14 elsa census-exporter.py[1622627]: nodes.json v2        https://freifunk-halle.org/hopglass-server/nodes.json
Mar 05 02:20:14 elsa census-exporter.py[1622627]: nodes.json v2        https://hopglass-backend.hamburg.freifunk.net/nodes.json
Mar 05 02:20:14 elsa census-exporter.py[1622627]: meshviewer        https://map.harz.freifunk.net/data/meshviewer.json
Mar 05 02:20:14 elsa census-exporter.py[1622627]: nodes.json v2        https://hannover.freifunk.net/api/nodes.json
Mar 05 02:20:15 elsa census-exporter.py[1622627]: nodes.json v2        https://karte.freifunk-heilsbronn.de/data/nodes.json
Mar 05 02:20:15 elsa census-exporter.py[1622627]: nodes.json v2        https://map.freifunk-hennef.de/data/nodes.json
Mar 05 02:20:15 elsa census-exporter.py[1622627]: nodes.json v2        https://map.hochstift.freifunk.net/data/nodes.json
Mar 05 02:20:15 elsa census-exporter.py[1622627]: meshviewer        https://map.freifunk-ingolstadt.de/yanic/meshviewer.json
Mar 05 02:20:15 elsa census-exporter.py[1622627]: meshviewer        https://api.karlsruhe.freifunk.net/yanic/meshviewer/meshviewer.json
Mar 05 02:20:15 elsa census-exporter.py[1622627]: nodes.json v2        https://hopglass.freifunk.in-kiel.de/nodes.json
Mar 05 02:20:15 elsa census-exporter.py[1622627]: 2023-03-05 02:20.15 Exception caught while fetching url ex=ConnectionError(MaxRetryError("HTTPSConnectionPool(host='falterturm.freifunk-vec.de', port=443): Max retries exceeded with url: /meshviewer.json (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f5eba590be0>: Failed to establish a new connection: [Errno -2] Name or service not known'))"))
Mar 05 02:20:16 elsa census-exporter.py[1622627]: meshviewer        https://ffkwkarte.freifunk-koenigswinter.de/data/koenigswinter/meshviewer.json
Mar 05 02:20:16 elsa census-exporter.py[1622627]: meshviewer        http://karte.leipzig.freifunk.net:8018/meshviewer2/data/meshviewer.json
Mar 05 02:20:16 elsa census-exporter.py[1622627]: meshviewer        https://map.fflev.de/data/meshviewer.json
Mar 05 02:20:16 elsa census-exporter.py[1622627]: meshviewer        https://map.freifunk-lippe.de/map/data/meshviewer.json
Mar 05 02:20:17 elsa census-exporter.py[1622627]: nodes.json v2        https://map.luebeck.freifunk.net/data/nodes.json
Mar 05 02:20:26 elsa census-exporter.py[1622627]: 2023-03-05 02:20.26 Exception caught while fetching url ex=ConnectionError(MaxRetryError("HTTPConnectionPool(host='gw4.freifunk-lueneburg.de', port=80): Max retries exceeded with url: /meshviewer/data/meshviewer.json (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5eba598ca0>: Failed to establish a new connection: [Errno 113] No route to host'))"))
Mar 05 02:20:31 elsa census-exporter.py[1622627]: 2023-03-05 02:20.31 Exception caught while fetching url ex=ConnectTimeout(MaxRetryError("HTTPConnectionPool(host='gateway-01.mesh.pjodd.se', port=80): Max retries exceeded with url: /hopglass/nodes.json (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7f5eba0dbf10>, 'Connection to gateway-01.mesh.pjodd.se timed out. (connect timeout=5)'))"))
Mar 05 02:20:31 elsa census-exporter.py[1622627]: meshviewer        https://map.freifunk-mwu.de/data/meshviewer.json
Mar 05 02:20:31 elsa census-exporter.py[1622627]: nodes.json v1        https://api.marburg.freifunk.net/nodes.json
Mar 05 02:20:31 elsa census-exporter.py[1622627]: meshviewer        https://map.ffmuc.net/data/meshviewer.json
Mar 05 02:20:32 elsa census-exporter.py[1622627]: meshviewer        https://map.freifunk-myk.de/data/meshviewer.json
Mar 05 02:20:32 elsa census-exporter.py[1622627]: nodes.json v2        https://karte.freifunk-muensterland.de/data/nodes.json
Mar 05 02:20:32 elsa census-exporter.py[1622627]: nodes.json v2        https://karte.freifunk-neuendettelsau.de/data/nodes.json
Mar 05 02:20:32 elsa census-exporter.py[1622627]: nodes.json v2        https://karte.freifunk-neustadt-aisch.de/data/nodes.json
Mar 05 02:20:42 elsa census-exporter.py[1622627]: 2023-03-05 02:20.42 Exception caught while fetching url ex=ConnectTimeout(MaxRetryError("HTTPSConnectionPool(host='map.freifunk-niersufer.de', port=443): Max retries exceeded with url: /data/meshviewer.json (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f5eba1724f0>, 'Connection to map.freifunk-niersufer.de timed out. (connect timeout=5)'))"))
Mar 05 02:20:42 elsa census-exporter.py[1622627]: nodes.json v2        https://map.freifunknord.de/data/nodes.json
Mar 05 02:20:42 elsa census-exporter.py[1622627]: meshviewer        https://mesh.freifunknord.de/iz/meshviewer.json
Mar 05 02:20:42 elsa census-exporter.py[1622627]: 2023-03-05 02:20.42 Unexpected HTTP status code    status_code=404 url=https://mesh.freifunknord.de/ploh/meshviewer.json
Mar 05 02:20:42 elsa census-exporter.py[1622627]: meshviewer        https://map02.freifunk-nordheide.de/data/meshviewer.json
Mar 05 02:20:43 elsa census-exporter.py[1622627]: meshviewer        https://map.freifunk-nordhessen.de/data/meshviewer.json
Mar 05 02:20:43 elsa census-exporter.py[1622627]: meshviewer        https://map.ffnw.de/data/meshviewer.json
Mar 05 02:20:43 elsa census-exporter.py[1622627]: meshviewer        https://coco.ffoh.de/meshviewer/data/meshviewer.json
Mar 05 02:20:43 elsa census-exporter.py[1622627]: nodes.json v1        https://meshviewer.pinneberg.freifunk.net/data/nodes.json
Mar 05 02:20:43 elsa census-exporter.py[1622627]: meshviewer        https://regensburg.freifunk.net/data/meshviewer.json
Mar 05 02:20:43 elsa census-exporter.py[1622627]: nodes.json v2        https://map.freifunk-rheinbach.de/data/nodes.json
Mar 05 02:20:43 elsa census-exporter.py[1622627]: meshviewer        https://map.ffrn.de/data/meshviewer.json
Mar 05 02:20:43 elsa census-exporter.py[1622627]: meshviewer        https://map.freifunk-rhein-sieg.net/data/lohmar1/domain01/meshviewer.json
Mar 05 02:20:43 elsa census-exporter.py[1622627]: meshviewer        https://map.freifunk-rhein-sieg.net/data/lohmar1/domain03/meshviewer.json
Mar 05 02:20:44 elsa census-exporter.py[1622627]: meshviewer        https://map.freifunk-rhein-sieg.net/data/lohmar2/domain05/meshviewer.json
Mar 05 02:20:44 elsa census-exporter.py[1622627]: meshviewer        https://map.freifunk-rhein-sieg.net/data/lohmar3/domain04/meshviewer.json
Mar 05 02:20:44 elsa census-exporter.py[1622627]: meshviewer        https://map.freifunk-rhein-sieg.net/data/lohmar3/domain07/meshviewer.json
Mar 05 02:20:44 elsa census-exporter.py[1622627]: meshviewer        https://map.freifunk-rhein-sieg.net/data/lohmar3/domain11/meshviewer.json
Mar 05 02:20:44 elsa census-exporter.py[1622627]: meshviewer        https://map.freifunk-rhein-sieg.net/data/siegburg1/domain01/meshviewer.json
Mar 05 02:20:44 elsa census-exporter.py[1622627]: meshviewer        https://map.freifunk-rhein-sieg.net/data/siegburg2/domain02/meshviewer.json
Mar 05 02:20:44 elsa census-exporter.py[1622627]: meshviewer        https://map.freifunk-rhein-sieg.net/data/siegburg2/domain05/meshviewer.json
Mar 05 02:20:44 elsa census-exporter.py[1622627]: meshviewer        https://map.freifunk-rhein-sieg.net/data/tdf4/tdf/meshviewer.json
Mar 05 02:20:44 elsa census-exporter.py[1622627]: meshviewer        https://map.freifunk-rhein-sieg.net/data/tdf5/inn/meshviewer.json
Mar 05 02:20:44 elsa census-exporter.py[1622627]: meshviewer        https://map.freifunk-rhein-sieg.net/data/tdf6/flu/meshviewer.json
Mar 05 02:20:44 elsa census-exporter.py[1622627]: meshviewer        https://map.freifunk-rhein-sieg.net/data/voreifel1/domain15/meshviewer.json
Mar 05 02:20:44 elsa census-exporter.py[1622627]: meshviewer        https://map.roenkhausen.de/meshviewer/data/meshviewer.json
Mar 05 02:20:44 elsa census-exporter.py[1622627]: meshviewer        https://netmon.freifunk-rothenburg.de/meshviewer/data/meshviewer.json
Mar 05 02:20:44 elsa census-exporter.py[1622627]: meshviewer        https://map.freifunk.ruhr/data/rgw/meshviewer.json
Mar 05 02:20:44 elsa census-exporter.py[1622627]: nodes.json v2        https://mgmt.saar.freifunk.net/data/nodes.json
Mar 05 02:20:45 elsa census-exporter.py[1622627]: meshviewer        https://map.freifunk-suedholstein.de/data/meshviewer.json
Mar 05 02:20:45 elsa census-exporter.py[1622627]: meshviewer        https://new.freifunk-suedpfalz.de/netzausbau/data/meshviewer.json
Mar 05 02:20:45 elsa census-exporter.py[1622627]: nodes.json v2        https://services.freifunk-suedwest.de/hopglass/nodes.json
Mar 05 02:20:45 elsa census-exporter.py[1622627]: meshviewer        https://map.freifunk-stuttgart.de/data/meshviewer.json
Mar 05 02:20:45 elsa census-exporter.py[1622627]: meshviewer        http://maps.tackin.de/data/meshviewer.json
Mar 05 02:20:45 elsa census-exporter.py[1622627]: meshviewer        https://map.freifunk-troisdorf.de/data/api/meshviewer.json
Mar 05 02:20:45 elsa census-exporter.py[1622627]: meshviewer        https://map.freifunk-3laendereck.net/map-data/fftut/meshviewer.json
Mar 05 02:20:46 elsa census-exporter.py[1622627]: nodes.json v2        https://map.freifunk-uelzen.de/data/nodes.json
Mar 05 02:20:46 elsa census-exporter.py[1622627]: 2023-03-05 02:20.46 Exception caught while fetching url ex=SSLError(MaxRetryError("HTTPSConnectionPool(host='map.freifunk-ulm.de', port=443): Max retries exceeded with url: /data/meshviewer.json (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1123)')))"))
Mar 05 02:20:46 elsa census-exporter.py[1622627]: meshviewer        https://mapdata.freifunk-vogtland.net/meshviewer.json
Mar 05 02:20:46 elsa census-exporter.py[1622627]: meshviewer        https://api.karlsruhe.freifunk.net/yanic/meshviewer/ffwp/meshviewer.json
Mar 05 02:20:46 elsa census-exporter.py[1622627]: meshviewer        https://map.freifunk-winterberg.net/data/ar/meshviewer.json
Mar 05 02:20:46 elsa census-exporter.py[1622627]: meshviewer        https://map.freifunk-winterberg.net/data/bs/meshviewer.json
Mar 05 02:20:46 elsa census-exporter.py[1622627]: meshviewer        https://map.freifunk-winterberg.net/data/doerfer/meshviewer.json
Mar 05 02:20:46 elsa census-exporter.py[1622627]: meshviewer        https://map.freifunk-winterberg.net/data/hd/meshviewer.json
Mar 05 02:20:46 elsa census-exporter.py[1622627]: meshviewer        https://map.freifunk-winterberg.net/data/hlb/meshviewer.json
Mar 05 02:20:46 elsa census-exporter.py[1622627]: meshviewer        https://map.freifunk-winterberg.net/data/ln/meshviewer.json
Mar 05 02:20:46 elsa census-exporter.py[1622627]: meshviewer        https://map.freifunk-winterberg.net/data/mb/meshviewer.json
Mar 05 02:20:46 elsa census-exporter.py[1622627]: meshviewer        https://map.freifunk-winterberg.net/data/mdb/meshviewer.json
Mar 05 02:20:46 elsa census-exporter.py[1622627]: meshviewer        https://map.freifunk-winterberg.net/data/mq/meshviewer.json
Mar 05 02:20:46 elsa census-exporter.py[1622627]: meshviewer        https://map.freifunk-winterberg.net/data/sdlh/meshviewer.json
Mar 05 02:20:47 elsa census-exporter.py[1622627]: meshviewer        https://map.freifunk-winterberg.net/data/su/meshviewer.json
Mar 05 02:20:47 elsa census-exporter.py[1622627]: meshviewer        https://map.freifunk-winterberg.net/data/wa/meshviewer.json
Mar 05 02:20:47 elsa census-exporter.py[1622627]: meshviewer        https://map.freifunk-winterberg.net/data/wtbg/meshviewer.json
Mar 05 02:20:47 elsa census-exporter.py[1622627]: 47150 unique nodes
Mar 05 02:20:47 elsa census-exporter.py[1622627]: 521 duplicates skipped
Mar 05 02:20:47 elsa systemd[1]: gluon-census.service: Succeeded.
Mar 05 02:20:47 elsa systemd[1]: gluon-census.service: Consumed 5.509s CPU time.