search

LukePrior / nbn-upgrade-map

Interactive map showing premises eligible for the NBN FTTP upgrade program.
https://nbn.lukeprior.com/
MIT License
137 stars 11 forks source link

Update GNAF database from upstream - August 2024 #389

Closed lyricnz closed 2 months ago

lyricnz commented 2 months ago

Like #370

Overview

  • Nationally, the Aug 2024 update of G-NAF shows an increase of 44,191 addresses overall (+0.28%). The total number of addresses in G-NAF now stands at 15,698,769 of which 14,864,193 or 94.68% are principal.
  • The confidence level distribution nationally for the Aug 2024 release has the confidence 2 addresses increasing by 41,258 from the May 2024 release with the total percentage of confidence 2 addresses increasing to 68.67% of principal addresses. The number of confidence 1 addresses has decreased by 2,617 and represent 14.22% of all principal addresses while confidence 0 addresses have increased by 5,357 and represent 17.11% of addresses.
  • With respect to address reliability, there was an increase of 45,286 address level, a decrease of 1,082 in street locality addresses and a decrease of 206 in locality level addresses. The reliability of principal addresses in G-NAF is now made up of 97.61% address level, 2.36% street locality addresses and 0.03% locality addresses.
lyricnz commented 2 months ago

Should I remove the conversion to sqllite from the job? Nobody except me probably uses it anyway (the postgres DB image is too big for my puny machine).

LukePrior commented 2 months ago

I don't mind either way, if you find it useful feel free to leave it in.

lyricnz commented 2 months ago

It failed to run last time in the GHA anyway

lyricnz commented 2 months ago

I'm doing a local test of the new image here now, but my machine struggles to even rebuild the image.

lyricnz commented 2 months ago

Seemed to work fine (using local lukeprior/nbn-upgrade-map-db:202408 image)

❯ ./code/main.py --suburb Marian --state QLD
2024-09-08 13:35:54,607 INFO MainThread Checking for externally updated geojson results...
2024-09-08 13:36:01,535 INFO MainThread ...done
2024-09-08 13:36:01,645 INFO MainThread Creating DB index...
2024-09-08 13:36:01,682 INFO MainThread Selecting explicit Marian, QLD
2024-09-08 13:36:01,682 INFO MainThread Processing Marian, QLD
2024-09-08 13:36:01,682 INFO MainThread Fetching all addresses for Marian, QLD
2024-09-08 13:36:01,728 INFO MainThread Fetched 1819 addresses from database
2024-09-08 13:36:01,733 INFO MainThread Loaded 1631 addresses from output file
2024-09-08 13:36:01,734 INFO MainThread Submitting 1819 requests to add NBNco data...
...
2024-09-08 13:40:33,658 INFO nbn_3 Completed 1819 requests
2024-09-08 13:40:33,664 INFO MainThread Completed. Tally of tech types: {'WIRELESS': 418, 'FTTN': 872, 'FTTP': 331, None: 188, 'NULL': 6, 'SATELLITE': 4}
2024-09-08 13:40:33,665 INFO MainThread Location ID types: {'LOC': 1631, 'None': 188}
2024-09-08 13:40:33,666 INFO MainThread Writing results to results/QLD/marian.geojson
2024-09-08 13:40:34,077 INFO MainThread Updating progress.json

Diff looks reasonable.

LukePrior commented 2 months ago

https://github.com/LukePrior/nbn-upgrade-map/actions/runs/10758533291

LukePrior commented 2 months ago

Ok looks like that went ok

LukePrior commented 2 months ago

Getting some no valid suggestion warnings in the current automatic CI run but not sure if more than usual baseline.