search

LukePrior / nbn-upgrade-map

Interactive map showing premises eligible for the NBN FTTP upgrade program.
https://nbn.lukeprior.com/
MIT License
124 stars 11 forks source link

Update GNAF database from upstream #370

Closed lyricnz closed 3 months ago

lyricnz commented 4 months ago

They May 2024 release is available from our upstream https://github.com/minus34/gnaf-loader/releases/tag/202405

lyricnz commented 4 months ago

I built the image here, using https://github.com/LukePrior/nbn-upgrade-map/blob/main/extra/db/update.sh

Then started the image, and kicked off a run

❯ ./code/main.py --suburb Marian --state QLD
2024-05-24 15:49:01,777 INFO MainThread Checking for externally updated geojson results...
2024-05-24 15:49:09,197 INFO MainThread ...done
2024-05-24 15:49:09,299 INFO MainThread Creating DB index...
2024-05-24 15:49:09,339 INFO MainThread Selecting explicit Marian, QLD
2024-05-24 15:49:09,340 INFO MainThread Processing Marian, QLD
2024-05-24 15:49:09,340 INFO MainThread Fetching all addresses for Marian, QLD
2024-05-24 15:49:09,384 INFO MainThread Fetched 1819 addresses from database
2024-05-24 15:49:09,389 INFO MainThread Loaded 1598 addresses from output file
2024-05-24 15:49:09,389 INFO MainThread Submitting 1819 requests to add NBNco data...
...
2024-05-24 15:50:35,774 INFO nbn_6 Completed 1819 requests
2024-05-24 15:50:35,782 INFO MainThread Completed. Tally of tech types: {'WIRELESS': 409, 'FTTN': 942, 'FTTP': 227, None: 221, 'NULL': 5, 'SATELLITE': 15}
2024-05-24 15:50:35,783 INFO MainThread Location ID types: {'LOC': 1599, 'None': 220}
2024-05-24 15:50:35,784 INFO MainThread Writing results to results/QLD/marian.geojson
2024-05-24 15:50:36,419 INFO MainThread Updating progress.json

Seemed to work fine

lyricnz commented 3 months ago

From G-NAF Data Release Report May 2024

Nationally, the May 2024 update of G-NAF shows an increase of 46,261 addresses overall(+0.30%). The total number of addresses in G-NAF now stands at 15,654,578 of which 14,820,195 or 94.67% are principal.

The confidence level distribution nationally for the May 2024 release has the confidence 2 addresses increasing by 40,753 from the February 2023 release with the total percentage of confidence 2 addresses increasing to 68.59% of principal addresses. The number of confidence 1 addresses has increased by 777 and represent 14.28% of all principal addresses while confidence 0 addresses have decreased by 528 and represent 17.13% of addresses.

With respect to address reliability, there was an increase of 43,434 address level, a decrease of 2,584 in street locality addresses and an increase of 152 in locality level addresses. The reliability of principal addresses in G-NAF is now made up of 97.59% address level, 2.37% street locality addresses and 0.03% locality addresses.

LukePrior commented 3 months ago

Alright I set the job to run let's see how that goes.

https://github.com/LukePrior/nbn-upgrade-map/actions/runs/9248125037

LukePrior commented 3 months ago

Failed due to lack of storage space on the GitHub runner.

LukePrior commented 3 months ago

Looks like the Docker image was pushed but not the GitHub release

lyricnz commented 3 months ago

Looks like it only failed in the SQLite conversion at the end. I don't think anyone except me, so I wouldn't worry too much.

https://github.com/LukePrior/nbn-upgrade-map/actions/runs/9248125037/job/25437948831#step:10:42

LukePrior commented 3 months ago

Seems to be working fine, can probably be closed if you are happy without that.