search

LukePrior / nbn-upgrade-map

Interactive map showing premises eligible for the NBN FTTP upgrade program.
https://nbn.lukeprior.com/
MIT License
134 stars 11 forks source link

Create process to accelerate detection of service changes #155

Closed kiwisg124 closed 1 year ago

kiwisg124 commented 1 year ago

This proposed process attempts to harness internet volunteers to pre-process suburb(s) at intervals and so reduce efforts to find the service changes and present more timely results to internet access seekers.

The goal being to reduce the number of API calls performed by the NBN upgrade map application by distributing the workload.

Every address within a state and suburb is examined at periodic intervals.

Motivated service users will be checking their own area for signs of progress.

Should they be proficient, they might submit their findings of locations with a change to the NBN upgrade application team.

For privacy reasons, the fact of a volunteer being assigned might be recorded as a binary flag 01 against the suburb.

And the information upload limited to address ID entries commonly LOC######

Example: A city in NSW will be orderable Dec-23 and last processed 30-May. At that time 67 FTTP among 6734 service entries in geojson file.

Today results of v2details API finds 82 / 6824 with FTTP technology type (still running).

For suburbs with a small FTTx cohort processing a changed service list will avoid many calls.

For suburbs with HFC there will be many potentially wasted calls if all are checked every time.

LukePrior commented 1 year ago

Some good ideas here, coordinating with volunteers will take time to implement if we proceed with such a plan.

I do take particular note of not rechecking HFC addresses by default as these will only get upgraded in the case of a individual upgrade.

jebbusjebbus commented 1 year ago

more than happy to lend a hand as long as someone explains the process of how to do it to me.

kiwisg124 commented 1 year ago

Clarification (idea): (1) support services processing on demand during the next available batch window by adding a suburb trigger file to an agreed GIT folder (2) enhance main.py support an optional premises filename criteria command line argument when the file name criteria is provided, read the files(s) to determine in-scope services instead of every address provided by database in STATE/SUBURB query results retrieve each service mentioned in the file Call V2 details for the specified LOC ID parse formatted address2 in HTML response if the last word is Australia .. take second to last word as STATE take fourth to last word as SUBURB examine technology type, reasonCode, altReasonCode to detect FTTH and services with upgrade potential

Looking at DB design now.

lyricnz commented 1 year ago

The DB is just a subset of the upstream GNAF database. It's nothing to do with NBN as such (other than they use/used a copy of the same database).

lyricnz commented 1 year ago

With latest code, all announced-suburbs will be reprocessed every 30 days. Unannounced suburbs will be processed as quickly as possible, looks like about the same. I'm not sure additional process is required?

LukePrior commented 1 year ago

I'm going to adjust the dates around a bit to find a good balance between announced and all update frequency but yeah I think it should be fine.