openaustralia / planningalerts

Find out and have your say about what's being built and knocked down in your area.
https://www.planningalerts.org.au
Other
95 stars 50 forks source link

Scraper.py - #1816

Open Bigben83 opened 6 months ago

Bigben83 commented 6 months ago

Hello I have managed to get the scraper.py to scrape, by running sudo python3.6 scraper.py and it is giving an output as below. but how do I see this information in my local version of planningalerts, it appears to be saving it to a sqlite db ? or is there something I'm missing ?

I have cloned https://github.com/43South/tasmaniada and am using this repo but I dont know how you get it to show in a local copy of planningalerts that I have running.

sudo python3.6 scraper.py
INFO:root:starting
INFO:root:breakoday: found 6 applications
INFO:root:brighton: found 3 applications
INFO:root:burnie: found 5 applications
ERROR:root:failed to run for centralcoast
ERROR:root:failed to run for centralhighlands
ERROR:root:failed to run for circularhead
ERROR:root:failed to run for dorset
ERROR:root:failed to run for flinders
INFO:root:georgetown: found 0 applications
INFO:root:glamorganspringbay: found 16 applications
INFO:root:glenorchy: found 2 applications
INFO:root:kentish: found 3 applications
ERROR:root:failed to run for kingborough
INFO:root:kingisland: found 0 applications
ERROR:root:failed to run for latrobe
ERROR:root:failed to run for meander
ERROR:root:failed to run for northernmidlands
INFO:root:tasman: found 11 applications
INFO:root:waratahwynyard: found 6 applications
ERROR:root:failed to run for westcoast
INFO:root:westtamar: found 9 applications
DEBUG:root:{'authority_label': 'breakoday', 'council_reference': 'DA 024-2024', 'address': '36 John Street, Cornwall, Tasmania, Australia', 'description': 'Residential - Retrospective Approval of Annex', 'info_url': 'https://www.bodc.tas.gov.au/wp-content/uploads/2018/06/Advert-Plans-DA-024-2024.pdf', 'date_scraped': '2024-04-03T15:02:50.284148', 'date_received': None, 'on_notice_from': None, 'on_notice_to': '2024-04-03'}
DEBUG:root:{'authority_label': 'breakoday', 'council_reference': 'DA 244-2023', 'address': '126 Scamander Avenue, Scamander, Tasmania, Australia', 'description': 'Residential - New Shed', 'info_url': 'https://www.bodc.tas.gov.au/wp-content/uploads/2018/06/Advert-Plans-DA-244-2023.pdf', 'date_scraped': '2024-04-03T15:02:50.284148', 'date_received': None, 'on_notice_from': None, 'on_notice_to': '2024-04-03'}
DEBUG:root:{'authority_label': 'breakoday', 'council_reference': 'DA 079-2023', 'address': 'King Street (Access over Crown Land, 2 Dunn Street) Mathinna, Tasmania, Australia', 'description': 'Residential - Retrospective Approval of Demolition of Dwelling, Dwelling Alterations & Additions', 'info_url': 'https://www.bodc.tas.gov.au/wp-content/uploads/2018/06/Advert-Plans-DA-079-2023.pdf', 'date_scraped': '2024-04-03T15:02:50.284148', 'date_received': None, 'on_notice_from': None, 'on_notice_to': '2024-04-16'}

Just wondering if anyone can give me a bit of a steer on this project. Keen to get all the Tasmanian Councils feeding into PlanningAlerts Website.