search

planningalerts-scrapers / issues

Only for keeping track of all issues related to scraping
0 stars 0 forks source link

Banyule City Council #515

Open mlandauer opened 2 years ago

mlandauer commented 2 years ago

This issue has been automatically created by PlanningAlerts for Banyule City Council

It uses the scraper planningalerts-scrapers/banyule

Only close this issue once the authority is working again on PlanningAlerts. Otherwise a new issue will just automatically created. Also, if there is a duplicate issue close the other one in favour of this one.

Front logo Front conversations

mlandauer commented 2 years ago

Looks like applications are still available at https://www.banyule.vic.gov.au/Planning-building/Review-local-planning-applications/Planning-applications-on-public-notice

mlandauer commented 2 years ago

Oh lordy. Looks like they're using something called Incapsula to block scrapers. Unbelievable.

mlandauer commented 2 years ago

@katska this might be really tall order but do you think you could try getting in touch with the council and see if they could unblock the development application page from the "incapsula" system that they're using to block scrapers. I guess it depends on whether they think PlanningAlerts is a valuable enough service to their citizens.

katska commented 2 years ago

yup. I'll do that now. Sorry I didn't see before @mlandauer

katska commented 2 years ago

I've a draft email to them, and would like to be clear about the request we're making. Would I be asking that they allow requests from morph.io or can we be more specific? @mlandauer do you agree and is that right or would you add something/else?

mlandauer commented 12 months ago

@katska Either that they disable incapsula for the application register pages starting at https://www.banyule.vic.gov.au/Planning-building/Review-local-planning-applications/Planning-applications-on-public-notice or that they whitelist morph.io (via IP or whatever), whichever they are willing and able to do.

katska commented 11 months ago

draft email as per #557 drafted, awaiting feedback

katska commented 10 months ago

I wrote to them last year and didn't hear back. Sent them a reminder .

katska commented 10 months ago

Council responds: 'It appears that the security architecture provided by our website vendor does not allow the planning application data to be gathered from our website.  This is not an intentional block on Planning Alerts but part of the general website security that we have.  There are no immediate plans to change this.'