311 CSR reports prevent web scraping with python

[Charlie-Henry]

Related to work here https://github.com/cityofaustin/atd-data-tech/issues/12923

Hichame created some CSR reports for us. They are stored as CSVs at a URL. We are unable to integrate with 311 due to these files having a web app firewall called incapsula which specifically prevents us scraping the data from these files using python. Is there a way to disable this web app firewall or to allow us to authenticate to get around this?

Here's more background:

• We need to automate the process of reporting the status and response times for CSRs. Currently, staff has to wait for an automated email monthly in order to view the response time. Then, manually process that email's attachment.
• This is for a project for TPW's department director and for one other division.
• We were also told that that it is not possible to update the 311 dataset in the open data portal with the few additional columns that we need.

[Charlie-Henry]

After meeting with Andy at Motorola, he whitelisted the city's IP ranges so they are no longer flagged by incapsula!