adrianshort / uk_planning_scraper

A Ruby gem to get planning applications data from UK council websites.
GNU Lesser General Public License v3.0
27 stars 19 forks source link

URI::encode throws warning: URI.escape is obsolete #42

Open adrianshort opened 3 years ago

adrianshort commented 3 years ago

Currently from these lines in northgate.rb:

results_url = URI::encode(base_url + response2.headers['Location'].gsub!('PS=10', 'PS=99999'))

app.info_url = URI::encode(generic_url + cells[0].at('a')[:href].strip)

We need to improve the URL escaping by understanding what's precisely required here and doing it properly.

This post gives some guidance:

https://docs.knapsackpro.com/2020/uri-escape-is-obsolete-percent-encoding-your-query-string