search

nhsx / au-azure-databricks

NHSX AU-Data Engineering - Azure Databricks Analytics
MIT License
7 stars 8 forks source link

Refactor URL selection as a function #2

Closed craig-shenton closed 3 years ago

craig-shenton commented 3 years ago

Refactor as a function

_Originally posted by @craig-shenton in https://github.com/nhsx/au-azure-databricks/pull/1#discussion_r710852038_

search_url = "https://ons-inspire.esriuk.com/arcgis/rest/services/Health_Boundaries/"
url_1 = "https://ons-inspire.esriuk.com"
url_2 = '/0/query?where=1%3D1&outFields=*&outSR=4326&f=json'
page = requests.get(search_url)
response = urlreq.urlopen(search_url)
soup = BeautifulSoup(response.read(), "lxml")
ccg_url = soup.find_all('a', href=re.compile("Clinical_Commissioning_Groups_April"))[-1].get('href')

full_url = url_1 + ccg_url + url_2 
with urlopen(full_url) as response:
  ccg_geojson = json.load(response)
mattia-ficarelli commented 3 years ago

complete - function added to the function notebook