script/update fails with FileNotFoundError for North East Derbyshire

[zarino]

Running script/update in the vagrant VM, on commit 504a819 (basic-views branch).

vagrant@stretch:/vagrant/caps$ script/update
getting the csv
replacing headers
getting plans
Error North East Derbyshire District Council https://www.ne-derbyshire.gov.uk/component/edocman/climate-change-action-plan-2019-2030: 404 Client Error: Not Found for url: https://www.ne-derbyshire.gov.uk/component/edocman/climate-change-action-plan-2019-2030
adding authority codes
adding authority info
adding missing authorities
Councils without authority codes 0
[]
adding website urls to councils
13 councils with no website
[['Derry City and Strabane District Council', nan], ['Derry City and Strabane District Council', nan], ['Antrim and Newtownabbey Borough Council', nan], ['Ards and North Down Borough Council', nan], ['Armagh City, Banbridge and Craigavon Borough Council', nan], ['Belfast City Council', nan], ['Causeway Coast and Glens Borough Council', nan], ['Derry City and Strabane District Council', nan], ['Fermanagh and Omagh District Council', nan], ['Lisburn and Castlereagh City Council', nan], ['Mid and East Antrim Borough Council', nan], ['Mid Ulster District Council', nan], ['Newry, Mourne and Down District Council', nan]]
adding text to the csv
council                                          Aberdeen City Council
search_link          https://www.google.com/search?q=Aberdeen+City+...
unfound                                                              X
credit                                                  @RewilderBeast
url                                                                NaN
date_retrieved                                              07/05/2020
time_period                                                        NaN
type                                                               NaN
scope                                                              NaN
status                                                             NaN
well_presented                                                     NaN
baseline_analysis                                                  NaN
notes                Council voted not to declare a climate emergen...
plan_due                                                           NaN
plan_path                                                          NaN
file_type                                                          NaN
charset                                                            NaN
authority_code                                                     ABE
authority_type                                                     NaN
wdtk_id                                                             39
mapit_area_code                                                    NaN
website_url                            https://www.aberdeencity.gov.uk
text                                                               NaN
Name: 0, dtype: object

[…snip…]

council                         North East Derbyshire District Council
search_link          https://www.google.com/search?q=North+East+Der...
unfound                                                            NaN
credit                                                             CMa
url                  https://www.ne-derbyshire.gov.uk/component/edo...
date_retrieved                                              01/02/2020
time_period                                                  2019-2030
type                                                       Action plan
scope                                                              NaN
status                                                        Approved
well_presented                                                     Yes
baseline_analysis                 Analysis of NEDD corporate emissions
notes                                                              NaN
plan_due                                                           NaN
plan_path                                                          NaN
file_type                                                          NaN
charset                                                            NaN
authority_code                                                     NED
authority_type                                                     NMD
wdtk_id                                                            626
mapit_area_code                                                    DIS
website_url                           http://www.ne-derbyshire.gov.uk/
text                                                               NaN
Name: 259, dtype: object
Traceback (most recent call last):
  File "script/../manage.py", line 15, in <module>
    execute_from_command_line(sys.argv)
  File "/vagrant/venv/lib/python3.5/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/vagrant/venv/lib/python3.5/site-packages/django/core/management/__init__.py", line 375, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/vagrant/venv/lib/python3.5/site-packages/django/core/management/base.py", line 323, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/vagrant/venv/lib/python3.5/site-packages/django/core/management/base.py", line 364, in execute
    output = self.handle(*args, **options)
  File "/vagrant/caps/caps/management/commands/import_plans.py", line 35, in handle
    document_file = open(join(settings.PLANS_DIR, plan_filename), "rb")
FileNotFoundError: [Errno 2] No such file or directory: 'data/plans/north-east-derbyshire-district-council-c869fc2'

[crowbot]

This happens when a plan can't be downloaded from the URL in the sheet. I've fixed this particular instance by getting a valid URL, but I think we should make a code fix too. I've created a ticket for that here.

[crowbot]

Closing as this specific instance is resolved.