351 add highly serviced stops scot

[jwestw]

Order to merge pull requests: 1) This one: #360 2) #361 3) #366

Pull Request submission

• Renamed SDG_11.2.1_main.py into main.py as this is more in line with standard practice (I know this wasn't part of the ticket and I honestly can't remember what prompted me to make this change at this point. I can change it back if it's going to create any regression errors)
• The main thing I was actually doing here was getting the highly serviced stops df into the scotland script, which I have done by simply importing that data object (stops_geo_df) from main. (Oh actually this is why I changed the name of the script, so I could simply write from main import.... whereas from SDG_11.2.1 import... didn't work due to the full stops.) I know importing a data object like this probably isn't best practice, but it prevents use and repeating code carrying out the same data import twice. When the pipelines are merged, this will not be a problem.
• After the stops are buffered, renamed the df to buffd_la_stops_geo_df to make it clearer what's going on.
• Created a new "points in polygons" function. This is more similar to the steps in the documentation here.
• I have created the .gitkeep files so the scotland data folder structure will be kept

Insert any instructions to help the reviewer, e.g. "install new requirements from requirements.txt"

Nothing extra needed in terms of dependencies

*Let the reviewer know what data files are needed (to be grabbed from sync)

Make sure you have all the Scotland data from sync! It won't run without those

Closes or fixes

Closes #351

Code

• [x] Requirements My/our code matches the requirements of the ticket
• [x] Functionality: New functions meet requirements in issue ticket
• [x] Compliant Code Code is as PEP 8 compliant as I can humanly make it
• [x] Code runs The code runs on my machine
• [x] Clean Code
  • [x] Code has been linted (use your favourite linter)
  • [x] Code adheres to DRY

Documentation

Any new code includes all the following forms of documentation:

• [x] Function Documentation Docstrings within the function(s') definition(s) have been created
  • [x] Includes parameters and returns for all major functions
  • [x] Includes data types
• [ ] Updated Documentation: Working doc has been updated and a new ticket to update the master documentation has been created. N/A????

Data

• [x] All data needed to run this script has been added to Sync

Testing

• [x] Unit tests Unit tests have been created and are passing or a new ticket to create tests has been created See: https://github.com/ONSdigital/SDG_11.2.1/issues/359

[james-westwood]

I just rebased master to this branch to kill of the merge conflicts. https://github.com/ONSdigital/SDG_11.2.1/compare/005a81d62c7e213bdc9e069b7d2fd2ff2e2282db..497e18b0fabc0a0823b077ed4b1b9abefe32b985 Should be all ready to go now.

[paigeh-fsa]

When import from main in Scotland, then will run the whole script. Therefore, need to add if name = main block.

[paigeh-fsa]

---

Peer Review Section

• [ ] All requirements install from (updated) requirements.txt
• [ ] Documentation has been created and is clear - check the google Doc
• [ ] Doctrings (Google format) have been created and accurately describe the function's functionality
• [ ] Unit tests pass, or if not present a new ticket to create tests has been created
• [ ] Code runs The code runs on reviewer's machine

Final approval (post-review)

The author has responded to my review and made changes to my satisfaction.

• [ ] I recommend merging this request.

---

Review comments

Insert detailed comments here!

These might include, but not exclusively:

• bugs that need fixing (does it work as expected? and does it work with other code that it is likely to interact with?)
• alternative methods (could it be written more efficiently or with more clarity?)
• documentation improvements (does the documentation reflect how the code actually works?)
• additional tests that should be implemented (do the tests effectively assure that it works correctly?)
• code style improvements (could the code be written more clearly?)

Your suggestions should be tailored to the code that you are reviewing. Be critical and clear, but not mean. Ask questions and set actions.

[jwestw]

@nkshaw23 I have fixed the problem of importing the df from main now. the `if name == 'main'" block was in the wrong place.

[jwestw]

The code now runs into another problem (at least for me). This is unrelated to my ticket and is detailed here: https://github.com/ONSdigital/SDG_11.2.1/issues/367

[nkshaw23]

Code ran through with the following output - seems slightly suspicious to me that every person is served but need to look in QGIS to see whats going on. This happened for another LA aswell so need to investigate.

A few general comments for the scotland process (not necessarily related to the MR code)

1) The oa to la mapper is massive. It contains mappings for the whole of the UK and we only use 2 columns. Could strip the dataset and re-upload to sync?

2) A couple of warnings of mixed dytpes

[Antonio-John]

The pipeline is running successfully however there is still a need to QA the figures this is covered in ticket #352 A ticket about resolving the warnings has been created here #370