419 duck db version

[james-westwood]

@paigeh1 this is the DuckDB code

Pull Request submission

Insert detailed bullet points about your changes here!

Insert any instructions to help the reviewer, e.g. "install new requirements from requirements.txt"

*Let the reviewer know what data files are needed (to be grabbed from sync)

Closes or fixes

Fixes #

Code

• [ ] Requirements My/our code matches the requirements of the ticket
• [ ] Functionality: New functions meet requirements in issue ticket
• [ ] Compliant Code Code is as PEP 8 compliant as I can humanly make it
• [ ] Code runs The code runs on my machine
• [ ] Clean Code
  • [ ] Code has been linted (use your favourite linter)
  • [ ] Code adheres to DRY

Documentation

Any new code includes all the following forms of documentation:

• [ ] Function Documentation Docstrings within the function(s') definition(s) have been created
  • [ ] Includes parameters and returns for all major functions
  • [ ] Includes data types
• [ ] Updated Documentation: Working doc has been updated and a new ticket to update the master documentation has been created.

Data

• [ ] All data needed to run this script has been added to Sync

Testing

• [ ] Unit tests Unit tests have been created and are passing or a new ticket to create tests has been created

---

Peer Review Section

• [ ] All requirements install from (updated) requirements.txt
• [ ] Documentation has been created and is clear - check the google Doc
• [ ] Doctrings (Google format) have been created and accurately describe the function's functionality
• [ ] Unit tests pass, or if not present a new ticket to create tests has been created
• [ ] Code runs The code runs on reviewer's machine

Final approval (post-review)

The author has responded to my review and made changes to my satisfaction.

• [ ] I recommend merging this request.

---

Review comments

Insert detailed comments here!

These might include, but not exclusively:

• bugs that need fixing (does it work as expected? and does it work with other code that it is likely to interact with?)
• alternative methods (could it be written more efficiently or with more clarity?)
• documentation improvements (does the documentation reflect how the code actually works?)
• additional tests that should be implemented (do the tests effectively assure that it works correctly?)
• code style improvements (could the code be written more clearly?)

Your suggestions should be tailored to the code that you are reviewing. Be critical and clear, but not mean. Ask questions and set actions.

[paigeh-fsa]

Issue with loading all csvs, going to look into this. Appears to just be loading London atm

[paigeh-fsa]

Next steps:

• [ ] Rerun the script
• [ ] Check output of the data is what we expect
• [ ] Make the final output data as the input data used in the pipeline (SDG_eng_wales.py)

[jwestw]

Listing some requirements for the xlsx output:

• headers on 5th row (starting from 1, not 0)
• 3 tabs to be named:
  • Mid-{year} Persons
  • Mid-{year} Males
  • Mid-{year} Females
• Columns to be named: ["OA11CD", "LSOA11CD", "All Ages".....] then "0", "1", "2"....through to "89" and "90+"
• Other tabs and information probably not needed
• File type is "Spreadsheet (application/vnd.openxmlformats-officedocument.spreadsheetml.sheet)"

[jwestw]

I am stuck on running this code after having re-downloaded the archive csvs again. This is a problem I had before and was previously (if I remember correctly) to do with the ordering of columns in the column dtypes dictionary.

I have read the csvs in via the command line head -n 1 path_to_csv and this showed that the column order was the same as my dictionary. Same for reading it in Pandas.

The dtypes are correct too I believe.

The code which is trying to be converted to Int (which shouldn't be) is from the "LAD11CD" column.

[james-westwood]

This now works to rename columns and create all tabs in the excel