370 add the data quality report to the csv as a separate sheet

[reecehill]

This pull request includes significant refactoring and enhancements to the CSV handling functionality in the project/npda/general_functions module. The changes primarily focus on organizing the CSV-related functions into a dedicated subdirectory, adding new functionalities, and improving existing ones.

Key changes include:

Refactoring and Reorganization:

• Moved CSV-related functions to a new subdirectory project/npda/general_functions/csv and updated import paths accordingly. [1] [2] [3] [4] [5] [6]

New Functionality:

• Added a new function download_file to handle file downloads and refactored download_csv to use this function. Introduced a new download_xlsx function for downloading XLSX files. [1] [2]

Enhancements:

• Introduced the csv_parse function to parse CSV files into pandas DataFrames, handling various edge cases and data type conversions.
• Added the write_errors_to_xlsx function to generate an XLSX file highlighting validation errors in the CSV data.

Bug Fixes:

• Fixed a bug in the serialize_error function to ensure keys are correctly serialized as integers.

Additional Improvements:

• Ensured that errors are written to an XLSX file if the CSV file is created during the upload process.

[reecehill]

Note to self:

Due to a circular import when importing project.npda.general_functions.csv_upload, I have:

1. Moved csv-related functions into a submodule project.npda.general_functions.csv

2. To further improve distinction between read_csv() pd.read_csv(),

   project.npda.general_functions.read_csv.read_csv()

   has been renamed (and moved) to

   project.npda.general_functions.csv.csv_parse.csv_parse()

[mbarton]

Seen on STAGING (created by @reecehill and merged by @eatyourpeas 7 minutes and 41 seconds ago) Please check your changes!