nismod / ukpopulation

Population and demographics projection module, developed for ITRC/MISTRAL
MIT License
13 stars 7 forks source link

ukpopulation-43 Rebuild the test data with new 2018 data. #44

Closed BenjaminIsaac0111 closed 4 years ago

BenjaminIsaac0111 commented 4 years ago

Migrated to new 2018 data.

Edited tests now work on with new census data rebuilt using setup_test_data.py.

NOTE: If you need to migrate to newly released data, you will need to download the full data into .ukpopulation/cache with an API Key specified in your environment. This can either be done by placing a file called NOMIS_API_KEY inside the download target dir that contains your personal API Key off nomisweb. There will be new file hashes for only the EN files starting with NM_200*1.py since the data has changed. After you have downloaded the data into .ukpopulation/cache you should run the setup_test_data.py after changing the the file names inside this script to point to the newly downloaded data - this will build smaller versions of the new data and place them into the tests/raw_data directory. Once this has been completed, you will then need to re-download the data using the DUMMY(which is contained in tests/raw_data/NOMIS_API_KEY) key directly into test/raw_data to copy the hashes of the file names and overwrite the file names of the API key loaded data setup by setup_test_data.py contained in raw data. This can be done either by running a unittest since which will attempt to download the data in raw_files in setup.

IMPORTANT: These changes are purely to get the tests to run and check the code works. The way this change was handled is very convoluted and should/will be changed in future so that setting up the test data is handled using a setup script of some sort.