nickmalleson commented 3 years ago

This PR deals with issue https://github.com/Urban-Analytics/RAMP-UA/issues/255 , removal of the old, non-quant code and data. There are some minor things that have been noted in the discussion about the issue https://github.com/Urban-Analytics/RAMP-UA/issues/255#issuecomment-792907978 (no need to repeat them here)

Main changes:

Removed the code in population_initialisation.py that checked for 'non-quant' mode and ran the devon-specific shop and school allocation
The quant_object is no longer class-level parameter and instead is passed directly to population_initialisation constructor and then on to the functions that need it
Removed the quant-dir parameter in the default parameters file https://github.com/Urban-Analytics/RAMP-UA/blob/445649fa018e1348e423e3945b74054864752f97/model_parameters/default.yml#L12
Updated some tests and removed the tests that checked the Devon-specific files had been read correctly.

nickmalleson commented 3 years ago

@rory-improbable when you get a minute (not urgent) would you mind having a look at this for me? It relates to changes I am making on branch nickmalleson-issue-255 I have removed some old code that read in bespoke shopping and retail venues for Devon and everything seems to work OK when I run it, but the OpenCL tests are failing and I can't work out why (error below).

The only change that springs to mind is to the test_update_flows_kernel.py file: https://github.com/Urban-Analytics/RAMP-UA/blob/7a6b5e1bf44f112fb48b7b4917527106b012bedf/tests/opencl/test_update_flows_kernel.py . I needed to add the quant_object argument when conducting the population initialisation. But I am not sure why this would break the test. Any ideas? As I said the OpenCL model seems to run fine. Thanks!

Testing started at 13:29 ...
/Users/nick/opt/anaconda3/envs/ramp-ua/bin/python /Applications/PyCharm.app/Contents/plugins/python/helpers/pycharm/_jb_pytest_runner.py --path /Users/nick/research_not_syncd/git_projects/RAMP-UA/tests -- --last-failed
Launching pytest with arguments --last-failed /Users/nick/research_not_syncd/git_projects/RAMP-UA/tests in /Users/nick/research_not_syncd/git_projects/RAMP-UA

============================= test session starts ==============================
platform darwin -- Python 3.7.10, pytest-5.4.1, py-1.10.0, pluggy-0.13.1 -- /Users/nick/opt/anaconda3/envs/ramp-ua/bin/python
cachedir: .pytest_cache
rootdir: /Users/nick/research_not_syncd/git_projects/RAMP-UA
collecting ... collected 6 items
run-last-failure: rerun previous 6 failures (skipped 9 files)

tests/opencl/test_snapshot_converter.py::test_global_id_lookup 
tests/opencl/test_snapshot_converter.py::test_processes_people_flows 
tests/opencl/test_snapshot_converter.py::test_get_place_data 
tests/opencl/test_snapshot_converter.py::test_get_coordinates 
Converting Home flows for all people: 100%|██████████| 3/3 [00:00<00:00, 17747.41it/s]
Converting Retail flows for all people: 100%|██████████| 3/3 [00:00<00:00, 15553.66it/s]
Storing location type for Home: 100%|██████████| 3/3 [00:00<00:00, 35746.91it/s]
Storing location type for Retail: 100%|██████████| 5/5 [00:00<00:00, 54899.27it/s]
Processing coordinate data for Retail: 5it [00:00, 8878.71it/s]
Storing coordinates for homes: 3it [00:00, 35246.25it/s]
Fatal Python error: Aborted

Thread 0x000070000525c000 (most recent call first):
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/threading.py", line 300 in wait
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/threading.py", line 552 in wait
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/tqdm/_monitor.py", line 60 in run
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/threading.py", line 926 in _bootstrap_inner
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/threading.py", line 890 in _bootstrap

Thread 0x0000000114adbe00 (most recent call first):
  File "/Users/nick/research_not_syncd/git_projects/RAMP-UA/microsim/opencl/ramp/simulator.py", line 191 in step_kernel
  File "/Users/nick/research_not_syncd/git_projects/RAMP-UA/tests/opencl/test_update_flows_kernel.py", line 72 in test_correct_flow_calculation_no_lockdown
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/_pytest/python.py", line 184 in pytest_pyfunc_call
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/pluggy/manager.py", line 87 in <lambda>
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/pluggy/manager.py", line 93 in _hookexec
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/_pytest/python.py", line 1479 in runtest
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/_pytest/runner.py", line 135 in pytest_runtest_call
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/pluggy/manager.py", line 87 in <lambda>
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/pluggy/manager.py", line 93 in _hookexec
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/_pytest/runner.py", line 217 in <lambda>
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/_pytest/runner.py", line 244 in from_call
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/_pytest/runner.py", line 217 in call_runtest_hook
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/_pytest/runner.py", line 186 in call_and_report
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/_pytest/runner.py", line 100 in runtestprotocol
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/_pytest/runner.py", line 85 in pytest_runtest_protocol
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/pluggy/manager.py", line 87 in <lambda>
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/pluggy/manager.py", line 93 in _hookexec
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/_pytest/main.py", line 272 in pytest_runtestloop
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/pluggy/manager.py", line 87 in <lambda>
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/pluggy/manager.py", line 93 in _hookexec
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/_pytest/main.py", line 247 in _main
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/_pytest/main.py", line 191 in wrap_session
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/_pytest/main.py", line 240 in pytest_cmdline_main
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/pluggy/manager.py", line 87 in <lambda>
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/pluggy/manager.py", line 93 in _hookexec
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/_pytest/config/__init__.py", line 125 in main
  File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pycharm/_jb_pytest_runner.py", line 43 in <module>

Process finished with exit code 134 (interrupted by signal 6: SIGABRT)

rory-improbable commented 3 years ago

@rory-improbable when you get a minute (not urgent) would you mind having a look at this for me? It relates to changes I am making on branch nickmalleson-issue-255 I have removed some old code that read in bespoke shopping and retail venues for Devon and everything seems to work OK when I run it, but the OpenCL tests are failing and I can't work out why (error below).

The only change that springs to mind is to the test_update_flows_kernel.py file: https://github.com/Urban-Analytics/RAMP-UA/blob/7a6b5e1bf44f112fb48b7b4917527106b012bedf/tests/opencl/test_update_flows_kernel.py . I needed to add the quant_object argument when conducting the population initialisation. But I am not sure why this would break the test. Any ideas? As I said the OpenCL model seems to run fine. Thanks!

Testing started at 13:29 ...
/Users/nick/opt/anaconda3/envs/ramp-ua/bin/python /Applications/PyCharm.app/Contents/plugins/python/helpers/pycharm/_jb_pytest_runner.py --path /Users/nick/research_not_syncd/git_projects/RAMP-UA/tests -- --last-failed
Launching pytest with arguments --last-failed /Users/nick/research_not_syncd/git_projects/RAMP-UA/tests in /Users/nick/research_not_syncd/git_projects/RAMP-UA

============================= test session starts ==============================
platform darwin -- Python 3.7.10, pytest-5.4.1, py-1.10.0, pluggy-0.13.1 -- /Users/nick/opt/anaconda3/envs/ramp-ua/bin/python
cachedir: .pytest_cache
rootdir: /Users/nick/research_not_syncd/git_projects/RAMP-UA
collecting ... collected 6 items
run-last-failure: rerun previous 6 failures (skipped 9 files)

tests/opencl/test_snapshot_converter.py::test_global_id_lookup 
tests/opencl/test_snapshot_converter.py::test_processes_people_flows 
tests/opencl/test_snapshot_converter.py::test_get_place_data 
tests/opencl/test_snapshot_converter.py::test_get_coordinates 
Converting Home flows for all people: 100%|██████████| 3/3 [00:00<00:00, 17747.41it/s]
Converting Retail flows for all people: 100%|██████████| 3/3 [00:00<00:00, 15553.66it/s]
Storing location type for Home: 100%|██████████| 3/3 [00:00<00:00, 35746.91it/s]
Storing location type for Retail: 100%|██████████| 5/5 [00:00<00:00, 54899.27it/s]
Processing coordinate data for Retail: 5it [00:00, 8878.71it/s]
Storing coordinates for homes: 3it [00:00, 35246.25it/s]
Fatal Python error: Aborted

Thread 0x000070000525c000 (most recent call first):
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/threading.py", line 300 in wait
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/threading.py", line 552 in wait
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/tqdm/_monitor.py", line 60 in run
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/threading.py", line 926 in _bootstrap_inner
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/threading.py", line 890 in _bootstrap

Thread 0x0000000114adbe00 (most recent call first):
  File "/Users/nick/research_not_syncd/git_projects/RAMP-UA/microsim/opencl/ramp/simulator.py", line 191 in step_kernel
  File "/Users/nick/research_not_syncd/git_projects/RAMP-UA/tests/opencl/test_update_flows_kernel.py", line 72 in test_correct_flow_calculation_no_lockdown
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/_pytest/python.py", line 184 in pytest_pyfunc_call
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/pluggy/manager.py", line 87 in <lambda>
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/pluggy/manager.py", line 93 in _hookexec
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/_pytest/python.py", line 1479 in runtest
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/_pytest/runner.py", line 135 in pytest_runtest_call
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/pluggy/manager.py", line 87 in <lambda>
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/pluggy/manager.py", line 93 in _hookexec
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/_pytest/runner.py", line 217 in <lambda>
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/_pytest/runner.py", line 244 in from_call
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/_pytest/runner.py", line 217 in call_runtest_hook
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/_pytest/runner.py", line 186 in call_and_report
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/_pytest/runner.py", line 100 in runtestprotocol
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/_pytest/runner.py", line 85 in pytest_runtest_protocol
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/pluggy/manager.py", line 87 in <lambda>
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/pluggy/manager.py", line 93 in _hookexec
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/_pytest/main.py", line 272 in pytest_runtestloop
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/pluggy/manager.py", line 87 in <lambda>
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/pluggy/manager.py", line 93 in _hookexec
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/_pytest/main.py", line 247 in _main
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/_pytest/main.py", line 191 in wrap_session
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/_pytest/main.py", line 240 in pytest_cmdline_main
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/pluggy/manager.py", line 87 in <lambda>
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/pluggy/manager.py", line 93 in _hookexec
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/Users/nick/opt/anaconda3/envs/ramp-ua/lib/python3.7/site-packages/_pytest/config/__init__.py", line 125 in main
  File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pycharm/_jb_pytest_runner.py", line 43 in <module>

Process finished with exit code 134 (interrupted by signal 6: SIGABRT)

hey @nickmalleson I can definitely have a look at this. The tests are a bit brittle so I'm sure it's something simple that is breaking them.

rory-improbable commented 3 years ago

@nickmalleson so I've checked out this branch and all the tests are actually passing for me locally now (once I downloaded the devon_data folder). Is this the same situation that you have where the tests are passing locally but not in CI? or are they failing for you locally?

rory-improbable commented 3 years ago

OK I think the problem may be that the CI agent running the tests doesn't have the devon data downloaded.

It seems like only the MacOS agent is failing, the tests seem to pass on Ubuntu.

This is the error from the MacOS agent, it's the same error I got when I didn't have the devon_data directory downloaded

 ==================================== ERRORS ====================================
________________ ERROR collecting tests/test_microsim_model.py _________________
tests/test_microsim_model.py:19: in <module>
    "quant_object": QuantRampAPI(os.path.join("devon_data", "QUANT_RAMP"))
microsim/quant_api.py:27: in __init__
    QuantRampAPI.read_data(self.QUANT_DIR)
microsim/quant_api.py:35: in read_data
    cls.dfPrimaryPopulation = pd.read_csv(os.path.join(QUANT_DIR,'primaryPopulation.csv'))
/usr/local/miniconda/envs/ramp-ua/lib/python3.7/site-packages/pandas/io/parsers.py:676: in parser_f
    return _read(filepath_or_buffer, kwds)
/usr/local/miniconda/envs/ramp-ua/lib/python3.7/site-packages/pandas/io/parsers.py:448: in _read
    parser = TextFileReader(fp_or_buf, **kwds)
/usr/local/miniconda/envs/ramp-ua/lib/python3.7/site-packages/pandas/io/parsers.py:880: in __init__
    self._make_engine(self.engine)
/usr/local/miniconda/envs/ramp-ua/lib/python3.7/site-packages/pandas/io/parsers.py:1114: in _make_engine
    self._engine = CParserWrapper(self.f, **self.options)
/usr/local/miniconda/envs/ramp-ua/lib/python3.7/site-packages/pandas/io/parsers.py:1891: in __init__
    self._reader = parsers.TextReader(src, **kwds)
pandas/_libs/parsers.pyx:374: in pandas._libs.parsers.TextReader.__cinit__
    ???
pandas/_libs/parsers.pyx:674: in pandas._libs.parsers.TextReader._setup_parser_source
    ???
E   FileNotFoundError: [Errno 2] File devon_data/QUANT_RAMP/primaryPopulation.csv does not exist: 'devon_data/QUANT_RAMP/primaryPopulation.csv'

nickmalleson commented 3 years ago

Thanks @rory-improbable I think we're getting there. I hadn't thought about the data being an issue. This is failing for me locally (on a mac) but the data may well be a data issue. @Sparrow0hawk and I can discuss on Thursday. Enjoy your holiday.

nickmalleson commented 3 years ago

The error above doesn't occur when running pytest from the command-line, so lets ignore it. The online tests will fail though because the required QUANT data aren't included as part of the tests. Will deal with this in https://github.com/Urban-Analytics/RAMP-UA/issues/267

Sparrow0hawk commented 3 years ago

Just want to consolidate what needs to be done for this PR (@nickmalleson) let me know if i've missed anything:

[ ] - tests for quant_api [#262]
[ ] - adding dummy data for quant_api [#267]
[ ] - tidying up docstrings (working on in quant_api-tidy branch)

nickmalleson commented 3 years ago

@Sparrow0hawk I think that this one is part of the PR:

[ ] - adding dummy data for quant_api [#267]

But two these aren't:

[ ] - tests for quant_api [#262]

This is the subject of https://github.com/Urban-Analytics/RAMP-UA/issues/262

[ ] - tidying up docstrings (working on in quant_api-tidy branch)

This is the subject of: https://github.com/Urban-Analytics/RAMP-UA/issues/257

nickmalleson commented 3 years ago

@Sparrow0hawk I think this is good to go. Do you want to have a quick look through? I don't have anything in particular that I think you should look at.

codecov[bot] commented 3 years ago

Codecov Report

Merging #263 (d4d123b) into master (b3dd9a3) will increase coverage by 2.92%. The diff coverage is 96.42%.

:exclamation: Current head d4d123b differs from pull request most recent head 6369eb0. Consider uploading reports for the commit 6369eb0 to get more accurate results

@@            Coverage Diff             @@
##           master     #263      +/-   ##
==========================================
+ Coverage   42.01%   44.94%   +2.92%     
==========================================
  Files          23       22       -1     
  Lines        3056     2848     -208     
==========================================
- Hits         1284     1280       -4     
+ Misses       1772     1568     -204

Impacted Files	Coverage Δ
microsim/population_initialisation.py	`90.98% <96.15%> (+3.88%)`	:arrow_up:
microsim/quant_api.py	`83.33% <100.00%> (+83.33%)`	:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update b3dd9a3...6369eb0. Read the comment docs.

nickmalleson commented 3 years ago

Made one more minor change. Rather than having a test_mode that created small flow matrices, I just wrote out the small matricies and included them in the dummy data so that the behaviour of quant in the tests is exactly the same as it is under normal conditions)

Note, code to generate the matrices:

pickle.dump(np.tile(np.array([0.001] * 8), (8, 1)), open("FILENAME.bin", 'wb'))

nickmalleson commented 3 years ago

There's a few scripts you've included in the tests/dummy_data directory which may be better in utilities or a new directory in tests specifically for scripts around test data generation.

I used these to hack together the test data. We don't need them, but I think it's worth keeping them in case we want to re-create the test data again. It was a painful process. I thnk they're better where they are rather than in utilities, as putting them there implies that they have some useful function? Which they don't any more. But am happy with your choice.

Sparrow0hawk commented 3 years ago

There's a few scripts you've included in the tests/dummy_data directory which may be better in utilities or a new directory in tests specifically for scripts around test data generation.

I used these to hack together the test data. We don't need them, but I think it's worth keeping them in case we want to re-create the test data again. It was a painful process. I thnk they're better where they are rather than in utilities, as putting them there implies that they have some useful function? Which they don't any more. But am happy with your choice.

True, perhaps worth leaving a short README.md in this directory just to give a summary of what these files are for and brief guidance of how they work?

Urban-Analytics / RAMP-UA

Removing non-quant code and data #263

Codecov Report