SashaWeinstein
commented
2 years ago The next upgrade is having the tests and export process point to the same iterable of accessor functions. That way we don't have to worry about adding new accessor functions to the both processes. Once that is done if the tests pass we will know with certainty that we won't run into issues with merging on indices in the export process.
Expand testing
There are three reasons I want to expand testing at this point 1) We have good helpful existing tests. 2) The run-time for producing our indicators has gone dramatically better now that we aren't relying on the MDAT API or survey package. This makes tests much more useful to developers 3) I chose to stop writing tests last week and focus only on getting exports out to DO. I made this choice because we had deadlines coming up fast. However there were so many mistakes in the export process that I think even over a short timespan this choice wasted more time than it saved.
Good existing tests
We have good tests that test whether the accessors for each indicator 1) return a dataframe and 2) that dataframe has the correct geographies as the index. These tests catch potential issues with collate process before collate is run. They also (obviously) catch broken filepaths, outdated imports, etc.
Next steps
We want to run these good existing tests on all indicators. Then we want to produce a list of the exact column labels we expect from various ingestion scripts and check that the outputted data has exactly those columns