Use libcovebods for data checks rather than developing more here

openownership / bodsdata

Data analysis tools to help analysts, journalists and anyone wanting to examine and dive into beneficial ownership data published in line with the Beneficial Ownership Data Standard

https://bods-data.openownership.org/

Other

9 stars 0 forks source link

Use libcovebods for data checks rather than developing more here #26

Open odscjames opened 1 year ago

odscjames commented 1 year ago

This library already has a lot of data checks coded and tested.

While it may not be suitable for use straight away (eg some of the ways it currently works use high memory with large files and that may not be suitable here, it only supports JSON not JSON Lines) it's in the plan to tackle those issues anyway so being forced to tackle them now is only a good thing!

radix0000 commented 1 year ago

Once we have battle tested the current basic consistency checks and have a more concrete idea of the requirements, it will probably make sense to revisit this at some point.