search

frictionlessdata / pilot-catalyst

Pilot with the Catalyst cooperative
2 stars 2 forks source link

Added FK checks to goodtables #3

Closed roll closed 4 years ago

roll commented 4 years ago

@cmgosnell @zaneselvans

Please test/play with it. Errors are possible at the moment. I will polish it during integration to PUDL and based on your feedback.

pip install --upgrade git+https://github.com/frictionlessdata/goodtables-py.git@foreign-keys

All the information is here:

cmgosnell commented 4 years ago

hey @roll! sorry for the delay on this. I incorporated the fk checks and they just worked! it made the validation way, way slower but that is to be expected in some regards.

One question: you gave the example of running checks=['structure', 'schema', 'foreign-key'] in validate... but are there tests that we are avoided by doing this vs not specifying any specific checks?

roll commented 4 years ago

@cmgosnell Hi, no worries. BTW recently we got an improvement to our FK check system - https://github.com/frictionlessdata/tableschema-py/pull/254. It speeds up it drastically. I'll apply it here also.

By default checks=['structure', 'schema'] so the line above runs all of the default + foreign-key

roll commented 4 years ago

@cmgosnell It's released as goodtables@2.3 with a faster algorithm. Could you please try?