search

ckan / ckanext-scheming

Easy, shareable custom CKAN schemas
Other
83 stars 162 forks source link

Skip validation #331

Open frafra opened 2 years ago

frafra commented 2 years ago

In certain scenarios, validation should be skipped. One of them is https://github.com/aptivate/ckanext-datasetversions.

Would it be possible to add an option to skip the validation? Something like '__skipvalidation': True set in data_dict that validate(...) could check.

https://github.com/ckan/ckanext-scheming/blob/3aa6ed7f134d14b882c5cea474d6519eed36bac9/ckanext/scheming/plugins.py#L222-L228

wardi commented 2 years ago

@frafra I think you're saying you want to store fields from old dataset schema versions so you want to skip the errors that might happen due to a schema change.

Skipping validation won't give you this in general because validation is also used to convert fields to and from formats that can be stored in the model. A lot more work would be required to support changing metadata schemas while wanting to keep old working versions in the same db.

frafra commented 2 years ago

Thanks @wardi. I do not want to do that due to schema changes (I extend the validators for that, to handle such kind of schema migration). I need that as ckanext-datasetversions creates a dataset to establish relations with other datasets, but such "parent/meta" dataset should not contain any resource or metadata, except the one needed to establish a child/parent relationship between datasets.

I would imagine there could be other cases where validation could be skipped, but I do not have any other cases for the moment.

wardi commented 2 years ago

sounds like it would be better to use a different schema for your "parent/meta" datasets that have no fields defined except the ones you need.

frafra commented 2 years ago

I think there could be some issues in mixing the types, but I can try.

frafra commented 2 years ago

There is an issue indeed: datasets created with a different type are not going to be listed in the same group, which is problematic, as even if each dataset type could have a "meta" type, it would lack of tags and useful information, so sorting and search done on the "meta" version would be rather useless.

frafra commented 1 year ago

The only solution I found was to implement a mechanism to skip validation, as suggested in the first post.