Use of pydantic for PyData prototype?

[hammer]

Hey @eric-czech do you think pydantic would be useful for our IO and configuration approach?

[eric-czech]

That looks great, good find! I was playing around with it and found a few reasonable ways to attach documentation for more interactive configuration.

I've been wondering what the closest python equivalent to pureconfig is (which I loved) -- looks like that's pretty close.

[hammer]

Nice! What do you think about using it for providing schema-specific diagnostics for failed IO? I did not look closely at it but that use case is the one that jumped out at me given how often bioinformatics scripts fail on badly formatted inputs.

[eric-czech]

Hm maybe? Some thoughts:

• It could make sense for VCF variant and INFO fields if we wanted to assert some things like:
  • Can't set variant id as "."
  • AF must be in INFO
  • ALT alleles must be bi-allelic
• I could see it being useful for genomic feature tables since formats for those vary much more and picking a target schema for us or a user to hit would be helpful
• Importing sample info could be another use case though I bet most users would rather import directly to arrays/dataframes (to attach to our xr.Datasets) than use some single record factory method

If it doesn't JIT compile, then it's probably a nonstarter for IO though. My guess is that it won't.