scicloj / tablecloth

Dataset manipulation library built on the top of tech.ml.dataset
https://scicloj.github.io/tablecloth
MIT License
305 stars 27 forks source link

Update type interface to use type hierarchy in tablecloth.api.util #76

Closed ezmiller closed 2 years ago

ezmiller commented 2 years ago

Goal

In #71 , @genmeblog pointed out that there's a type hierarchy in the tablecloth.api.utils namespace. Following some discussion in #73 , this PR aims to set up a type interface where users mainly interact the types of elements in a column in terms of the "general" (rather than concrete) types. In other words, a user will want to know if an array has :integer rather than :int64 values.

Solution

I added some new functions in tablecloth.api.utils to help work with the type hierarchy tablecloth.api.utils/type-sets:

Then I adjusted typeof and typeof? to use this hierarchy, so now:

Please see the tests in this PR for examples.

Open Questions

ezmiller commented 2 years ago

@genmeblog just a reminder to take a look at this when you get a chance. :)

genmeblog commented 2 years ago

I did a review. I see mainly one issue with changing typeof contract. The other two are just optimization proposals.

ezmiller commented 2 years ago

@genmeblog: I changed things here so that:

ezmiller commented 2 years ago

@genmeblog if you have a moment, would you like to take another look at this?