search

prio-data / views_partitioning

A python package for partitioning data for the ViEWS project.
Other
0 stars 0 forks source link

Selecting multiple partitions #3

Open Peder2911 opened 2 years ago

Peder2911 commented 2 years ago

There are cases where it would be useful to select multiple partitions at once, combining them into a single output. For example, if one would like to train with both the test and train partitions to do holdout evaluation:

A suggested syntax is:

partitioner = DataPartitioner({"A":{"train":(1,10),"test":(11,20),"holdout":(21,30)}})

train_and_test = partitioner("A", ("train", "test"))
hhegre commented 2 years ago

Should this be discussed in conjunction with syntax for partitioning across units (cross-validation) rather than across times?

Peder2911 commented 2 years ago

No, this only relates to time-partitioning.

hhegre commented 2 years ago

I might not have been clear. We need syntax for CV partition that follows the same logic. It might be useful to think of these two usages simultaneously? This type of combination across partitions could be useful also for CV.

Peder2911 commented 2 years ago

Sure, but unit partitioning doesn't have the same implementation, even though it might seem similar. I'd open a separate issue for unit partitioning, mentioning that it should have the same kind of partitioning features as time-partitioning.