worldbank / REaLTabFormer

A suite of auto-regressive and Seq2Seq (sequence-to-sequence) transformer models for tabular and relational synthetic data generation.
https://worldbank.github.io/REaLTabFormer/
MIT License
203 stars 23 forks source link

Support for pre-computation, saving, and loading of the sensitivity threshold #16

Open avsolatorio opened 1 year ago

avsolatorio commented 1 year ago

One of the current bottlenecks in fitting the non-relational model is the pre-computation of the sensitivity threshold.

A solution to remedy this is to allow for the pre-computation of the sensitivity threshold outside the fit function. One can specify a file containing the pre-computed value when fitting with the data. The file can be a JSON the contains the parameters for computing the sensitivity threshold, together with the results itself.

When the file is passed to the fit function, the function must first check if the parameters used in the pre-computation are consistent with the parameters passed in the fit function. Then, simply load it and skip the computation.

This is an excellent first issue if anyone is interested in contributing! :)