ESSS / pytest-regressions

Pytest plugin for regression testing: https://pytest-regressions.readthedocs.io
MIT License
182 stars 36 forks source link

would it be possible to export num_regression as yml files ? #159

Open 12rambau opened 9 months ago

12rambau commented 9 months ago

It's not a matter of making the actual test, it's a matter of checking a problem if no arise.

I'm moving from data_regression to num_regression as the value I compare are all numbers and changes from time to time (at the 10th decimal so not relevant). the previously generated file was super easy to humanly review looking like this:

AOT: 72
B1: 238.7171666246534
B11: 948.6636123014868
B12: 750.7564658432063
B2: 320.9578522813208
B3: 415.14734055961685
B4: 426.16661204940726
B5: 604.8148600957898
B6: 1007.6004411393997
B7: 1140.246659944543
B8: 1217.7626543987888
B8A: 1200.12724981094
B9: 1389.4672800604985
MSK_CLDPRB: 0.27464078648853035
MSK_SNWPRB: 0
NDVI: 0.47685885754442053
QA10: 0
QA20: 0
QA60: 0
SCL: 3.324502142677085
TCI_B: 33.158558104360964
TCI_G: 42.684144189563895
TCI_R: 43.699306780942756
WVP: 681.9462692210739

Now the output is .csv and it's barely readable (from a human standpoint):

,AOT,B1,B11,B12,B2,B3,B4,B5,B6,B7,B8,B8A,B9,MSK_CLDPRB,MSK_SNWPRB,NDVI,QA10,QA20,QA60,SCL,TCI_B,TCI_G,TCI_R,WVP
0,72,238.7171666246534,948.66361230148675,750.75646584320634,320.95785228132081,415.14734055961685,426.16661204940726,604.81486009578975,1007.6004411393997,1140.2466599445429,1217.7626543987888,1200.1272498109399,1389.4672800604985,0.27464078648853035,0,0.47685885754442053,0,0,0,3.3245021426770851,33.158558104360964,42.684144189563895,43.699306780942756,681.94626922107386

I would like to get the best of 2 worlds, the display of an easy to read yaml file (data_regression) and the comparaison of numercial value (num_regression). Do you think it could be an added feature ?

either an opt-in file format in num_regression or a special handle of numercal values in data_regresion ?

nicoddemus commented 9 months ago

Hey @12rambau,

Hmm good point. I think this would be a nice feature to have.

either an opt-in file format in num_regression or a special handle of numercal values in data_regresion ?

I think we should focus on the purpose of the function rather, than the data format the existing functions use, so I think an option to use a different data-format in num_regression makes more sense.

Something like:

class NumericFileFormat(Enum):
    Yaml = auto()
    Csv = auto()

class NumericRegressionFixture(DataFrameRegressionFixture):
    """
    Numeric Data Regression fixture implementation used on num_regression fixture.
    """

    def check(
        self,
        data_dict: Dict[str, Any],
        basename: Optional[str] = None,
        fullpath: Optional["os.PathLike[str]"] = None,
        tolerances: Optional[Dict[str, Dict[str, float]]] = None,
        default_tolerance: Optional[Dict[str, float]] = None,
        data_index: Optional[Sequence[int]] = None,
        fill_different_shape_with_nan: bool = True,
        *,
        file_format = NumericFileFormat.Csv,
    ) -> None:

Would be happy review a PR with that change if you want to contribute.

cc @tadeu @tarcisiofischer

12rambau commented 8 months ago

I'm happy to contribute, i just need to find the time to do so 😢 Whenver it will be rainy outside i'll have a look