search

iterative / dvc

🦉 ML Experiments and Data Management with Git
https://dvc.org
Apache License 2.0
13.42k stars 1.17k forks source link

dvc metrics/plots diff local files #7993

Closed shortcipher3 closed 4 months ago

shortcipher3 commented 2 years ago

You can use git diff to compare to local files.

git diff [<options>] --no-index [--] <path> <path>

It would be nice to have the same functionality with dvc, for example I have a model and then a quantized model, since there are many ways to quantize a model I would like to have stages that take the original model and create the quantized model - so they belong in the same commit.

For reporting purposes and understanding I would like to show the change in the metrics/plots between the original model and the quantized model.

karajan1001 commented 2 years ago

Also for model distillation.

pared commented 2 years ago

@shortcipher3 regarding the plots it should be already available in latest release:

#dvc.yaml
stages: # <- note that plots is on the same level as stages
  ....
plots:
  quantized_vs_no_quantized:
    x: x_field # needs to be present in both files
    y:
      quantized_data_file.json: metric_name
      non_quantized_data_file.csv: metric_column
    template: linear
    title: "Comparing quantized vs non quantized"
dberenbaum commented 2 years ago

There's some background discussion on this in https://github.com/iterative/dvc/issues/5693.