search

iterative / vscode-dvc

Machine learning experiment tracking and data versioning with DVC extension for VS Code
https://marketplace.visualstudio.com/items?itemName=Iterative.dvc
Apache License 2.0
197 stars 29 forks source link

Custom Plots to support Metric vs Metric or Param vs Param or Metric vs Param #5216

Open daveraghav opened 10 months ago

daveraghav commented 10 months ago

Currently the custom plots only support param vs metric. It would be great if the custom plots can support metric vs metric, param vs param, or even metric vs param.

shcheklein commented 8 months ago

@daveraghav could you please give a bit more context? how do you use extension, how critical is this feature for you? (It would be great to jump on a call for me to learn more about you scenarios and cases).

daveraghav commented 8 months ago

Thanks @shcheklein for looking into this. Here is the context: for some of the training algorithms I'm running, that do self-optimisation, for example, automatic feature selection, or instances where I could be running hyperparameter optimisation as part of the training script, the parameters like number of features, or specific algorithm parameters are often part of output of one of the stages. so they're not really the metrics itself but params. so in these cases I would like to plot those outs (metrics as per dvc, but params as per my training process), vs the actual accuracy metrics. In these cases I've had to pre-define the plots, as part of the pipeline. It would be handy to allow plotting as part of custom plots through the extension directly. I hope that makes sense. Please let me know if further context is needed. Happy to jump on a call.

daveraghav commented 6 months ago

Was there any update to this issue?

shcheklein commented 6 months ago

@daveraghav sorry for the radio-silence here. I'll check with the team if we have capacity to take this. I've DMed you on LI - let's jump on call and discuss the details - it can help a lot.