wandb / wandb

The AI developer platform. Use Weights & Biases to train and fine-tune models, and manage models from experimentation to production.
https://wandb.ai
MIT License
9.19k stars 676 forks source link

[CLI]: MlflowImporter IGNORE step value in MlflowRun #5755

Open ykihong0 opened 1 year ago

ykihong0 commented 1 year ago

Describe the bug

We can log metrics in mlflow run with various (logging) step size. However, wandbify() (https://github.com/wandb/wandb/blob/c5fe3d56b155655980611d32ef09df35cd336872/wandb/apis/importers/mlflow.py#L38) in MlflowRun of MlflowImporter ignore these specific step value and overwrite its index of metric history as step.

  def metrics(self):
      def wandbify(metrics):
          for step, t in enumerate(metrics):
              d = {m.key: m.value for m in t}
              d["_step"] = step
              yield d

      metrics = [
          self.mlflow_client.get_metric_history(self.run.info.run_id, k)
          for k in self.run.data.metrics.keys()
      ]
      metrics = zip(*metrics)  # transpose
      return wandbify(metrics)
This is not a syntax error. There is no exception traceback

Additional Files

No response

Environment

WandB version: 0.15.4

OS: macOS Ventura 13.2.1

Python version: 3.8.12

Versions of relevant libraries: mlflow==2.3.0

Additional Context

No response

rsanandres-wandb commented 1 year ago

Hello @ykihong0 !

As mentioned, I will be making a bug report on this. The issues look related so thank you for reporting both issues!

ykihong0 commented 1 year ago

Hi @rsanandres-wandb ! Thanks for taking this issue! I hope this bug will be solved! :)