iterative / dvclive

📈 Log and track ML metrics, parameters, models with Git and/or DVC
https://dvc.org/doc/dvclive
Apache License 2.0
161 stars 35 forks source link

`log_image` fails to save an image if the path was provided, but the `name` does not have an extension #743

Closed natikgadzhi closed 7 months ago

natikgadzhi commented 9 months ago

In an example like this:

import time
import random

from dvclive import Live

params = {"learning_rate": 0.002, "optimizer": "Adam", "epochs": 20}

with Live() as live:
    # log a parameters
    for param in params:
        live.log_param(param, params[param])

    # simulate training
    offset = random.uniform(0.02, 0.1)
    for epoch in range(1, params["epochs"]):
        fuzz = random.uniform(0.01, 0.1)
        accuracy = 1 - (2**-epoch) - fuzz - offset
        loss = (2**-epoch) + fuzz + offset

        # ... 

        # here's the critical piece: 
        live.log_image("image", "/Users/nategadzhi/Downloads/Untitled.jpg")
        live.next_step()
        time.sleep(0.2)

when live.log_image is trying to save an image, we provide:

  1. DVCLive will try and open the image with PIL
  2. DVCLive then makes dvc_live.Image object with the provided name. Notice that the name is not a valid filename, it does not have an extension.
  3. Then, Image will try to dump itself, but it's output_path does not have an extension either, because it's basically output_dir/subdir/name.
  4. PIL fails to save the image with ValueError: unknown file extension:

Solutions

Document and provide a better error

We can document this behavior, make it obvious that name will become the filename of the saved image, and that it must be a valid filename with an extension. And we can fail earlier with a nicer error message.

Try to infer the extension from the image

Alternatively, we can infer the format of the image from the image itself. If the name does not have an extension, we can apply f"{name}/{val.format}".

I think inferring the type from the image file is a better idea, but I'm not an expert in ML context specifically, and how folks are used to working with their data.

Happy to help with a fix if you point me in the right direction! /cc @dberenbaum @shcheklein

dberenbaum commented 9 months ago

Thanks @natikgadzhi! I would be fine with either of those proposed solutions. If you are interested in fixing it yourself, there are existing suffixes against which you can check the file extension. If you go with option 1, you can include the check in the could_log method. If you go with option 2, you would have to add some logic to log_image to modify the name arg.