Logged curves are not showing

[marixko]

Hello,

As you can see in the image below, I have several curves under a particular experiment. However, none of them are being shown in Logged Curves.

For instance, z_sigma looks like this after downloading the file from comet.ml:

{"x": [0.25, 0.75, 1.25, 1.75, 2.25, 2.75, 3.25, 3.75, 4.25, 4.75], "y": [0.3497, 0.2354, 0.1604, 0.1545, 0.1135, 0.1316, 0.1072, 0.164, 0.184, 0.1609], "name": "z_sigma"}

I made a new test experiment, for which I only log this curve:

  x = [0,1,2,3,4,5,6]
  y = [0,1,2,3,4,5,6]
  experiment.log_curve("test", x=x , y=y )

And for this test experiment, logged curve is working fine:

Then I copied and pasted these same 3 lines of code into my original experiment and still no curves are being shown. I also tried setting "curveName" : "test" and nothing happens:

Any idea on how to solve this?

comet_ml version: 3.15.4 python version: 3.8.5

[dsblank]

@marixko Thanks for the report! Looking into this...

[dsblank]

I was able to create a logged curve graph (see video below). I notice that your Options are slightly different from the Featured "Logged Curve" Panel. Is this a copy, or one that you have edited yourself?

My experiment: https://www.comet.ml/dsblank/logged-curve/fb1e23d54fb049b485dbb3f19a303c35?experiment-tab=panels&viewId=L2cKgdHYLiYTLcLOkYk8xrxjU

[marixko]

Thanks for your quick answer! I was able to create a curve graph the same way you have done in your video. The issue happens in another scenario. I tried to think what would be the differences between the two scenarios, and I could only think on those:

• log_curve are being called in a for loop
• a total of 14 curves are being logged, instead of one

About the Options in the logged curve panel, I only edited the "curveName" line by following this example.

I just created this project with one working example so I can share it here: https://www.comet.ml/marixko/testing-logged-curve/view/1ISC6NJi5bVI9bqTskFfvul4l I added the logged curve panel into this project with no edits at all. The "test" curve under assets is the one that was working fine in another experiment that I ran that log_curve outside the loop (see plot from my original post).

[dsblank]

Ok, I see. Yes, each Panel is written individually, and the two that you were looking at ("Logged Curve", and "Plotting Curve with Step") have some differences on how they deal with multiple experiments, and multiple curves per experiment.

BUT, great news: we just released creating Panels in Python and it makes this super easy. Here is the code for selecting experiments, and then selecting the asset:

from comet_ml import API, ui
import matplotlib.pyplot as plt

api = API()

experiment = ui.dropdown('Select experiment:', api.get_panel_experiments())
assets = experiment.get_asset_list("curve")
asset_json = ui.dropdown("Select a curve:", assets, format_func=lambda item: item["fileName"])
curve = experiment.get_asset(asset_json["assetId"], return_type="json")
plt.plot(curve["x"], curve["y"])
ui.display(plt)

Just paste this into a new Python Panel and save it. Let us know if this would work for you.

[dsblank]

More details: https://www.comet.ml/docs/user-interface/python-panels/overview/

[marixko]

Very cool feature, thanks! I was able to plot a single curve for a single experiment using the code you provided, although that is not really useful for me. The kind of plot that I need is similar to the metrics plotted per steps/wall/duration/epoch but with a custom x-axis. The motivation is: I need to evaluate the performance of my models per bins of my target value because some models will perform way better/worse at specific bins. So what I want to plot in the x-axis is the center of each target bin. I even tried to log the curves using log_metric and specifying a custom x-axis through "step". It worked fine until I realized that log_metric transforms anything that goes in "step" to integer type. It would be amazing to have the possibility of logging this through log_metric. In any case, is there any way to generalize that piece of python code for plotting one curve (with a specific file name) for multiple experiments in the same plot?

[dsblank]

Ok, thanks for the detailed information. Two points:

1. We are working allowing floating-point steps, which would allow log_metric() to work. It is in development.
2. You can do anything that you want with Custom Panels :)

Let me see if I can put together an example Python Panel ...

[dsblank]

Here is a slight variation of the above:

The logic is:

1. Go through all experiments and get all of the curve names
2. Allow user to select which curve name they want to see across all experiments that have it
3. Go through all experiments with that curve name and plot them

You can change the center of the bins of course, and do any matplotlib manipulation (eg, legends) that you wish.

Here is the source, and link to Project:

from comet_ml import API, ui
import matplotlib.pyplot as plt
from collections import defaultdict

api = API()

# first gather all of the filenames of curves:
file_names = defaultdict(int)
for experiment in api.get_panel_experiments():
    assets = experiment.get_asset_list("curve")
    for asset_json in assets:
        file_names[asset_json["fileName"]] += 1

file_name = ui.dropdown(
    'Select curve:', 
    sorted(file_names, key=lambda key: file_names[key], reverse=True), 
    format_func=lambda key: "%s (%s matches)" % (key, file_names[key]))

for experiment in api.get_panel_experiments():
    assets = experiment.get_asset_list("curve")
    for asset_json in assets:
        if asset_json["fileName"] == file_name:
            curve = experiment.get_asset(
                asset_json["assetId"], return_type="json")
            plt.plot(curve["x"], curve["y"], label=experiment.name)

plt.legend()
ui.display(plt)

Project: https://www.comet.ml/dsblank/logged-curve/view/LEjHLaYWTLePG9Wg6m40BSY6t

[marixko]

Thank you!!

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[github-actions[bot]]

This issue was closed because it has been stalled for 5 days with no activity.