[BUG] Can't access alternate media fields in patches views?

oguz-hanoglu commented 2 years ago

System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 20
FiftyOne installed from (pip or source): pip
FiftyOne version (run fiftyone --version): v0.17.2
Python version: 3.10.4

Describe the problem

When moved to patches_view, media_field buttons result in "This image failed to load. The file may not exist, or its type (image/png) may be unsupported"

Code to reproduce issue

my_dataset = fo.Dataset("test")
samples = []
for fod in tqdm(df.itertuples()):
    sample = fo.Sample(filepath= ...)
    sample["t_1"] = ...
    sample["t_2"] = ...
    sample["det"] = fo.Detections(
        detections=[fo.Detection(bounding_box=[top_left_x, top_left_y, width, height])])
    samples.append(sample)
my_dataset.add_samples(samples)
my_dataset.app_config.media_fields = ["filepath", "t_1", "t_2"]
session = fo.launch_app(my_dataset, port=1851, auto=False)
session.view = my_dataset.to_patches("det")

Other info / logs

Problem disappears when session.view = my_dataset.to_patches("det", other_fields=["t_1", "t_2"]) is used.

I guess, to_patches, by default, has to consider my_dataset.app_config.media_fields list as if it is already appended to other_fields.

What areas of FiftyOne does this bug affect?

[ ] App: FiftyOne application issue
[x] Core: Core fiftyone Python library issue
[ ] Server: Fiftyone server issue

Willingness to contribute

The FiftyOne Community encourages bug fix contributions. Would you or another member of your organization be willing to contribute a fix for this bug to the FiftyOne codebase?

[ ] Yes. I can contribute a fix for this bug independently.
[ ] Yes. I would be willing to contribute a fix for this bug with guidance from the FiftyOne community.
[x] No. I cannot contribute a bug fix at this time.

oguz-hanoglu commented 2 years ago

issue_media_field

brimoor commented 2 years ago

If you'd like your alternate media fields to be available in a patches view, you just need to include them in the other_fields argument of to_patches():

import fiftyone as fo
import fiftyone.zoo as foz
import fiftyone.utils.image as foui

dataset = foz.load_zoo_dataset("quickstart")

foui.transform_images(
    dataset,
    max_size=(-1, 64),
    output_field="thumbnail_path",
    output_dir="/tmp/thumbnails",
)

dataset.app_config.media_fields = ["filepath", "thumbnail_path"]
dataset.app_config.grid_media_field = "thumbnail_path"
dataset.save()

patches = dataset.to_patches("ground_truth", other_fields=["thumbnail_path"])

session = fo.launch_app(patches)

oguz-hanoglu commented 2 years ago

OK. I already explained this solution under "Other info / logs" parts, sorry if my point was not clear. My main point is it would be better if the default behavior already includes it. But for the thumbnail case, this would not be meaningful. Then my point becomes updating the media field in the app. Why would the media field show "thumbnail" if it is not available in patches view? (if only it is included in the other field, then displaying thumbnail under media field becomes useful) But this would also be non applicable for some reason. Then, my point becomes the error "This image failed to load. The file may not exist, or its type (image/png) may be unsupported" is misleading, at least for an average user, my humble opinion.

brimoor commented 2 years ago

That's a fair point. We should either include all media fields by default or remove the extra media fields from the selector unless they were specifically included.

quitmeyer commented 2 weeks ago

patches = dataset.to_patches("ground_truth", other_fields=["thumbnail_path"])
@brimoor Hey wanted to thank you for posting this. I was going nuts trying to figure out how to maintain the thumbnail view in patch view. I am having a really hard time with the performance of fiftyone, and this will help i hope

voxel51 / fiftyone