search

evandez / neuron-descriptions

Natural Language Descriptions of Deep Visual Features, ICLR 2022
http://milan.csail.mit.edu
MIT License
62 stars 7 forks source link

Imagenet source images are always black #4

Closed JacksonWuxs closed 2 years ago

JacksonWuxs commented 2 years ago

Hi Evan,

Thank you so much for sharing the dataset! When I try to use the code milannotations.load to load the Imagenet dataset, I found some results don't follow our expectations. I think I have setup the environment correctly, but I still cannot load images from the Imagenet with a correct RGB format.

Here is the code I used to check:

import os
from src import milannotations

os.environ["MILAN_DATA_DIR"] = "../data"

def show_loaded_image(name):
    data = milannotations.load(name)
    print("index=0", data[0].images.reshape((-1, 3, 224 * 224)).max(axis=-1).values)
    print("index=1", data[1].images.reshape((-1, 3, 224 * 224)).max(axis=-1).values)
    print("index=2", data[2].images.reshape((-1, 3, 224 * 224)).max(axis=-1).values)
    print("index=-1", data[-1].images.reshape((-1, 3, 224 * 224)).max(axis=-1).values)

if __name__ == "__main__":
    show_loaded_image("alexnet/imagenet")
    show_loaded_image("resnet152/imagenet")

    show_loaded_image("alexnet/places365")
    show_loaded_image("resnet152/places365")

And here is the output coming from the test code:

load alexnet/imagenet: 100%|██████████████████████████████████████████████████████████████| 5/5 [00:04<00:00,  1.22it/s]
index=0 tensor([[0.0032, 0.0038, 0.0032],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0039],
        [0.0014, 0.0039, 0.0039],
        [0.0039, 0.0038, 0.0038],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0038],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0039]])
index=1 tensor([[0.0039, 0.0039, 0.0039],
        [0.0038, 0.0037, 0.0039],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0038, 0.0038],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0039]])
index=2 tensor([[0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0038, 0.0038],
        [0.0039, 0.0039, 0.0039],
        [0.0038, 0.0036, 0.0035],
        [0.0039, 0.0038, 0.0038],
        [0.0038, 0.0039, 0.0038],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0039],
        [0.0036, 0.0036, 0.0038],
        [0.0037, 0.0034, 0.0032],
        [0.0038, 0.0038, 0.0039],
        [0.0038, 0.0039, 0.0039],
        [0.0038, 0.0038, 0.0039]])
index=-1 tensor([[0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0039],
        [0.0037, 0.0039, 0.0037],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0039],
        [0.0037, 0.0037, 0.0039],
        [0.0039, 0.0039, 0.0039],
        [0.0038, 0.0035, 0.0039],
        [0.0038, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0039],
        [0.0038, 0.0038, 0.0038]])

load resnet152/imagenet: 100%|████████████████████████████████████████████████████████████| 5/5 [00:31<00:00,  6.26s/it]
index=0 tensor([[0.0039, 0.0039, 0.0039],
        [0.0032, 0.0033, 0.0034],
        [0.0037, 0.0037, 0.0036],
        [0.0031, 0.0037, 0.0039],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0038],
        [0.0039, 0.0038, 0.0034],
        [0.0037, 0.0032, 0.0035],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0038, 0.0038],
        [0.0038, 0.0036, 0.0029],
        [0.0039, 0.0039, 0.0039]])
index=1 tensor([[0.0028, 0.0028, 0.0031],
        [0.0039, 0.0039, 0.0039],
        [0.0038, 0.0038, 0.0036],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0037, 0.0034],
        [0.0034, 0.0033, 0.0033],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0038],
        [0.0039, 0.0038, 0.0034],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0039]])
index=2 tensor([[0.0039, 0.0039, 0.0039],
        [0.0039, 0.0036, 0.0035],
        [0.0039, 0.0039, 0.0035],
        [0.0039, 0.0039, 0.0039],
        [0.0034, 0.0037, 0.0038],
        [0.0037, 0.0037, 0.0036],
        [0.0033, 0.0032, 0.0039],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0038, 0.0038],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0035],
        [0.0032, 0.0030, 0.0028],
        [0.0039, 0.0039, 0.0039]])
index=-1 tensor([[0.0033, 0.0034, 0.0034],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0038],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0039],
        [0.0035, 0.0036, 0.0037],
        [0.0039, 0.0038, 0.0038],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0039, 0.0039],
        [0.0039, 0.0038, 0.0034],
        [0.0039, 0.0039, 0.0039]])

load alexnet/places365: 100%|█████████████████████████████████████████████████████████████| 5/5 [00:05<00:00,  1.02s/it]
index=0 tensor([[1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [0.8980, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000]])
index=1 tensor([[1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 0.9490, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 0.9725, 0.8353],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 0.9608, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000]])
index=2 tensor([[1.0000, 1.0000, 1.0000],
        [1.0000, 0.9608, 1.0000],
        [1.0000, 0.9882, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000]])
index=-1 tensor([[1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [0.8980, 0.9098, 0.9686],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 0.9961, 0.9922],
        [1.0000, 1.0000, 1.0000]])

load resnet152/places365: 100%|███████████████████████████████████████████████████████████| 5/5 [00:18<00:00,  3.80s/it]
index=0 tensor([[1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 0.9765, 1.0000],
        [0.9843, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000]])
index=1 tensor([[1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 0.9961, 0.9451],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [0.9843, 1.0000, 1.0000],
        [1.0000, 1.0000, 0.9843],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 0.9059, 0.9412]])
index=2 tensor([[1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 0.9569, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000]])
index=-1 tensor([[0.8863, 0.8902, 0.7961],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 0.9843],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [0.8196, 0.8667, 0.8196],
        [1.0000, 1.0000, 0.9804],
        [1.0000, 1.0000, 0.9922],
        [1.0000, 0.9765, 0.9765],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 0.9647],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000],
        [1.0000, 1.0000, 1.0000]])

Since the code doesn't raise any issue, I simply assume that my environment is setup correctly. But we can see that the maximum value of each image coming from Imagenet dataset is roughly 0.0039, which is less than 1 in the 255-level RGB scale. In other words, all source images of Imagenet are actually black. Meanwhile, the images of places365 are correct because the maximum values of places365 images are roughly 1.0.

Did I miss anything? May I know how to fix this issue?

Thanks a lot!

evandez commented 2 years ago

Ah, I think when it merges in the ImageNet images, it does not properly convert them from [0, 255] range to [0, 1] range (or the other way). I will implement a fix for this soon, sorry for the delay.

evandez commented 2 years ago

Okay, I think I've pushed a fix for this! Still have to thoroughly test, but you can pull and try again if you wish.

JacksonWuxs commented 2 years ago

Sorry to the late response. I am very appreciate to your fixed push! I will try it!

Thanks!