simple inference ?

[MohamedAliRashad]

I want to make simple inference for some images I have and the results are bad

from models.GCoNet import GCoNet
import torch
from pathlib import Path
from torchvision.io import read_image
from torchvision.utils import make_grid, save_image
from torchvision import transforms

dataset_path = Path("/path/to/data")

device = torch.device("cuda")
model = GCoNet()
model = model.to(device)
gconet_dict = torch.load("/path/to/weights")
# gconet_dict = torch.load("/shared_volume/rashad/salient_app/ultimate.pth")
trans = transforms.Compose([
            transforms.Resize((320, 320)),
            # transforms.ToTensor(),
            transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]),
        ])

model.to(device)
model.load_state_dict(gconet_dict)

model.eval()
images = [read_image(str(img_path))[None, ...] for img_path in dataset_path.glob("11/*")]
# print(images)
torch_images = torch.concat(images).float()
torch_images = trans(torch_images)
print(torch_images.shape)
with torch.no_grad():
    result = model(torch_images.cuda())[-1]
    result = torch.concat([torch.nn.functional.interpolate(res.unsqueeze(0), size=(res.shape[1], res.shape[2]), mode='bilinear', align_corners=True) for res in result]).sigmoid()
result = (result > result.mean()).float() * 255
print(result.shape)
print(result.max(), result.min())
save_image(make_grid(result), "salient_example.png")

And this is the result,

@HzFu @ZhengPeng7

[ZhengPeng7]

Hi Mohamed, can you provide the raw images for me? I'll use your inference codes here to make a test on images of both CoCA and your images. @MohamedAliRashad

[MohamedAliRashad]

They are standard JPG images, this is an example @ZhengPeng7

[ZhengPeng7]

Yeah, but I mean... can you give me some group of images? Since it's co-sod, the number of images in one group does affect the predictions.

[MohamedAliRashad]

They are very simple and intuitive images to salient through them, @ZhengPeng7

[ZhengPeng7]

Hi Mohamed, I made my inferene code on the colab online with the ultimate.pth, including one group of CoCA (accordion) and the group you gave (kettle). Here are the screenshots of the colab.

I still have some other urgent task nearly finished but not yet. So I will polish up my colab tomorrow for anyone to do the inference by our model online. Welcome to visit it.

By the way, if you want to use our model for general co-sod on images outside the available datasets, we recommend you to use the ultimate_duts_cocoseg (The best one).pth here, since it is trained by two datasets.

Maybe there is something mistaken in your inference codes, you can give it a double check or just wait for my updates on colab :)

[MohamedAliRashad]

I think the problem is in the preprocessing, but I am not sure where. If you could add the colab notebook I will be thankful @ZhengPeng7

[ZhengPeng7]

Hi, Mohamed, Sorry that I forgot this for days...😂 I just polish the processing of my codes in colab.

You can have a try on the colab-demo now.

I've checked it, the predicted results are exactly the same as what I show in README and uploaded to google-drive.

Hope this help you. And any new question, just comment.

[MohamedAliRashad]

@ZhengPeng7 To come late is better than not coming at all. Thanks for your efforts, I will close the issue now.