Error using 4-band images

[AgustinNormand]

Hello!

I'm trying to train the models with a dataset that has 4 bands: Red, Green, Blue, Near-Infrared.

I've tried a .tiff file format and also a .png format. The training in both cases seems to work, the validation iterations return reasonable metrics. So I assume that the .pth file is completely valid, the error I get is when executing the test, it cannot draw the mask resulting from the prediction.

Tiff format training and testing notebook Png format training and testing notebook

The error I get is

Traceback (most recent call last):
  File "/content/open-cd/tools/test.py", line 114, in <module>
    main()
  File "/content/open-cd/tools/test.py", line 110, in main
    runner.test()
  File "/usr/local/lib/python3.10/dist-packages/mmengine/runner/runner.py", line 1791, in test
    metrics = self.test_loop.run()  # type: ignore
  File "/usr/local/lib/python3.10/dist-packages/mmengine/runner/loops.py", line 435, in run
    self.run_iter(idx, data_batch)
  File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/mmengine/runner/loops.py", line 456, in run_iter
    self.runner.call_hook(
  File "/usr/local/lib/python3.10/dist-packages/mmengine/runner/runner.py", line 1807, in call_hook
    getattr(hook, fn_name)(self, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/mmengine/hooks/hook.py", line 277, in after_test_iter
    self._after_iter(
  File "/content/open-cd/opencd/engine/hooks/visualization_hook.py", line 106, in _after_iter
    self._visualizer.add_datasample(
  File "/usr/local/lib/python3.10/dist-packages/mmengine/dist/utils.py", line 401, in wrapper
    return func(*args, **kwargs)
  File "/content/open-cd/opencd/visualization/cd_local_visualizer.py", line 112, in add_datasample
    pred_img_data = self._draw_sem_seg(pred_img_data,
  File "/usr/local/lib/python3.10/dist-packages/mmseg/visualization/local_visualizer.py", line 115, in _draw_sem_seg
    self.draw_binary_masks(
  File "/usr/local/lib/python3.10/dist-packages/mmengine/dist/utils.py", line 401, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/mmengine/visualization/visualizer.py", line 894, in draw_binary_masks
    assert len(color) == 3
AssertionError

Thank you very much in advance

[NicolasLeoneti]

Hello, I am having the same problem, I am interested in the solution!

Thanks

[likyoo]

Could you provide the config file in your work_dir? And this may be helpful for you.

[AgustinNormand]

Hello, thank you very much for the answer, here is the link to the folder where I store the files related to the notebook.

I'll check that link you sent me, thanks!

[likyoo]

Try:

palette=[[0, 0, 0], [255, 255, 255]]

[AgustinNormand]

I did a quick workout, just a couple of iterations, and it seems to have worked.

I am going to run a training with several iterations, and verify that the model is working correctly, generating the predictions of the labels, and so on. I'll confirm this tomorrow, and I'll close the issue if there aren't any errors. Thank you so much!

[AgustinNormand]

The question that arises to me is, using that palette of length 3, which I understand refers to the RGB bands, wouldn't it be making the model train ignoring the fourth band?

[NicolasLeoneti]

The question that arises to me is, using that palette of length 3, which I understand refers to the RGB bands, wouldn't it be making the model train ignoring the fourth band?

It worked for me too, but that's a good question.

[AgustinNormand]

The question that arises to me is, using that palette of length 3, which I understand refers to the RGB bands, wouldn't it be making the model train ignoring the fourth band?

Hello

I did a full workout with the 4 band data set and there is no performance difference with the 3 band data set.

Which makes me think that maybe the model is ignoring the fourth band, otherwise the performance should be modified even a bit.

Will it really be like that? Or does the palette have nothing to do with it?

Thank you!

[likyoo]

emmm.. I think the palette is just for visualization. plz make sure all bands are loaded. Check here, here, here, and so on.

[AgustinNormand]

You're right, I have to modify the models to specify that they should read the 4 bands,

If I manage to make it work, I'll put a link to the Colab notebook in case it helps someone else

Thank you

[NicolasLeoneti]

And what do I specify in the array of mean and std? Where do those values come from?

[AgustinNormand]

Hello, I am not able to find the part of the code that is reading the files only with 3 bands, avoiding the reading of the fourth one. Where do you read the images? Is it hardcoded that only 3 bands are read?

I noticed this because of the following: I modified several files, I added a made-up value in the mean array (because I don't know where it comes from either), also in the std array, and I set the bgr_to_rgb flag to false. I changed the input channels of the tinynet convolution.

After these changes I get the error:

Traceback (most recent call last):
  File "/home/agustin/Desktop/Change_Detection/Repositorios/fork-open-cd/tools/train.py", line 104, in <module>
    main()
  File "/home/agustin/Desktop/Change_Detection/Repositorios/fork-open-cd/tools/train.py", line 100, in main
    runner.train()
  File "/home/agustin/Desktop/Change_Detection/Repositorios/fork-open-cd/venv/lib/python3.10/site-packages/mmengine/runner/runner.py", line 1745, in train
    model = self.train_loop.run()  # type: ignore
  File "/home/agustin/Desktop/Change_Detection/Repositorios/fork-open-cd/venv/lib/python3.10/site-packages/mmengine/runner/loops.py", line 278, in run
    self.run_iter(data_batch)
  File "/home/agustin/Desktop/Change_Detection/Repositorios/fork-open-cd/venv/lib/python3.10/site-packages/mmengine/runner/loops.py", line 301, in run_iter
    outputs = self.runner.model.train_step(
  File "/home/agustin/Desktop/Change_Detection/Repositorios/fork-open-cd/venv/lib/python3.10/site-packages/mmengine/model/base_model/base_model.py", line 113, in train_step
    data = self.data_preprocessor(data, True)
  File "/home/agustin/Desktop/Change_Detection/Repositorios/fork-open-cd/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/agustin/Desktop/Change_Detection/Repositorios/fork-open-cd/opencd/models/data_preprocessor.py", line 222, in forward
    inputs = [(_input - self.mean) / self.std for _input in inputs]
  File "/home/agustin/Desktop/Change_Detection/Repositorios/fork-open-cd/opencd/models/data_preprocessor.py", line 222, in <listcomp>
    inputs = [(_input - self.mean) / self.std for _input in inputs]
RuntimeError: The size of tensor a (6) must match the size of tensor b (8) at non-singleton dimension 0

I added a print before the error to see the shape, and it is torch.Size([6, 256, 256])