vavanade
commented
1 year ago On other images it seems to work fine but the masks are slightly shifted. Is that expected behavior?
Open vavanade opened 1 year ago
vavanade
commented
1 year ago On other images it seems to work fine but the masks are slightly shifted. Is that expected behavior?
no-Seaweed
commented
1 year ago On other images it seems to work fine but the masks are slightly shifted. Is that expected behavior?
Same here.
alfinata
commented
1 year ago On other images it seems to work fine but the masks are slightly shifted. Is that expected behavior?
Closer inspection suggest the problem still persists but small, maybe because of big image size. I already asked this but no response yet.
xuchengithub
commented
1 year ago this is bug i think
draw_binary_masks function have problems
# img,
# extent=(0, self.width, self.height, 0),
# interpolation='nearest')
endaoguansanlu
commented
1 year ago Checklist
- I have searched related issues but cannot get the expected help.
- The bug has not been fixed in the latest version.
Describe the bug
There seems to be a horizontal strip of pixels in the middle of the image where the pixels seem to just be copied from the rows above. Maybe that is why the segmentation mask has an offset to the image.
Reproduction
- What command or script did you run?
python demo/image_demo.py demo/demo.png configs/pspnet/pspnet_r50-d8_4xb2-40k_cityscapes-512x1024.py pspnet_r50-d8_512x1024_40k_cityscapes_20200605_003338-2966598c.pth --device cuda:0 --out-file result.jpgEnvironment
sys.platform: linux Python: 3.10.6 (main, Mar 10 2023, 10:55:28) [GCC 11.3.0] CUDA available: True numpy_random_seed: 2147483648 GPU 0: NVIDIA GeForce RTX 3090 CUDA_HOME: None GCC: x86_64-linux-gnu-gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0 PyTorch: 2.0.1+cu117 PyTorch compiling details: PyTorch built with: - GCC 9.3 - C++ Version: 201703 - Intel(R) oneAPI Math Kernel Library Version 2022.2-Product Build 20220804 for Intel(R) 64 architecture applications - Intel(R) MKL-DNN v2.7.3 (Git Hash 6dbeffbae1f23cbbeae17adb7b5b13f1f37c080e) - OpenMP 201511 (a.k.a. OpenMP 4.5) - LAPACK is enabled (usually provided by MKL) - NNPACK is enabled - CPU capability usage: AVX2 - CUDA Runtime 11.7 - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86 - CuDNN 8.5 - Magma 2.6.1 - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.7, CUDNN_VERSION=8.5.0, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=0 -fabi-version=11 -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wunused-local-typedefs -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_DISABLE_GPU_ASSERTS=ON, TORCH_VERSION=2.0.1, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=1, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, TorchVision: 0.15.2+cu117 OpenCV: 4.7.0 MMEngine: 0.7.3 MMSegmentation: 1.0.0+e64548fAlso tried it with PyTorch 1.13.1 and a different model (seg2former) with the same result.
hey guy! Did you solve this problem?
Solexist
commented
1 year ago Exactly same bug. I have located the problem. The visualization works in this way: draw each labeled area with specified color onto current image, repeat it for the number of labels times.
However, during each drawing, the up half of the image is stretched for 1 pixel. The right half of the image is also stretched for 1 pixel. Since different areas was drawn one by one, they separate by different distances, and the original image was stretched by nearly 20 pixels.
Here is how the output looks like after the first area was drawn:
This bug may be due to a mismatch of versions of mmengine and matplotlib, but I found that there is trick in mmengine/visualization/visualizer.py/Visualizer.set_image(self, image)
@master_only
def set_image(self, image: np.ndarray) -> None:
"""Set the image to draw.
Args:
image (np.ndarray): The image to draw.
"""
assert image is not None
image = image.astype('uint8')
self._image = image
self.width, self.height = image.shape[1], image.shape[0]
self._default_font_size = max(
np.sqrt(self.height * self.width) // 90, 10)
# add a small 1e-2 to avoid precision lost due to matplotlib's
# truncation (https://github.com/matplotlib/matplotlib/issues/15363)
self.fig_save.set_size_inches( # type: ignore
(self.width + 1e-2) / self.dpi, (self.height + 1e-2) / self.dpi)
# self.canvas = mpl.backends.backend_cairo.FigureCanvasCairo(fig)
self.ax_save.cla()
self.ax_save.axis(False)
self.ax_save.imshow(
image,
extent=(0, self.width, self.height, 0),
interpolation='none')An issue was mentioned (https://github.com/matplotlib/matplotlib/issues/15363), but it seems unsolved, so this trick should work. However, if I simply remove the trick by deleting 1e-2, the visualization becomes normal:
Could somebody check if this trick is already outdated and unnecessary or not? I followed exactly the steps to install mmsegmentation, I got same bug by both installing it from source and running it by dockerfile.
Solexist
commented
1 year ago On other images it seems to work fine but the masks are slightly shifted. Is that expected behavior?
As my last reply, the 1-pixel shift is made for each area drawn, so with fewer labeled areas you got smaller shifts. This is dangerous becouse the bug may be hard to notice if you only label a few areas.
Solexist
commented
1 year ago I found another interesting thing. On the official user guide webpage, the output of the visualization has the same bug, which means that we are just under the same condition as the developers, so relax.
CJCHEN1230
commented
10 months ago @Solexist Hi, I encountered the same problem. I'm really curious that seldom people talked about that. It seems a big problem. I remove 1e-2 like you said. But it's still same. Could you please tell me the solution more precise?
mkutu
commented
7 months ago Wondering if anyone has looked into this? @MengzhangLI @xiexinch @MeowZheng? It seems like this is still a pretty big issue as inferencing results are still shifted. Any thoughts?
Checklist
Describe the bug
There seems to be a horizontal strip of pixels in the middle of the image where the pixels seem to just be copied from the rows above. Maybe that is why the segmentation mask has an offset to the image.
Reproduction
What command or script did you run?
Environment
Also tried it with PyTorch 1.13.1 and a different model (seg2former) with the same result.