search

haruishi43 / equilib

🌎→🗾Equirectangular (360/panoramic) image processing library for Python with minimal dependencies only using Numpy and PyTorch
Apache License 2.0
163 stars 21 forks source link

Artifacts when converting high resolution cubemap to low resolution equirectangular #24

Open Yukun-Huang opened 17 hours ago

Yukun-Huang commented 17 hours ago

Hello! I try to convert a high resolution cubemap (1024x6144x3) to a low resolution equirectangular (e.g., 512x1024x3) using equilib.cube2equi, but severe artifacts appeared. Does anyone know how to fix this?

Cubemap (1024x6144x3) -> Equirectangular (512x1024x3): equi_512

Cubemap (1024x6144x3) -> Equirectangular (2048x4096x3): equi_2048

haruishi43 commented 15 hours ago

@Yukun-Huang thanks for raising the issue. For the low resolution conversion, what interpolation did you use (I see you tried to use NN #22 )? NN interpolation results in even more artifacts for low resolution conversion (the bilinear interpolation seems to be the best I can do for now... but PRs are always welcome!). If you need a "high-quality" low-resolution equirectangular image, I would recommend just down-sampling the high-resolution equirectangular image you created.

Yukun-Huang commented 9 hours ago

@haruishi43 Thanks for your great equilib! I am using the default bilinear interpolation. Yes, down-sampling the high-resolution equirectangular image is indeed a feasible solution, but it is not elegant lol.

In fact, I know that these artifacts (moiré pattern) are often encountered when downsampling high-resolution images to low-resolution images. In OpenCV, I can usually use INTER_AREA interpolation to get the best moiré-free results. What I really want to ask is, why do panoramic image processing libraries, such as equilib and py360convert, usually only support nearest and bilinear interpolation? Would it be very difficult to add other interpolation methods like INTER_AREA?

haruishi43 commented 46 minutes ago

@Yukun-Huang I do support other interpolations in the other transformation (e.g. equi2pers), but they are not tailored to heavy downsampling. As I said in #22, NN interpolation does not work well at all, so bilinear is the best I can do for now. You can override the current grid sampling with others (cv2, scipy, etc...) if you want. Or you can extend the current bilinear sampling by changing the coefficients so that it is close to the cv2's INTER_AREA interpolation since INTER_AREA seems to be bilinear interpolation with different coefficients.