search

NVlabs / instant-ngp

Instant neural graphics primitives: lightning fast NeRF and more
https://nvlabs.github.io/instant-ngp
Other
16.03k stars 1.93k forks source link

JPEG images from DJI work fine in colormap2nerf.py. Fail loading in nerf_loader.cu #731

Open bgyarfas opened 2 years ago

bgyarfas commented 2 years ago

I have JPEGs for a scan I took with a DJI Mini SE drone. colormap2nerf.py deals with them fine but they fail to load in the testbed (errors out with a "image not found: ").

After some digging, the error isn't that the image isn't found but it's failing to load with STB. Diving deeper, it appears the DJI JPEGs have a JPEG Marker 0 that STB does not like (see STB Issue #1182). It doesn't appear that STB intend to ignore file issues like this so it likely won't get fixed upstream. If I get a chance I'll open a PR to add an error message after stbi_load(... if the file fails to load due to "file errors" as the "image not found: " message can be misleading.

I was able to work around the issue by first using the original files to run through colormap2nerf.py (since it appears to leverage the GPS coordinates in the metadata). Then I run the files through convert_image.py and output the same file name. Which technically will re-encode them which isn't ideal but until I figure out how to fix the files in-place it was the quickest solution I could come up with.

Thanks for the great work!

bgyarfas commented 2 years ago

Continued digging (mainly by seeing how Pillow deals with the files just fine)... It appears the files are actually "Multi-Picture Format" (link) meaning they have an embedded thumbnail that is described by a special metadata block. STB doesn't know how to handle this type of file correctly so it fails.

bgyarfas commented 1 year ago

Here is a quick gist to load the files with Pillow then resave them. This strips the offending JPEG marker block.