search

Neos-Metaverse / NeosPublic

A public issue/wiki only repository for the NeosVR project
193 stars 9 forks source link

Add support for MPO stereo images #86

Open wel97459 opened 6 years ago

wel97459 commented 6 years ago

This is a common stereo image format, would be nice to have it available for import.

It is a jpeg format that has both left and right pairs in it.

Frooxius commented 6 years ago

Hmm, can you send me some example files? I'll have to check if the library I'm using for image decoding supports there, otherwise it might be potentially a bit problematic, but I'll see.

wel97459 commented 6 years ago

Here is a MPO file to investigate with. P1030933.MPO

Frooxius commented 6 years ago

Thanks! I've looked more into it, it looks like it's just a bunch of JPG files concatenated after each other. I can write simple custom code to decode all of them using the FreeImage library.

Question is, how should this be handled in Neos? Should they import as separate images and then use shader/component to render each only for one eye?

Or should they import as an image strip side-by-side? I haven't also found any info so far on their spatial arrangement, which one is left and which one is right. I'll look into it some more later.

wel97459 commented 6 years ago

It looks like the format is left image first, right second. It should be loaded and use with a shader/component to render left image for left eye, and right image for right eye. Just like how you can load a side-by-side JPG left right 3d.

I'm not sure there is a use case for having it just load the images, since the user can take the images out of the MPO and split them before trying to import them into Neos.

Frooxius commented 6 years ago

Hmm it seems to be the case of this image, but I found a spec: http://www.cipa.jp/std/documents/e/DC-007_E.pdf

It actually defines an extension to the JPG metadata, which contains some information. There can be more than two JPG images and could possibly be arranged in different ways (it defines single dimension shot and multi dimensional shots).

I'm not sure I could read these headers without writing a custom reader (just for the headers). Also question is, how to handle multi-dimensional images too.

Frooxius commented 6 years ago

Found another example, this one seems to have it reverse, the first image is left, the second is right. While in yours the first seems to be the right eye. https://dmitrybrant.com/2011/02/08/the-fujifilm-mpo-3d-photo-format

Psychpsyo commented 3 years ago

If this gets added, the option to output stereo images as MPO would also be nice since I do that every now and then and currently have to use an external tool to convert the left/right image that Neos outputs to an MPO.