search

spite / android-lens-blur-depth-extractor

MIT License
292 stars 34 forks source link

Support for Seene variation of the same? #3

Closed neopaf closed 8 years ago

neopaf commented 8 years ago

(was referred to this project by Sketchfab support team)

I've tried to import some of my Seene-app (iOS) export format, which Creators claim is same as Google Lens Blur format. But failed.

Please quick-look is the format is totally broken = unsupportable. Maybe there is something simple to be changed = fixed?

Attaching some examples. img_4027 img_4026 img_4025

neopaf commented 8 years ago

@zettlerm, hello

spite commented 8 years ago

It seems like they've stripped the GDepth:Data attribute from the file. Other attributes are there, but the actual depth is nowhere to be found.

Here's a snapshot of an hex view of one of the files provided. You'll see there's GDepth:Format/Near/Far but no Data:

screen shot 2016-07-28 at 00 42 24

Who have you contacted from Seene that told you about compatibility? Could you ask them about this issue?

neopaf commented 8 years ago

I will. I'm a developer myself, but not very deep into .JPEG format or ways to hide XMP there.

But I see some 2 (two!) XMP sections there, and one of them seem to have the data. Used this tool to extract it: http://www.sno.phy.queensu.ca/~phil/exiftool/

Attached: 35063c38-545a-11e6-9441-b069def5d258-exif.txt

I did try to convert that BASE64 and it seems to have what's needed: 35063c38-545a-11e6-9441-b069def5d258-depth

What would you say? Maybe data got into second XMP and it is still possible to extract & use it?

seene-club commented 8 years ago

Hello,

verified with Seene-Toolkit that XMP is there. You'll find the depthmap in the section GDepth:Data

screenhunter_271 jul 28 12 17 screenhunter_272 jul 28 12 19

hope that information is helpful! cheers, Mathias

spite commented 8 years ago

Ah, Ok. Couple of things here...

spite commented 8 years ago

Ok, fixed. The lib is updated -make sure you check index.html on how to load the color buffer now-.

The tool https://www.clicktorelease.com/tools/lens-blur-depth-extractor/ and the player https://www.clicktorelease.com/code/depth-player/ are updated too.

Thanks!

seene-club commented 8 years ago

The Seene format loads now! Great work, Jaume. Thank you very much!

But one more thing. There are so less points in the point cloud (wireframe too). Your examples look much better. Tried the sliders on the right, but did not got more depth points. Any idea?

screenhunter_273 jul 28 14 35 screenhunter_274 jul 28 14 35 screenhunter_275 jul 28 14 36 screenhunter_276 jul 28 14 44

spite commented 8 years ago

Yes, the mesh size/tessellation is based on the image size. Lens Blur images are usually pretty large, but the seene ones seem to report to be 240x240, so the mesh is much smaller. Use the Quad Side parameter on the viewer, set it to 1, and you'll have more quads. Changing the filter might help, too.

spite commented 8 years ago

Now I've also realised that 240x240 is the reported size of the Depth Map. Will fix, too.

spite commented 8 years ago

Fixed.

seene-club commented 8 years ago

Awesome, Jaume. Seenes look great now with your viewer. You made my day! Thank you very much for the fantastic work!

screenhunter_278 jul 28 15 51 screenhunter_279 jul 28 15 51 screenhunter_280 jul 28 15 51

spite commented 8 years ago

Great! Thanks for the feedback and the nice photos for testing!

neopaf commented 8 years ago

Jaume, thank you very much. This was fast, and very pleasant. Thank you very much for your effort and for hosting of this. We were directed to you by Sketchfab support people.

I've found one more detail, will file as a separate issue.

neopaf commented 8 years ago

Jaume, is depth map is (240 × 240) and main image (1936 × 1936) somehow related to Quad?

Maybe Quad is just "take each N-th point from depthmap?" 1=take every point 2=take one, ignore one point 3=take one, ignore two points ?