spite / android-lens-blur-depth-extractor

MIT License
292 stars 34 forks source link

Google Pixel 4 images"produce "No depth data found" message #6

Open GeorgeAdamon opened 5 years ago

GeorgeAdamon commented 5 years ago

I'm trying to extract depth from Portrait mode images taken with a Pixel 4 (Android 10), but when uploading the images to the depth extractor, I get a no depth data found message. The metadata of the images suggest that the depth data is indeed encoded.

`

Depth RangeInverse Diopters 0.086360 0.129485 android/depthmap OpticalAxis android/confidencemap 256 ed2wPQAAAACwGLE9AAAAAA9UsT0AAAAAlY+xPQAAAABCy7E9AAAAABgHsj0AAAAAGEOyPQAAAAA/f7I9AAAAAI67sj0AAAAACPiyPQAAAACpNLM9AAAAAHNxsz0AAAAAZ66zPQAAAACG67M9AAAAAMwotD0AAAAAPWa0PQAAAADZo7Q9AAAAAJ3htD0AAAAAjB+1PQAAAACmXbU9AAAAAOubtT0AAAAAWtq1PQAAAAD0GLY9AAAAALpXtj0AAAAAqpa2PQAAAADG1bY9AAAAAA4Vtz0AAAAAg1S3PQAAAAAilLc9AAAAAO7Ttz0AAAAA5hO4PQAAAAAMVLg9AAAAAF2UuD0AAAAA3NS4PQAAAACIFbk9AAAAAGFWuT0AAAAAaJe5PQAAAACc2Lk9AAAAAP8Zuj0AAAAAj1u6PQAAAABOnbo9AAAAADvfuj0AAAAAViG7PQAAAAChY7s9AAAAABqmuz0AAAAAw+i7PQAAAACcK7w9AAAAAKNuvD0AAAAA27G8PQAAAABD9bw9AAAAANs4vT0AAAAAo3y9PQAAAACcwL09AAAAAMYEvj0AAAAAIUm+PQAAAACtjb49AAAAAGrSvj0AAAAAWhe/PQAAAAB7XL89AAAAAM6hvz0AAAAAVOe/PQAAAAALLcA9AAAAAPZywD0AAAAAE7nAPQAAAABl/8A9AAAAAOlFwT0AAAAAoYzBPQAAAACN08E9AAAAgKwawj0AAACAAGLCPQAAAICIqcI9AAAAgEbxwj0AAACAODnDPQAAAIBfgcM9AAAAgLzJwz0AAACAThLEPQAAAIAXW8Q9AAAAgBSkxD0AAACASu3EPQAAAIC1NsU9AAAAgFeAxT0AAACAMMrFPQAAAIBAFMY9AAAAgIlexj0AAACACKnGPQAAAIDA88Y9AAAAgLE+xz0AAACA2YnHPQAAAIA71cc9AAAAgNUgyD0AAACAqWzIPQAAAIC2uMg9AAAAgP4EyT0AAACAf1HJPQAAAIA6nsk9AAAAgDHryT0AAACAYjjKPQAAAIDOhco9AAAAgHXTyj0AAACAWCHLPQAAAIB4b8s9AAAAgNK9yz0AAACAagzMPQAAAIA/W8w9AAAAgFCqzD0AAACAnvnMPQAAAIArSc09AAAAgPWYzT0AAACA/ejNPQAAAIBEOc49AAAAgMmJzj0AAACAjdrOPQAAAICQK889AAAAgNN8zz0AAACAV87PPQAAAIAZINA9AAAAgB1y0D0AAACAYcTQPQAAAIDmFtE9AAAAgKxp0T0AAACAtbzRPQAAAID/D9I9AAAAgItj0j0AAACAW7fSPQAAAIBtC9M9AAAAgMFf0z0AAACAWrTTPQAAAIA2CdQ9AAAAgFde1D0AAACAu7PUPQAAAIBmCdU9AAAAgFVf1T0AAACAibXVPQAAAIADDNY9AAAAgMRi1j0wAEO9yrnWPR2tLb8WEdc9HJWnv6to1z2pU/i/h8DXPRuJJMCrGNg9YehMwBdx2D2oR3XAy8nYPXfTjsDIItk9GgOjwA582T2+MrfAntXZPWFiy8B4L9o9BJLfwJuJ2j2nwfPACuTaPaX4A8HEPts9dxAOwciZ2z1JKBjBGPXbPRpAIsG1UNw97FcswZ+s3D29bzbB1AjdPY+HQMFYZd09YZ9KwSnC3T0yt1TBRh/ePQTPXsG0fN491uZowW/a3j2n/nLBejjfPXkWfcHTlt89JZeDwX713z0Oo4jBeVTgPfeujcHDs+A94LqSwWAT4T3IxpfBTXPhPbHSnMGN0+E9mt6hwR404j2D6qbBA5XiPWz2q8E79uI9VQKxwcZX4z09DrbBprnjPSYau8HZG+Q9DybAwWJ+5D34McXBP+HkPeE9ysFzROU9yUnPwf2n5T2yVdTB3QvmPZth2cEVcOY9hG3ewaPU5j1teePBijnnPVWF6MHInuc9PpHtwWAE6D0nnfLBUmroPRCp98Gc0Og9+bT8wUA36T1x4ADCQp7pPWVmA8KcBeo9WuwFwlJt6j1OcgjCZdXqPUL4CsLUPes9N34NwqCm6z0rBBDCyg/sPSCKEsJSeew9FBAVwjjj7D0IlhfCfE3tPf0bGsIhuO098aEcwicj7j3mJx/Ci47uPdqtIcJS+u49zjMkwnpm7z3DuSbCBNPvPbc/KcLwP/A9rMUrwkCt8D2gSy7C9BrxPZTRMMIKifE9iVczwof38T193TXCaWbyPQAAOMKw1fI9AAA4wlxF8z0AADjCcbXzPQAAOMLuJfQ9AAA4wtGW9D0AADjCHgj1PQAAOMLVefU9AAA4wvPr9T0AADjCfF72PQAAOMJy0fY9AAA4wtJE9z0AADjCnrj3PQAAOMLXLPg9AAA4wn6h+D0AADjCkRb5PQAAOMITjPk9AAA4wgUC+j0AADjCZ3j6PQAAOMI37/o9AAA4wnpm+z0AADjCL977PQAAOMJUVvw9AAA4wu3O/D0AADjC+kf9PQAAOMJ7wf09AAA4wnA7/j0AADjC27X+PQAAOMK9MP89AAA4whSs/z0AADjC8RMAPgAAOMIVUgA+AAA4wnaQAD4AADjCEs8APgAAOMLtDQE+AAA4wgRNAT4AADjCWYwBPgAAOMLsywE+AAA4wr4LAj4AADjC0EsCPgAAOMIfjAI+AAA4wq/MAj4AADjCfw0DPgAAOMKPTgM+AAA4wt+PAz4AADjCctEDPgAAOMJFEwQ+AAA4wlpVBD4AADjCsZcEPgAAOMI ` ![00100lrPORTRAIT_00100_BURST20191031015532163_COVER](https://user-images.githubusercontent.com/30707361/67913198-236b4c00-fb84-11e9-8841-036e7671aaee.jpg)
PiotrDabrowskey commented 4 years ago

I have the same problem on Pixel 3 with Android 11 and Google Camera 7.5.107. Any chance to get this solved?

thyung commented 3 years ago

Pixel phone portrait mode photo is concatenated of 4 JFIF structure https://en.wikipedia.org/wiki/JPEG_File_Interchange_Format . Each JFIF structure is an jpeg image.

A JFIF structure starts with marker 0xFFD8 and ends with marker 0xFFD9. Therefore, we can split a portrait mode image into 4 jpeg files.

The following python code prints the marker positions and splits PXL_20210107_114027740.PORTRAIT.jpg into,

  1. pxl_out_0.jpg: display image
  2. pxl_out_1.jpg: original image
  3. pxl_out_2.jpg: depthmap with 256 grey level
  4. pxl_out_3.jpg: dummy image filled with 255
with open('PXL_20210107_114027740.PORTRAIT.jpg', mode='rb') as infile:
    buffer = infile.read()

bufferlen = len(buffer)
pos = 0
pos_d8 = 0
n = 0
i = 0
while i < bufferlen:
    if buffer[i] == 0xff:
        pos = i
        i += 1
        if buffer[i] == 0xd8:
            print('ffd8: {0}'.format(pos))
            pos_d8 = pos
        elif buffer[i] == 0xd9:
            print('ffd9: {0} len: {1}'.format(pos, pos - pos_d8 + 2))
            with open('pxl_out_{0}.jpg'.format(n), mode='wb') as outfile:
                n += 1
                outfile.write(buffer[pos_d8: pos + 2])
    i += 1