marek-simonik / record3d

Accompanying library for the Record3D iOS app (https://record3d.app/). Allows you to receive RGBD stream from iOS devices with TrueDepth camera(s).
https://record3d.app/
GNU Lesser General Public License v2.1
383 stars 57 forks source link

RuntimeError: Cannot connect to device #0, try different index. #13

Closed y0pta closed 3 years ago

y0pta commented 3 years ago

os: Windows 10 device: IPhone 12 Pro Max record3d version: 1.5.4

Hi! I'm trying to run demo_main.py with IPhone 12 Pro Max. I've install iTunes, build library from sources, connect device and press record button. Then run demo script and get the error for each device from range [-1, 10]:

Searching for devices
0 device(s) found
[libusbmuxd] usbmuxd_get_device_list: error opening socket!
Traceback (most recent call last):
  File "D:/ARVR-DL/utilities/record3d/demo-main.py", line 74, in <module>
    app.connect_to_device(dev_idx=0)
  File "D:/ARVR-DL/utilities/record3d/demo-main.py", line 30, in connect_to_device
    .format(dev_idx))
RuntimeError: Cannot connect to device #0, try different index.

Process finished with exit code 1

While debugging, figured out, that empty list of avaliable devices is returned here.

Any suggestion?

Thanks in advance!

marek-simonik commented 3 years ago

Hello,

something similar does happen to me as well; I found that this is caused either by:

Can you please also open iTunes and make sure you have chosen that you "Trust This Computer" and that you can e.g. browse the contents of your iPhone via iTunes File Sharing inside iTunes?

If that would not work, then you can try the usual advice: restarting your computer :) – but I assume you have already done that.

Let me know if that helped or not. Thank you.

– Marek

y0pta commented 3 years ago

Thanks for your quick reply! Still can't resolve the problem:( I need depth data from the app. I loaded data to pc after recording a video without streaming mode. Could you please suggest me, how to get depth values from frame? Tried:

std::ifstream input("1.depth", std::ios::binary);

    std::vector<uint8_t> depthEncoded(
        (std::istreambuf_iterator<char>(input)),
        (std::istreambuf_iterator<char>()));

    input.close();

    size_t depthEncodedSize = depthEncoded.size();

    size_t depthSize = 32 * 960 * 720;
    uint8_t* depth = new uint8_t[depthSize + 1];

    uint8_t* lzfseScratchBuffer_ = new uint8_t[lzfse_decode_scratch_size()];

    lzfse_decode_buffer(depthEncoded.data(),
        depthEncodedSize,
        depth,
        depthSize, lzfseScratchBuffer_);

After that I receive buffer with a single value in each byte. Thank you in advance!

marek-simonik commented 3 years ago

Regarding streaming: can you see and browse (e.g. copy files from/to it) your iPhone in iTunes?

Regarding depth decoding: please try the following snippet (float depth values are stored in the depth buffer):

bool isLiDAR = true; // Change this according to the video type
size_t frameWidth = isLiDAR ? 192 : 480;
size_t frameHeight = isLiDAR ? 256 : 640;

std::ifstream input("1.depth", std::ios::binary);

std::vector<uint8_t> depthEncoded(
        (std::istreambuf_iterator<char>(input)),
        (std::istreambuf_iterator<char>()));

input.close();

size_t depthEncodedSize = depthEncoded.size();
size_t depthSize = sizeof(float) * frameWidth * frameHeight;
float* depth = new float[depthSize];

lzfse_decode_buffer((uint8_t*)depth,
                    depthSize,
                    depthEncoded.data(),
                    depthEncodedSize, nullptr);

// [Optional] visualize the depth using OpenCV
cv::Mat depthImg {(int)frameHeight, (int)frameWidth, CV_32F, depth};
cv::imshow("Depth", depthImg);
cv::waitKey(0);

delete[] depth;
y0pta commented 3 years ago

No, I cannot see my device in ITunes, but can copy images. Apple support recommended refresh drivers and reboot both PC and IPhone, but it's doesn't work in my case.

Thanks a lot:) I noticed, that you use depthSize, that differs from corresponding image. How to further match the image and depth? Can I use cv::resize?

marek-simonik commented 3 years ago

If you cannot see your iPhone in iTunes, then you will not be able to use the USB streaming feature, because it relies on having iTunes support. Please try on a different computer to see if the problem is related to your iPhone or to you current OS.

How to further match the image and depth?

Yes, I would recommend using cv::resize().