Hao-Starrr / humanoid-arm-retarget

Humanoid robot arms retarget algorithm with VisionPro app
GNU General Public License v3.0
36 stars 4 forks source link

can not show realsensecamera in VR #4

Open chendust opened 4 days ago

chendust commented 4 days ago

This is a great project, and I want to pay tribute to the developers. I have replicated all the functions of the project on my own robot, except for the real-time display of the RealSense images in the VR application you provided. I streamed the server IP as 192.168.1.106, and I tried to change the IP address in the project to my own IP address (Figure 1), but when I opened the application, it was still blank. I then tried to solve the problem, and I successfully opened the connection in the VR browser (Figure 2), which indicated that it was a non-private connection. I skipped past that and successfully saw the images. Figure 3 shows the debugging error in Xcode, which appears to be an issue with URL parsing. Please tell me how to fix it. 1 2 3 image

Hao-Starrr commented 4 days ago

It is probably because https. Apple does not allow WebXR on non-https connections. To test the application locally, we need to create a self-signed certificate and install it on the client. You need a ubuntu machine and a router. Connect the VisionPro and the ubuntu machine to the same router. See https://github.com/OpenTeleVision/TeleVision Local streaming part.

chendust commented 4 days ago

It is probably because https. Apple does not allow WebXR on non-https connections. To test the application locally, we need to create a self-signed certificate and install it on the client. You need a ubuntu machine and a router. Connect the VisionPro and the ubuntu machine to the same router. See https://github.com/OpenTeleVision/TeleVision Local streaming part.

Thank you for your reply. I carefully reviewed the process and found two issues. 1. The certificate file uploaded is invalid, and the correct file needs to be generated by running the command "mkcert -CAROOT". 2. Trust the certificate in the universal settings of VR. After fixing these issues, the problem has been resolved. However, I still have a minor issue. The image displayed in VR is a bit small, and is there a way to display it full screen?

Hao-Starrr commented 4 days ago

you can resize the webview in visionpro app

WebView(url: URL(string: "https://192.168.11.214:8012/?ws=wss://192.168.11.214:8012")!)
                .frame(width: 1500, height: 1200)

and can play with these parameters in silverscreen.py

ImageBackground(
                                color_image,
                                format="jpeg",
                                quality=40,
                                key="left-image",  # todo: figure out what this means
                                interpolate=True,
                                aspect=1.778,
                                distanceToCamera=2,
                                position=[0, -0.5, -2],
                                rotation=[0, 0, 0],
                            ),
Hao-Starrr commented 4 days ago

If that doesn't change, Vuer is the library we used in visualization. You may find in https://docs.vuer.ai/en/latest/