Can't see Video on PC (Windows 10) with python

[bender2po]

Describe the bug Can't see Video on PC (Windows 10) with python.

To Reproduce Steps to reproduce the behavior:

1. Go to 'cmd'
2. Type 'python keyboard-pygame.py --video'
3. See error: I cannot see any video stream of the robot.

Expected behavior I am expecting to see a video pop-up and which displays camera-view of the robot.

Screenshots

Desktop (please complete the following information):

• OS: Windows 10

Smartphone (please complete the following information):

• Device: Motorola 5gs plus

[thias15]

It's not tested on Windows but should work. On the phone are you on the FreeRoam screen? Only on that screen video is implented right now. Does the connection work otherwise, i.e. can you control the bot with the keyboard?

[bender2po]

Thank you for the response.

I checked for 'Default' and 'FreeRoam' screens. Video stream is not working for both.

I think connection is established because I can see speed and steering values changing on the FreeRoam-screen on the phone, according to the keyboard input.

Currently, I am performing only open-loop testing, i.e., the bot is not connected yet.

[thias15]

Ok, video should also work open loop. Just the latency is currently still a bit too high to drive. Have you agreed to the permissions for camera and audio? @izivkov do you have any idea what the problem might be?

[izivkov]

From the screenshot, looks like the python app is disconnected at this time. The last line reads "Received"only, which means the program did not receive any data from the Bot, and this causes the python program to exit. Try restarting the python and establish connection again. If no video, try switching between FreeRoam and Default screen, a few times. In FreeRaom screen, should see video.

[bender2po]

I don't think connection disconnected because I can still control the values (speed, steering on FreeRoam) via keyboard.

I tried as @izivkov suggested. Still no luck. Please see screen-shot below:

[izivkov]

Hmm, there should be "VIDEO_SERVER_URL" and "VIDEO_COMMAND" sent to the python program, which I don't see.

[izivkov]

You also updated to the latest app version, right? The python script works with the latest APK.

[bender2po]

I am also expecting to see "VIDEO_SERVER_URL" and "VIDEO_COMMAND" in order to satisfy the python's 'if-conditions'.

1. Camera, Location, Microphone, Storage permissions were given.
2. I tried connecting Computer (python) and Phone (OpenBot) to the router as well as another phone's hot spot. Same result in the both scenarios. That is video is not working. However, I could play with keyboard on computer, and, see speedometer and steering GUI on OpenBot app is changing nicely (ie, no visual delay).
3. I am using 'OpenBot-0.3.1.apk'. Is this the latest one?

[izivkov]

OK, forget what I said about the network connection type. I deleted this comment. This is not the problem since you are communicating to the python script. So, just check the following:

1. You are running the latest APK for the bot app.
2. Your camera permission is enabled in the setting for the OpenBot app.
3. You are using a third device for hotspot.

[bender2po]

Hi @izivkov, please see my comments on the previous message. I think we posted those messages together. (Please ignore this message in case you have already noticed my previous comments)

[izivkov]

You can download the app from here, using the barcode:

https://github.com/intel-isl/OpenBot/tree/master/android

Yes, I think you have the right version. OpenBot-0.3.1.apk

Not sure what could be wrong. Maybe try another phone. We have tried it with a few phones, like Moto power one, some Samsung phones, but maybe there is something different.

[bender2po]

The barcode directs me to the version 'OpenBot-0.3.1.apk'. Which is already in use.

[izivkov]

Yes, I need to investigate a bit more. All the basics are correct and it should work. Maybe you can help by trying with a different phone.

[bender2po]

I just tried with Oneplus-Nord. Still no luck.

[thias15]

Can you try the latest OpenBot app version from master? You can download it from here.

[bender2po]

@thias15 It is getting stuck here:

[thias15]

Ok, you are almost there now. In principle you should then see a window with the video stream open. It may take a few seconds. Does it just get stuck there indefinitely? Did you move/rotate the phone in the meantime? This can cause issues since on every rotation it will try to reconnect.

[bender2po]

Yes, it is stuck indefinitely!!

I locked the phone screen in Potrait in one scenario and Landscape in an another scenario. In both the scenarios the python-script got stuck. (FYI, the phone getting heated up nicely. I guess the phone is doing hard work )

[izivkov]

See if you can access this URL using VLC rtsp://192.168.133.125:1935 while this is running.

Other than that, make sure the video window is not hidden behind something, and try with both phones.

[bender2po]

Thank you for debugging with me.

For some reason it started working.. Now I can see the pygame window with stream...I am not sure of the reason.

Cheers!!

[thias15]

Ok, great to hear it! If you figure out at some point what the issue was, it would be very helpful to know for others.

[izivkov]

👍

On Mon, May 10, 2021 at 9:49 AM thias15 @.***> wrote:

Ok, great to hear it! If you figure out at some point what the issue was, it would be very helpful to know for others.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/intel-isl/OpenBot/issues/190#issuecomment-836724915, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA7M37SFFEBTCCJL6ERQUNTTM7P7ZANCNFSM44OMPSWA .

[bender2po]

Sure, I will post here if I figure out the reason.

[moneypi]

Hello, I met the same problem, any help? Thanks!

[bender2po]

@moneypi I am not expert in this field. This is what worked for me (and a workaround as well):

1. Installed the debug version of android app from here: https://github.com/intel-isl/OpenBot/actions/runs/818493537
2. Replaced some part of 'play_video' method in the VideoPlayer with below snippet

   def play_video(self):
       if not self.stream:
           print(f'Sream not set')
           return
       print(f'Opening the stream...{self.stream}')
       cap = cv2.VideoCapture(self.stream)
       while(cap.isOpened()):
           ret, img = cap.read()
           #img = cv2.rotate(img, cv2.ROTATE_180)
           cv2.imshow('Video', img)
           cv2.waitKey(1)

3. (optional) reduce pygame window size from setup_screen
4. Make sure your PC and phone are connected to same WiFi source
5. Open the debug app and select 'Free Roam'. On your PC: run the python code with '--video' (if it is not connected, then give a few more clean trials). Now, you would see two pop-ups: one for stream and another for pygame control. You should select the pygame window to use keyboard control.

[moneypi]

Thanks, my friend fix this for me. He comment andGate.addCondition("camera permission"); from https://github.com/intel-isl/OpenBot/blob/master/android/app/src/main/java/org/openbot/env/RtspServer.java and rebuild Android APP. And I got Video Stream.

FYI, my phone is Redmi K30 5G/MIUI 12.1.8.

[izivkov]

Hmm, you need camera permissions. You have bypassed it for now, but can cause problems in the future.

The first think I would check is if the app has camera permissions from the setup.

If not, you can set it manually. But I would uninstall the app, build it again, and it should ask you to grant permission for the camera.

If not, you maybe running a pretty old version of Android. What is your Android version.

On Wed., May 26, 2021, 11:44 p.m. moneypi, @.***> wrote:

Thanks, my friend fix this for me. He comment andGate.addCondition("camera permission"); from https://github.com/intel-isl/OpenBot/blob/master/android/app/src/main/java/org/openbot/env/RtspServer.java and rebuild Android APP. And I got Video Stream.

FYI, my phone is Redmi K30 5G/MIUI 12.1.8.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/intel-isl/OpenBot/issues/190#issuecomment-849295854, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA7M37XL526QUQEN32D5PNLTPW5YFANCNFSM44OMPSWA .

[moneypi]

Android Version 11 And，when running the app first time, I got the question: Take photos and record videos. And I select "Allow only while using the app".

I am not familiar with Android, maybe that's the reason.

[izivkov]

Thanks, I'll check if the code has a problem with android 11.

On Thu., May 27, 2021, 1:47 a.m. moneypi, @.***> wrote:

Android Version 11 And，when running the app first time, I got the question: Take photos and record videos. And I select "Allow only while using the app".

I am not familiar with Android, maybe that's the reason.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/intel-isl/OpenBot/issues/190#issuecomment-849340921, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA7M37URACPL5EXAK7VU7KDTPXMF7ANCNFSM44OMPSWA .

[moneypi]

My friend told me that is the problem of MIUI, which is different with the standard Android.