Live preview and video recording at the same time

[karmelcorn]

Hi, does the ricoh theta usb api support live streaming and recording of the video stream at the same time?

Thank you.

[DaisukeHohjoh]

Hi, @karmelcorn

Unfortunately, RICOH THETA does not support video recording and live streaming over usb at the same time.

---

I can suggest ..

THETA X supports video recording in-camera and live streaming at the same time, with setting "Video liveview" to ON. In this case, you can receive live streaming over HTTP (not over USB unfortunately) during video recording. Live preview resolution is 1024*512px. You can see this article as well.

Another suggestion, you may be able to use USB Live Streaming mode to stream video frames from THETA to PC side over USB, and recording at PC side.

Best regards, Daisuke Hohjoh

[karmelcorn]

@DaisukeHohjoh, Thank you for your reply.

Greatly appreciate it if I can get help on a few more questions:

1. I was trying the streaming over wifi, but I am getting intermittent errors when I connect to the wifi on windows. Please see below ping and curl logs. Do you know why that happens and how I can make it more stable?

C:\Users\xxx>ping 192.168.1.1

Pinging 192.168.1.1 with 32 bytes of data:
Reply from 192.168.1.1: bytes=32 time=2ms TTL=64
Reply from 192.168.1.1: bytes=32 time=4ms TTL=64
Reply from 192.168.1.1: bytes=32 time=9ms TTL=64
Reply from 192.168.1.1: bytes=32 time=2ms TTL=64

Ping statistics for 192.168.1.1:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 2ms, Maximum = 9ms, Average = 4ms

C:\Users\xxx>curl 192.168.1.1/osc/info
{"api":["/osc/info","/osc/state","/osc/checkForUpdates","/osc/commands/execute","/osc/commands/status"],"apiLevel":[2],"_bluetoothMacAddress":"58:38:79:8F:48:F6","endpoints":{"httpPort":80,"httpUpdatesPort":80},"firmwareVersion":"2.10.1","gps":true,"gyro":true,"manufacturer":"Ricoh Company, Ltd.","model":"RICOH THETA X","serialNumber":"14140359","supportUrl":"https://theta360.com/en/support/","uptime":10748,"_wlanMacAddress":"2C:6D:C1:21:45:26"}
C:\Users\xxx>ping 192.168.1.1

Pinging 192.168.1.1 with 32 bytes of data:
PING: transmit failed. General failure.
PING: transmit failed. General failure.
PING: transmit failed. General failure.
PING: transmit failed. General failure.

Ping statistics for 192.168.1.1:
    Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),

C:\Users\xxx>curl 192.168.1.1/osc/info
curl: (7) Failed to connect to 192.168.1.1 port 80 after 1 ms: Couldn't connect to server

C:\Users\xxx>curl 192.168.1.1/osc/info
{"api":["/osc/info","/osc/state","/osc/checkForUpdates","/osc/commands/execute","/osc/commands/status"],"apiLevel":[2],"_bluetoothMacAddress":"58:38:79:8F:48:F6","endpoints":{"httpPort":80,"httpUpdatesPort":80},"firmwareVersion":"2.10.1","gps":true,"gyro":true,"manufacturer":"Ricoh Company, Ltd.","model":"RICOH THETA X","serialNumber":"14140359","supportUrl":"https://theta360.com/en/support/","uptime":10773,"_wlanMacAddress":"2C:6D:C1:21:45:26"}
C:\Users\cong>ping 192.168.1.1

Pinging 192.168.1.1 with 32 bytes of data:
Reply from 192.168.1.1: bytes=32 time=6ms TTL=64
Reply from 192.168.1.1: bytes=32 time=4ms TTL=64
Reply from 192.168.1.1: bytes=32 time=2ms TTL=64
Reply from 192.168.1.1: bytes=32 time=2ms TTL=64

Ping statistics for 192.168.1.1:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 2ms, Maximum = 6ms, Average = 3ms

C:\Users\xxx>ping 192.168.1.1

Pinging 192.168.1.1 with 32 bytes of data:
PING: transmit failed. General failure.
PING: transmit failed. General failure.
PING: transmit failed. General failure.
PING: transmit failed. General failure.

Ping statistics for 192.168.1.1:
    Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),

C:\Users\xxx>curl 192.168.1.1/osc/info
curl: (7) Failed to connect to 192.168.1.1 port 80 after 1 ms: Couldn't connect to server

2. I also tried out the live streaming function with OBS and VLC. Seems like the stream is a stitched 360 video. Which means I cannot pan the video stream like a regular 360 video. Although it seems like if we do live streaming to Youtube, the youtube player is able to handle the playback of the live stream as a 360 video. But when I record the stream on OBS, it just creates a flat non-360 video file.

Any idea what I need to do to capture the live stream into a proper 360 video?

Thank you!

[DaisukeHohjoh]

Hi @karmelcorn ,

1. Actually RICOH fixied similar problem as yours that THETA X wi-fi connection was dropped frequantly especially with Windows10/11 PC, for firmware version v2.20.1. If your problem is happened with THETA X and also its firmware version is earlier than v2.20.1, could you please try to update firmware to the latest one ?

2. As you said, USB Live Streaming does not have specified metadata to identify that it is a 360 video stream. So, you need to put 360 video metadata to recorded video file. RICOH does not release such tool, but Google releases useful tool Spatial Media Metadata Injector.

Best regards, Daisuke Hohjoh

[karmelcorn]

Hi @DaisukeHohjoh ,

Thank you so much for your reply.

Yes, after updating to the latest 2.30.0 firmware, the THETA X wi-fi connection is now stable. Thank you.

As for using the Live-streaming mode over USB, I am unable to capture the media stream when the camera is in live-streaming mode. In live-streaming mode, in device manager, I see that the Theta X shows up as a camera device. However when I try to use it in Google Meet or in a .Net Windows application (using MediaCature api), it says no video stream found.

However, OBS and VLC is able to capture the video stream from the Theta X. So I think it has something to do with OBS and VLC being able to detect the UVC driver. Do you know how I can "communicate" with the UVC driver like OBS and VLC?

Thank you

[codetricity]

@karmelcorn I have the following examples in this document.

THETA X live streaming over usb cable to Windows and showing stream on:

1. Microsoft Teams Web browser
2. WebRTC getUserMedia (web page) webrtc.github.io
3. Google Meet
4. Zoom
5. VLC
6. OBS
7. Unity with 360 navigation from VR headset
8. Pot Player with 360 navigation
9. two THETA camera simultaneously. (I have tested 3 simultaneously)

---

The above examples are using the USB cable. For live preview over WiFi, I have an example showing the live preview on Windows desktop while simultaneously saving the video into the THETA X body.

The livePreview has lower resolution and framerate then the USB cable and is only used to preview the video while recording into the camera.

The application shown is only to test the API and not intended for production use.

---

If you want to do something like 4K streaming to an RTMP server (like YouTube) while saving the 4K video inside the camera, see this test with the HDR Wireless Live Streaming plugin. You may want to move your question to the community forum as it is going off-topic from the original question about the API specification and capability of the livePreview API.

[karmelcorn]

Thank you, @codetricity.

Following your examples, I am now able to get live-streaming over USB working in Google Meet too.

Can you please point me to the link to the community forum and I will try to move my question over. I am new and sorry that I am a bit lost in all the different repo and forums.

I have one more question, for your live-streaming + video recording demo, do you have to press the physical button on the camera to start the video recording or can you control the start of the recording via the app too?

[codetricity]

Can you please point me to the link to the community forum

This is a community forum, not an official RICOH forum.

https://community.theta360.guide/

Also, please note that I do not work for RICOH. I'm just a guy in the community.

I have one more question, for your live-streaming + video recording demo, do you have to press the physical button on the camera to start the video recording or can you control the start of the recording via the app too?

The demonstration I linked to show live preview on Windows desktop does not include the code to start video capture or put the camera in video mode. The RICOH THETA webAPI can handle these features and should be easy to implement with a standard HTTP POST request.

This is another example of showing livePreview on Windows desktop (Windows 11 WSL) https://github.com/codetricity/ricoh_theta_mjpeg_opencv

The approach above uses OpenCV, so you can easily add things like object detection, edge detection, and visual analysis. If you use Access Point instead of Client Mode, you do not need the digest authentication.