How to record video?

[joshchristopherson]

After enabling recording in cmakelists, no video seems to be being recorded - I only see an empty mo4 being created. #streamon and #streamoff commands are also being sent. So how can I record video?

[vss2sn]

Could you confirm whether you are able to see the video stream from the drone and whether you are allowing the code to exit cleanly (by calling exit from the joystick) or stopping it using Ctrl+C or some such method? Also, did you delete the old files and run cmake again and then make to ensure the changes in the CMake actually take effect?

[vss2sn]

Closing due to lack of response. Feel free to reopen if the issue persists.

[joshchristopherson]

Sorry for the delay, I wasn't able to work on this for a while.

I deleted the CMake files and remade them, and I have recording enabled in the CMakeList. Also, I have a streamon and streamoff command in the main.cpp, and I tried letting the code exit cleanly on its own. I still get empy .mp4 files, so I'm not sure what I'm doing wrong.

[vss2sn]

No problem; re-opening the issue. I understand that the commands are being sent. Could you please confirm whether the live stream of the video appears on your screen, and attach your main.cpp and a log file of all the output from a sample run of the code, when using the joystick to exit? I have been unable to reproduce this issue, and this would be helpful.

[samehmohamed88]

I can confirm I am seeing the same issue. I also have segmentation faults due to pangolin which maybe causal?

I followed the instructions in the comments on line 131 of video_socket.cpp, which does solve the seg fault, but I have not checked if that solves the recording issue.

Still investigating this as well ...

My setup is:

Ubuntu 20.04 on MSI GS65. Intel Core i7 8th Gen, 16GB RAM.

[vss2sn]

@joshchristopherson In addition to my previous comment, could you let me know whether you're running slam? Also could you add std::cout << __LINE__ << std::endl; // not '\n'; before and after the video write and release lines (video_socket.cpp, 124 and 158) inside the #ifdef blocks and let me know whether they get printed? All of the above should help debug the issue.

@aboarya That's definitely causal. Let me know whether the issue persists when that the seg fault no longer does, in which case there might be another issue as well.

[joshchristopherson]

@vss2sn

#include <mutex>
#include <condition_variable>

#include "utils.hpp"
#include "tello.hpp"

#ifdef USE_TERMINAL
#include "command_terminal.hpp"
#endif

#ifdef USE_CONFIG
#include "config_handler.hpp"
#endif

int main(){
  asio::io_service io_service;
  asio::io_service::work work(io_service);

  std::condition_variable cv_run;

  std::string path_to_tello = "~/code/tello";
#ifdef USE_CONFIG

  std::map<std::string, std::unique_ptr<Tello>>  m = handleConfig(path_to_tello + "/config.yaml", io_service, cv_run);

  if(m.count("0.prime.0") > 0){
    Tello& t = *m["0.prime.0"];
    t.cs->addCommandToQueue("command");
    t.cs->addCommandToQueue("sdk?");
    t.cs->addCommandToQueue("command");
    t.cs->addCommandToQueue("sdk?");
    t.cs->addCommandToQueue("streamon");
    t.cs->addCommandToQueue("takeoff");
    t.cs->executeQueue();
    //t.cs->addCommandToQueue("forward 20");
    //t.cs->addCommandToQueue("back 20");
    t.cs->addCommandToQueue("delay 1");
    t.cs->addCommandToFrontOfQueue("stop");
    t.cs->addCommandToQueue("streamoff");
    // t.cs->stopQueueExecution();
    t.cs->doNotAutoLand();
    t.cs->addCommandToQueue("land");
  }
  else{
    utils_log::LogErr() << "The requested drone does not exist.";
  }

#else

  Tello t(io_service, cv_run, "192.168.10.1", "8889", "11111", "8890", path_to_tello + "/camera_config.yaml", path_to_tello + "/orb_vocab.dbow2");

  t.cs->addCommandToQueue("command");
  t.cs->addCommandToQueue("sdk?");
  t.cs->addCommandToQueue("streamon");
  t.cs->addCommandToQueue("takeoff");
  t.cs->executeQueue();
  //t.cs->addCommandToQueue("forward 20");
  //t.cs->addCommandToQueue("back 20");
  t.cs->addCommandToQueue("delay 1");
  t.cs->addCommandToFrontOfQueue("stop");
  t.cs->addCommandToQueue("streamoff");
  // t.cs->stopQueueExecution();
  t.cs->doNotAutoLand();
  t.cs->addCommandToQueue("land");

#endif

  {
    std::mutex mtx;
    std::unique_lock<std::mutex> lck(mtx);
    cv_run.wait_for(lck,std::chrono::seconds(300));
  }

  utils_log::LogWarn() << "----------- Done -----------";
  utils_log::LogWarn() << "----------- Landing -----------";
  // t.cs->exitAllThreads();
  io_service.stop();
  usleep(1000000); // Ensure this is greater than timeout to prevent seg faults
  utils_log::LogDebug() << "----------- Main thread returns -----------";
  return 0;
}

[2021-02-27 | 21:42:49 | Info---] Successfully sent command [rc 0 0 0 0] to address [192.168.10.1:8889]. |command_socket.cpp:93|
OpenCV: FFMPEG: tag 0x7634706d/'mp4v' is not supported with codec id 13 and format 'mp4 / MP4 (MPEG-4 Part 14)'
OpenCV: FFMPEG: fallback to use tag 0x00000020/' ???'
mkdir: cannot create directory ‘../snapshots’: File exists
[2021-02-27 | 21:42:49 | Info---] Added command [command] to queue. |command_socket.cpp:116|
[2021-02-27 | 21:42:49 | Info---] Added command [sdk?] to queue. |command_socket.cpp:116|
[2021-02-27 | 21:42:49 | Info---] Added command [streamon] to queue. |command_socket.cpp:116|
[2021-02-27 | 21:42:49 | Info---] Added command [takeoff] to queue. |command_socket.cpp:116|
[2021-02-27 | 21:42:49 | Info---] Executing queue commands. |command_socket.cpp:123|
[2021-02-27 | 21:42:49 | Info---] Added command [delay 1] to queue. |command_socket.cpp:116|
[2021-02-27 | 21:42:49 | Info---] Added command [streamoff] to queue. |command_socket.cpp:116|
[2021-02-27 | 21:42:49 | Debug--] Automatic landing disabled. |command_socket.cpp:204|
[2021-02-27 | 21:42:49 | Info---] Added command [land] to queue. |command_socket.cpp:116|
[2021-02-27 | 21:42:49 | Info---] Successfully sent command [stop] to address [192.168.10.1:8889]. |command_socket.cpp:93|
[2021-02-27 | 21:42:49 | Info---] Waiting to send command [command] as no response has been received for the previous command. Thread calling send command paused. |command_socket.cpp:157|
[2021-02-27 | 21:42:54 | Info---] Timeout - Attempt #0 for command [command]. |command_socket.cpp:81|
[2021-02-27 | 21:42:54 | Info---] Successfully sent command [command] to address [192.168.10.1:8889]. |command_socket.cpp:93|
[2021-02-27 | 21:42:54 | Info---] Waiting to send command [sdk?] as no response has been received for the previous command. Thread calling send command paused. |command_socket.cpp:157|
[2021-02-27 | 21:42:54 | Info---] Received response [] after sending command [sdk?] from address [192.168.10.1:8889]. |command_socket.cpp:55|
[2021-02-27 | 21:42:54 | Info---] Received response [ok] after sending command [sdk?] from address [192.168.10.1:8889]. |command_socket.cpp:55|
[2021-02-27 | 21:42:55 | Info---] Successfully sent command [sdk?] to address [192.168.10.1:8889]. |command_socket.cpp:93|
[2021-02-27 | 21:42:55 | Info---] Waiting to send command [streamon] as no response has been received for the previous command. Thread calling send command paused. |command_socket.cpp:157|
[2021-02-27 | 21:42:55 | Info---] Received response [unknown command: sdk?] after sending command [streamon] from address [192.168.10.1:8889]. |command_socket.cpp:55|
[2021-02-27 | 21:42:55 | Info---] Successfully sent command [streamon] to address [192.168.10.1:8889]. |command_socket.cpp:93|
[2021-02-27 | 21:42:55 | Info---] Waiting to send command [takeoff] as no response has been received for the previous command. Thread calling send command paused. |command_socket.cpp:157|
[2021-02-27 | 21:42:55 | Info---] Received response [ok] after sending command [takeoff] from address [192.168.10.1:8889]. |command_socket.cpp:55|
[2021-02-27 | 21:42:56 | Info---] Successfully sent command [takeoff] to address [192.168.10.1:8889]. |command_socket.cpp:93|
[2021-02-27 | 21:42:57 | Info---] Waiting to send command [streamoff] as no response has been received for the previous command. Thread calling send command paused. |command_socket.cpp:157|
[2021-02-27 | 21:43:01 | Info---] Timeout - Attempt #0 for command [delay 1]. |command_socket.cpp:81|
[2021-02-27 | 21:43:01 | Info---] Successfully sent command [streamoff] to address [192.168.10.1:8889]. |command_socket.cpp:93|
[2021-02-27 | 21:43:01 | Info---] Waiting to send command [land] as no response has been received for the previous command. Thread calling send command paused. |command_socket.cpp:157|
[2021-02-27 | 21:43:01 | Info---] Received response [ok] after sending command [land] from address [192.168.10.1:8889]. |command_socket.cpp:55|
[2021-02-27 | 21:43:02 | Info---] Successfully sent command [land] to address [192.168.10.1:8889]. |command_socket.cpp:93|
[2021-02-27 | 21:43:05 | Info---] Received response [ok] after sending command [land] from address [192.168.10.1:8889]. |command_socket.cpp:55|
[2021-02-27 | 21:47:49 | Warn---] ----------- Done ----------- |main.cpp:74|
[2021-02-27 | 21:47:49 | Warn---] ----------- Landing ----------- |main.cpp:75|
[2021-02-27 | 21:47:49 | Debug--] ----------- Command socket io_service thread exits ----------- |command_socket.cpp:33|
[2021-02-27 | 21:47:49 | Debug--] ----------- Video socket io_service thread exits ----------- |video_socket.cpp:50|
[2021-02-27 | 21:47:50 | Debug--] ----------- Main thread returns ----------- |main.cpp:79|
160
162
[2021-02-27 | 21:47:51 | Debug--] ----------- DNAL worker thread exits ----------- |command_socket.cpp:241|
[2021-02-27 | 21:47:51 | Debug--] ----------- Send queue commands thread exits ----------- |command_socket.cpp:173|

I actually am not getting the live feed on my screen. Here's my main.cpp and this is the output I'm getting. I added the prints in the locations you told me to, and it only seems like the release prints are happening. And it seems like the streamon and streamoff commands aren't getting acknowledged? I'm not running slam.

Thanks for your help.

[vss2sn]

@joshchristopherson At first glance, OpenCV: FFMPEG: tag 0x7634706d/'mp4v' is not supported with codec id 13 and format 'mp4 / MP4 (MPEG-4 Part 14)' OpenCV: FFMPEG: fallback to use tag 0x00000020/' ???' indicate that the cause of the error might be related the the OpenCV version and the format you are trying to save it in, especially since the lines around 124 didn't print, and that the video feed is not visible on the screen. Could you take a look at https://stackoverflow.com/questions/57792837/opencv-ffmpeg-tag-is-not-supported-with-codec-id-12-and-format-mp4-mp4 and https://stackoverflow.com/questions/38397964/cant-write-and-save-a-video-file-using-opencv-and-python/55576859 and see whether any of the suggested changes help? I'll try and recreate the error soon.

[vss2sn]

Based on the logs above, the streamon command is not being correctly received by the drone. I'd suggest adding delays between sending the commands and checking whether that solves the issue. If the video stream is visible on the screen, then there might be a different issue at play here, which I have not been able to reproduce.