search

facebookresearch / projectaria_tools

projectaria_tools is an C++/Python open-source toolkit to interact with Project Aria data
https://facebookresearch.github.io/projectaria_tools/docs/intro
Apache License 2.0
427 stars 55 forks source link

convert_vrs_to_mp4: module ffmpeg has no attribute input #119

Open kaufManu opened 1 month ago

kaufManu commented 1 month ago

Thanks for providing the convert_vrs_to_mp4 utility, which I find very useful. Unfortunately, I couldn't get it to finish.

After running pip install --upgrade projectaria-tools I had to additionally run pip install python-ffmpeg. This gets the export to start, but towards the end it errors out with:

File "/miniconda3/envs/aria/lib/python3.11/site-packages/projectaria_tools/utils/vrs_to_mp4_utils.py", line 75, in save_timestamp_to_mp4
    ffmpeg_input = ffmpeg.input(input_video_file)
                   ^^^^^^^^^^^^
AttributeError: module 'ffmpeg' has no attribute 'input'

May be it's a version mismatch in the ffmpeg package?

My environment:

I'm calling the export function as written in the docs, i.e.

import os
from projectaria_tools.utils.vrs_to_mp4_utils import convert_vrs_to_mp4

def export_videos(aria_vrs_file):
    parent_folder = os.path.abspath(os.path.join(aria_vrs_file, os.pardir))
    output_folder = os.path.join(parent_folder, "videos")
    os.makedirs(output_folder, exist_ok=True)
    output_video = os.path.join(output_folder, "rgb.mp4")
    convert_vrs_to_mp4(aria_vrs_file, output_video)
SeaOtocinclus commented 1 month ago

See here for the required dependencies https://facebookresearch.github.io/projectaria_tools/docs/data_utilities/advanced_code_snippets/vrs_to_mp4#install-dependencies

Installing moviepy should resolve your issue.

The module has been updated recently too, you can try it directly here https://github.com/facebookresearch/projectaria_tools/blob/main/projectaria_tools/utils/vrs_to_mp4_utils.py

kaufManu commented 1 month ago

Thanks for the quick reply. I already had moviepy installed - I'm including my full environment below.

I also tried the version of the script on main, which gives me the following error:

[MultiRecordFileReader][DEBUG]: Opened file 'recording.vrs' and assigned to reader #0
[VrsDataProvider][INFO]: streamId 211-1/camera-et activated
[VrsDataProvider][INFO]: streamId 214-1/camera-rgb activated
[VrsDataProvider][INFO]: streamId 231-1/mic activated
[VrsDataProvider][INFO]: streamId 247-1/baro0 activated
[VrsDataProvider][WARNING]: Unsupported TimeSync mode: APP, ignoring.
[VrsDataProvider][INFO]: streamId 1201-1/camera-slam-left activated
[VrsDataProvider][INFO]: streamId 1201-2/camera-slam-right activated
[VrsDataProvider][INFO]: streamId 1202-1/imu-right activated
[VrsDataProvider][INFO]: streamId 1202-2/imu-left activated
[VrsDataProvider][INFO]: streamId 1203-1/mag0 activated
MoviePy - Writing audio in /tmp/tmp2kwm5dmb/audio.mp3
MoviePy - Done.
Moviepy - Building video /tmp/tmp2kwm5dmb/video.mp4.
MoviePy - Writing audio in videoTEMP_MPY_wvf_snd.mp3
MoviePy - Done.
Moviepy - Writing video /tmp/tmp2kwm5dmb/video.mp4

t:  47%|████████████▊              | 765/1613 [00:50<01:03, 13.38it/s, now=None]
Moviepy - Done !
Moviepy - video ready /tmp/tmp2kwm5dmb/video.mp4
num_mp4_frames : 1613
down_sampling_factor_ : 1
num_skipped_frames : 0
num_duplicated_frames_ : 2
num_invalid_frames_ : 0
first_video_timestamp_ns : 3388420069433
end_video_timestamp_ns : 3442180048449
video_duration_ns : 53759979016
Running: `ffmpeg -i /tmp/tmp2kwm5dmb/video.mp4 -i /tmp/tmp1obaf1_1/description-metadata.txt -map 0 -map_metadata 1 -codec copy rgb_camera.mp4 -y`
Traceback (most recent call last):
  File "scripts/aria_export_videos.py", line 19, in export_rgb_video_utility
    convert_vrs_to_mp4(aria_vrs_file, output_video)
  File "/miniconda3/envs/aria/lib/python3.11/site-packages/projectaria_tools/utils/vrs_to_mp4_utils.py", line 170, in convert_vrs_to_mp4
    save_timestamp_to_mp4(temp_video_file, output_video, vrs_device_time_ns_array)
  File "/miniconda3/envs/aria/lib/python3.11/site-packages/projectaria_tools/utils/vrs_to_mp4_utils.py", line 107, in save_timestamp_to_mp4
    raise ValueError(
ValueError: ffmpeg failed with error code 1 and stderr: 
ffmpeg version 4.2.7-0ubuntu0.1 Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.4.0-1ubuntu1~20.04.1)
  configuration: --prefix=/usr --extra-version=0ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-nvenc --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/tmp/tmp2kwm5dmb/video.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.29.100
  Duration: 00:00:53.81, start: 0.000000, bitrate: 4468 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1408x1408, 4336 kb/s, 30 fps, 30 tbr, 15360 tbn, 60 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: mp3 (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 127 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
/tmp/tmp1obaf1_1/description-metadata.txt: Invalid data found when processing input

Process finished with exit code 1

All pip packages:

Package                   Version
------------------------- --------------
anyio                     4.3.0
argon2-cffi               23.1.0
argon2-cffi-bindings      21.2.0
arrow                     1.3.0
asttokens                 2.4.1
async-lru                 2.0.4
attrs                     23.2.0
Babel                     2.14.0
beautifulsoup4            4.12.3
bleach                    6.1.0
certifi                   2024.2.2
cffi                      1.16.0
charset-normalizer        3.3.2
comm                      0.2.2
contourpy                 1.2.0
cycler                    0.12.1
debugpy                   1.8.1
decorator                 4.4.2
defusedxml                0.7.1
executing                 2.0.1
fastjsonschema            2.19.1
fonttools                 4.50.0
fqdn                      1.5.1
h11                       0.14.0
httpcore                  1.0.4
httpx                     0.27.0
idna                      3.6
imageio                   2.34.0
imageio-ffmpeg            0.4.9
ipykernel                 6.29.3
ipython                   8.22.2
ipywidgets                8.1.2
isoduration               20.11.0
jedi                      0.19.1
Jinja2                    3.1.3
json5                     0.9.24
jsonpointer               2.4
jsonschema                4.21.1
jsonschema-specifications 2023.12.1
jupyter                   1.0.0
jupyter_client            8.6.1
jupyter-console           6.6.3
jupyter_core              5.7.2
jupyter-events            0.10.0
jupyter-lsp               2.2.4
jupyter_server            2.13.0
jupyter_server_terminals  0.5.3
jupyterlab                4.1.5
jupyterlab_pygments       0.3.0
jupyterlab_server         2.25.4
jupyterlab_widgets        3.0.10
kiwisolver                1.4.5
MarkupSafe                2.1.5
matplotlib                3.8.3
matplotlib-inline         0.1.6
mistune                   3.0.2
moviepy                   1.0.3
nbclient                  0.10.0
nbconvert                 7.16.3
nbformat                  5.10.3
nest-asyncio              1.6.0
notebook                  7.1.2
notebook_shim             0.2.4
numpy                     1.26.4
overrides                 7.7.0
packaging                 24.0
pandas                    2.2.1
pandocfilters             1.5.1
parso                     0.8.3
pexpect                   4.9.0
pillow                    10.2.0
pip                       23.3.1
platformdirs              4.2.0
plotly                    5.20.0
proglog                   0.1.10
projectaria-tools         1.5.2
prometheus_client         0.20.0
prompt-toolkit            3.0.43
psutil                    5.9.8
ptyprocess                0.7.0
pure-eval                 0.2.2
pyarrow                   15.0.2
pycparser                 2.21
pyee                      11.1.0
Pygments                  2.17.2
pyparsing                 3.1.2
python-dateutil           2.9.0.post0
python-ffmpeg             2.0.12
python-json-logger        2.0.7
pytz                      2024.1
PyYAML                    6.0.1
pyzmq                     25.1.2
qtconsole                 5.5.1
QtPy                      2.4.1
referencing               0.34.0
requests                  2.31.0
rerun-sdk                 0.14.1
rfc3339-validator         0.1.4
rfc3986-validator         0.1.1
rpds-py                   0.18.0
scipy                     1.12.0
Send2Trash                1.8.2
setuptools                68.2.2
six                       1.16.0
sniffio                   1.3.1
soupsieve                 2.5
stack-data                0.6.3
tenacity                  8.2.3
terminado                 0.18.1
tinycss2                  1.2.1
tornado                   6.4
tqdm                      4.66.2
traitlets                 5.14.2
types-python-dateutil     2.9.0.20240316
typing_extensions         4.10.0
tzdata                    2024.1
uri-template              1.3.0
urllib3                   2.2.1
wcwidth                   0.2.13
webcolors                 1.13
webencodings              0.5.1
websocket-client          1.7.0
wheel                     0.41.2
widgetsnbextension        4.0.10
chpeng-fb commented 1 month ago

Hi,

You should be able to run this with the latest version of projectaria-tools, which is 1.5.4.