search

ehendrix23 / tesla_dashcam

Convert Tesla dash cam movie files into one movie
Apache License 2.0
657 stars 81 forks source link

ffmpeg errors after 0.1.10 to 0.1.12 #62

Closed cabbey closed 5 years ago

cabbey commented 5 years ago

This maybe related to #51 but there are enough details that differ that I think it's worth posting it separately.

System setup is as follows:

Machine 1 has the SMB folder mounted and I have opened a terminal window and cd'd into the share, as such, $CWD is /run/user/1000/gvfs/smb-share:server=machine2.local,share=storage/TeslaCam. In that terminal I run tesla_dashcam as such: tesla_dashcam --merge --compression veryslow --quality HIGH --output Processed/ SavedClips/. Previously, running 0.1.10 I was able to process the videos successfully with this configuration. Tonight I ran for the first time in a couple months (finally filled the new, larger, USB drive I got after they flagged the old one as too slow.) and was informed there was a new build of tesla_dashcam. So I updated to 0.1.12, and now when I run as above I get this error:

    Processing 6 clips in folder SavedClips/2019-05-06_18-37-50 (3/238)
        Processing clip 1/6 from 05/06/19 18:30:40 and 59 seconds long.
        Processing clip 2/6 from 05/06/19 18:31:41 and 59 seconds long.
        Processing clip 3/6 from 05/06/19 18:32:41 and 59 seconds long.
        Processing clip 4/6 from 05/06/19 18:33:41 and 59 seconds long.
        Processing clip 5/6 from 05/06/19 18:34:42 and 59 seconds long.
        Processing clip 6/6 from 05/06/19 18:36:43 and 13 seconds long.
        Creating movie Processed/2019-05-06T18-30-40_2019-05-06T18-36-56.mp4, please be patient.
        Error trying to create movie Processed/2019-05-06T18-30-40_2019-05-06T18-36-56.mp4. RC: 1
        Command: ['ffmpeg', '-f', 'concat', '-safe', '0', '-i', '/tmp/tmpr00thmma.txt', '-i', '/tmp/tmpypqx88qr.txt', '-map_metadata', '1', '-map_chapters', '1', '-movflags', '+faststart', '-c', 'copy', '-y', 'Processed/2019-05-06T18-30-40_2019-05-06T18-36-56.mp4']
        Error: b"ffmpeg version 4.1.3-0ubuntu1 Copyright (c) 2000-2019 the FFmpeg developers\n  built with gcc 8 (Ubuntu 8.3.0-6ubuntu1)\n  configuration: --prefix=/usr --extra-version=0ubuntu1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared\n  libavutil      56. 22.100 / 56. 22.100\n  libavcodec     58. 35.100 / 58. 35.100\n  libavformat    58. 20.100 / 58. 20.100\n  libavdevice    58.  5.100 / 58.  5.100\n  libavfilter     7. 40.101 /  7. 40.101\n  libavresample   4.  0.  0 /  4.  0.  0\n  libswscale      5.  3.100 /  5.  3.100\n  libswresample   3.  3.100 /  3.  3.100\n  libpostproc    55.  3.100 / 55.  3.100\n[concat @ 0x55971b2ecb40] Impossible to open '/tmp/Processed/2019-05-06_18-31.mp4'\n/tmp/tmpr00thmma.txt: No such file or directory\n"
ehendrix23 commented 5 years ago

Been able to reproduce issue. Problem is that when concatenating ffmpeg, the output location because it is relative ends up being based on the location of. the input text files and not on your location.

For your output use an absolute path for now, not a relative path. Hence if you mount it on /run/user/1000/gvfs/smb-share then your command would be: tesla_dashcam --merge --compression veryslow --quality HIGH --output /run/user/1000/gvfs/smb-share/Processed/ SavedClips/

Let me know if that works. I'll put a fix in to change the relative path to an absolute path for next version. But above should allow you to use it till then.

FYI, tesla_dashcam also works on Mac and I do provide an executable for it. Just in case you were not aware. :-)

cabbey commented 5 years ago

That doesn't seem to help any...

cabbey@cheshire:/run/user/1000/gvfs/smb-share:server=machine2.local,share=storage/TeslaCam$ tesla_dashcam --merge --compression veryslow --quality HIGH --output `pwd`/Processed SavedClips/2019-09-02_14-31-00/
Discovered 1 folders containing total of 33 files, retrieving clip data.
There are 1 folders with 11 clips to process.
    Processing 11 clips in folder SavedClips/2019-09-02_14-31-00 (1/1)
        Processing clip 1/11 from 09/02/19 14:20:45 and 59 seconds long.
        Processing clip 2/11 from 09/02/19 14:21:46 and 59 seconds long.
        Processing clip 3/11 from 09/02/19 14:22:46 and 59 seconds long.
        Processing clip 4/11 from 09/02/19 14:23:46 and 59 seconds long.
        Processing clip 5/11 from 09/02/19 14:24:46 and 59 seconds long.
        Processing clip 6/11 from 09/02/19 14:25:47 and 59 seconds long.
        Processing clip 7/11 from 09/02/19 14:26:47 and 59 seconds long.
        Processing clip 8/11 from 09/02/19 14:27:47 and 59 seconds long.
        Processing clip 9/11 from 09/02/19 14:28:48 and 59 seconds long.
        Processing clip 10/11 from 09/02/19 14:29:48 and 59 seconds long.
        Processing clip 11/11 from 09/02/19 14:30:48 and 8 seconds long.
        Creating movie /run/user/1000/gvfs/smb-share:server=machine2.local,share=storage/TeslaCam/Processed/2019-09-02T14-20-45_2019-09-02T14-30-56.mp4, please be patient.
    Movie /run/user/1000/gvfs/smb-share:server=machine2.local,share=storage/TeslaCam/Processed/2019-09-02T14-20-45_2019-09-02T14-30-56.mp4 for folder SavedClips/2019-09-02_14-31-00 with duration 0:00:00 is ready.
    Creating movie /run/user/1000/gvfs/smb-share:server=machiine2.local,share=storage/TeslaCam/Processed/Processed.mp4, please be patient.
        Error trying to create movie /run/user/1000/gvfs/smb-share:server=machine2.local,share=storage/TeslaCam/Processed/Processed.mp4. RC: 1
        Command: ['ffmpeg', '-f', 'concat', '-safe', '0', '-i', '/tmp/tmp00_jr5jj.txt', '-i', '/tmp/tmpeey8x3po.txt', '-map_metadata', '1', '-map_chapters', '1', '-movflags', '+faststart', '-c', 'copy', '-y', '/run/user/1000/gvfs/smb-share:server=machine2.local,share=storage/TeslaCam/Processed/Processed.mp4']
        Error: b"ffmpeg version 4.1.3-0ubuntu1 Copyright (c) 2000-2019 the FFmpeg developers\n  built with gcc 8 (Ubuntu 8.3.0-6ubuntu1)\n  configuration: --prefix=/usr --extra-version=0ubuntu1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared\n  libavutil      56. 22.100 / 56. 22.100\n  libavcodec     58. 35.100 / 58. 35.100\n  libavformat    58. 20.100 / 58. 20.100\n  libavdevice    58.  5.100 / 58.  5.100\n  libavfilter     7. 40.101 /  7. 40.101\n  libavresample   4.  0.  0 /  4.  0.  0\n  libswscale      5.  3.100 /  5.  3.100\n  libswresample   3.  3.100 /  3.  3.100\n  libpostproc    55.  3.100 / 55.  3.100\n[mov,mp4,m4a,3gp,3g2,mj2 @ 0x5565a3d00500] moov atom not found\n[concat @ 0x5565a3cf6b40] Impossible to open '/run/user/1000/gvfs/smb-share:server=machine2.local,share=storage/TeslaCam/Processed/2019-09-02T14-20-45_2019-09-02T14-30-56.mp4'\n/tmp/tmp00_jr5jj.txt: Invalid data found when processing input\n"

On the off chance that it was the commas, colon, or equal signs in the path and some kind of quoting problem with the command line being built, I created a symlink in my home dir that points out to that path and tried using that instead:

cabbey@cheshire:~/test$ pwd
/home/cabbey/test
cabbey@cheshire:~/test$ ls
Processed  RecentClips  SavedClips
cabbey@cheshire:~/test$ /bin/pwd
/run/user/1000/gvfs/smb-share:server=machine2.local,share=storage/TeslaCam
cabbey@cheshire:~/test$ tesla_dashcam --merge --compression veryslow --quality HIGH --output `pwd`/Processed SavedClips/2019-09-02_14-31-00/
Discovered 1 folders containing total of 33 files, retrieving clip data.
There are 1 folders with 11 clips to process.
    Processing 11 clips in folder SavedClips/2019-09-02_14-31-00 (1/1)
        Processing clip 1/11 from 09/02/19 14:20:45 and 59 seconds long.
        Processing clip 2/11 from 09/02/19 14:21:46 and 59 seconds long.
        Processing clip 3/11 from 09/02/19 14:22:46 and 59 seconds long.
        Processing clip 4/11 from 09/02/19 14:23:46 and 59 seconds long.
        Processing clip 5/11 from 09/02/19 14:24:46 and 59 seconds long.
        Processing clip 6/11 from 09/02/19 14:25:47 and 59 seconds long.
        Processing clip 7/11 from 09/02/19 14:26:47 and 59 seconds long.
        Processing clip 8/11 from 09/02/19 14:27:47 and 59 seconds long.
        Processing clip 9/11 from 09/02/19 14:28:48 and 59 seconds long.
        Processing clip 10/11 from 09/02/19 14:29:48 and 59 seconds long.
        Processing clip 11/11 from 09/02/19 14:30:48 and 8 seconds long.
        Creating movie /home/cabbey/test/Processed/2019-09-02T14-20-45_2019-09-02T14-30-56.mp4, please be patient.
    Movie /home/cabbey/test/Processed/2019-09-02T14-20-45_2019-09-02T14-30-56.mp4 for folder SavedClips/2019-09-02_14-31-00 with duration 0:00:00 is ready.
    Creating movie /home/cabbey/test/Processed/Processed.mp4, please be patient.
        Error trying to create movie /home/cabbey/test/Processed/Processed.mp4. RC: 1
        Command: ['ffmpeg', '-f', 'concat', '-safe', '0', '-i', '/tmp/tmp59rq3blq.txt', '-i', '/tmp/tmphokyoy1u.txt', '-map_metadata', '1', '-map_chapters', '1', '-movflags', '+faststart', '-c', 'copy', '-y', '/home/cabbey/test/Processed/Processed.mp4']
        Error: b"ffmpeg version 4.1.3-0ubuntu1 Copyright (c) 2000-2019 the FFmpeg developers\n  built with gcc 8 (Ubuntu 8.3.0-6ubuntu1)\n  configuration: --prefix=/usr --extra-version=0ubuntu1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared\n  libavutil      56. 22.100 / 56. 22.100\n  libavcodec     58. 35.100 / 58. 35.100\n  libavformat    58. 20.100 / 58. 20.100\n  libavdevice    58.  5.100 / 58.  5.100\n  libavfilter     7. 40.101 /  7. 40.101\n  libavresample   4.  0.  0 /  4.  0.  0\n  libswscale      5.  3.100 /  5.  3.100\n  libswresample   3.  3.100 /  3.  3.100\n  libpostproc    55.  3.100 / 55.  3.100\n[mov,mp4,m4a,3gp,3g2,mj2 @ 0x560cd814e4c0] moov atom not found\n[concat @ 0x560cd8144b40] Impossible to open '/home/cabbey/test/Processed/2019-09-02T14-20-45_2019-09-02T14-30-56.mp4'\n/tmp/tmp59rq3blq.txt: Invalid data found when processing input\n"

And yeah, I know it runs on macOS... I started using it there originally. It's just that machine 2 is an old mac mini, the second intel based one, that runs several server processes and averages about 45% cpu usage baseline... whereas machine 1 is my very new workstation, running a 36 core i9 with a TB of ram and a GPU that ffmpeg can offload encode/decode to that's doing nothing else. Even with copying the files down and back up twice, it's significantly faster to process them on this machine. :) (to that double network trip end, an option to specify a path for the intermediary files to live on would be a great addition... I'd put them on a local disk in /tmp, or maybe a ramdisk.)

cabbey commented 5 years ago

And just to be sure I hadn't done something wonky... I just copied several folders worth of SavedClips local to my filesystem, processed them with the same command invocation, and got perfectly working output files.

cabbey@cheshire:~/local$ tesla_dashcam  --merge --compression veryslow --quality HIGH --output `pwd`/Processed SavedClips/2019-09-02_14-31-00/ 
Discovered 1 folders containing total of 33 files, retrieving clip data.
There are 1 folders with 11 clips to process.
    Processing 11 clips in folder SavedClips/2019-09-02_14-31-00 (1/1)
        Processing clip 1/11 from 09/02/19 14:20:45 and 59 seconds long.
        Processing clip 2/11 from 09/02/19 14:21:46 and 59 seconds long.
        Processing clip 3/11 from 09/02/19 14:22:46 and 59 seconds long.
        Processing clip 4/11 from 09/02/19 14:23:46 and 59 seconds long.
        Processing clip 5/11 from 09/02/19 14:24:46 and 59 seconds long.
        Processing clip 6/11 from 09/02/19 14:25:47 and 59 seconds long.
        Processing clip 7/11 from 09/02/19 14:26:47 and 59 seconds long.
        Processing clip 8/11 from 09/02/19 14:27:47 and 59 seconds long.
        Processing clip 9/11 from 09/02/19 14:28:48 and 59 seconds long.
        Processing clip 10/11 from 09/02/19 14:29:48 and 59 seconds long.
        Processing clip 11/11 from 09/02/19 14:30:48 and 8 seconds long.
        Creating movie /home/cabbey/local/Processed/2019-09-02T14-20-45_2019-09-02T14-30-56.mp4, please be patient.
    Movie /home/cabbey/local/Processed/2019-09-02T14-20-45_2019-09-02T14-30-56.mp4 for folder SavedClips/2019-09-02_14-31-00 with duration 0:10:02 is ready.
    Creating movie /home/cabbey/local/Processed/Processed.mp4, please be patient.
Movie /home/cabbey/local/Processed/Processed.mp4 with duration 0:10:02 has been created, enjoy.
Total processing time: 0:02:21
ehendrix23 commented 5 years ago

It did work, but looks like you ran into another issue. First time (with relative path) it did not create the movie that concatenates all the clips from 1 folder together.

Now with giving the absolute path (even in your 1st try) it does look like it was able to do the concatenation of all the clips from 1 folder. But when it then went on to create the merged movie (which would put different folders together into 1 video file) failed.

In there it is stating it was unable to open file ....storage/TeslaCam/Processed/2019-09-02T14-20-45_2019-09-02T14-30-56.mp4 Same issue when you were using the link.

If you look in your Processed folder after running the ones on the SMB share you should see the file 2019-09-02T14-20-45_2019-09-02T14-30-56.mp4 there as it was created. Or if you run without --merge you shouldn't see the error.

Not sure right now why ffmpeg is stating it is impossible to open that file though. Can you check permissions once that file is created?

Part of me thinks a bug resulting in the folder file (2019-09-02T14-20-45_2019-09-02T14-30-56.mp4) being removed before we create the merged Processed.mp4 file. But then if that is the case it would not work local either. Hence that would not be the case.

Other option is that the permissions on the file we create results in not being able to read it again. That would make more sense but normally speaking with SMB permissions are based on who mounts it and clearly we're able to read from the existing files on the SMB and write to it.

Clearly it works with providing the absolute path, even without using a link as it created the concatenated movie for that folder. So check and let me know if the file 2019-09-02T14-20-45_2019-09-02T14-30-56.mp4 exist and if permissions are set allowing us to read it.

ehendrix23 commented 5 years ago

On another note, not 100% sure if ffmpeg would be using the GPU. If you have an NVIDIA video card then using the parameter --gpu would h264_nvenc encoder resulting in GPU. You can also specify your own encoder using the --enc parameter in case you have something else then NVIDIA but there is an encoder for FFMPEG out there to allow it to leverage your GPU.

cabbey commented 5 years ago

hmm... yeah, I missed that line. It is creating a file, and it's large and accessible... but none of my video players can play it.

cabbey@cheshire:/run/user/1000/gvfs/smb-share:server=alice.local,share=darkstar/TeslaCam$ ls -lrt Processed/2019-09-02T14-20-45_2019-09-02T14-30-56.mp4 
-rwx------ 1 cabbey cabbey 63334341 Sep  4 21:20 Processed/2019-09-02T14-20-45_2019-09-02T14-30-56.mp4
cabbey@cheshire:/run/user/1000/gvfs/smb-share:server=alice.local,share=darkstar/TeslaCam$ ffmpeg -i Processed/2019-09-02T14-20-45_2019-09-02T14-30-56.mp4 -hide_banner
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x55b93b3b1840] moov atom not found
Processed/2019-09-02T14-20-45_2019-09-02T14-30-56.mp4: Invalid data found when processing input

And that explains the weird error about moov atom not found! I just copied the same source files local and ran the same command over them to generate locally... 

cabbey@cheshire:~/local$ tesla_dashcam --merge --compression veryslow --quality HIGH --output `pwd`/Processed/ `pwd`/SavedClips/2019-09-02_14-31-00/
Discovered 1 folders containing total of 33 files, retrieving clip data.
There are 1 folders with 11 clips to process.
    Processing 11 clips in folder /home/cabbey/local/SavedClips/2019-09-02_14-31-00 (1/1)
        Processing clip 1/11 from 09/02/19 14:20:45 and 59 seconds long.
        Processing clip 2/11 from 09/02/19 14:21:46 and 59 seconds long.
        Processing clip 3/11 from 09/02/19 14:22:46 and 59 seconds long.
        Processing clip 4/11 from 09/02/19 14:23:46 and 59 seconds long.
        Processing clip 5/11 from 09/02/19 14:24:46 and 59 seconds long.
        Processing clip 6/11 from 09/02/19 14:25:47 and 59 seconds long.
        Processing clip 7/11 from 09/02/19 14:26:47 and 59 seconds long.
        Processing clip 8/11 from 09/02/19 14:27:47 and 59 seconds long.
        Processing clip 9/11 from 09/02/19 14:28:48 and 59 seconds long.
        Processing clip 10/11 from 09/02/19 14:29:48 and 59 seconds long.
        Processing clip 11/11 from 09/02/19 14:30:48 and 8 seconds long.
        Creating movie /home/cabbey/local/Processed/2019-09-02T14-20-45_2019-09-02T14-30-56.mp4, please be patient.
    Movie /home/cabbey/local/Processed/2019-09-02T14-20-45_2019-09-02T14-30-56.mp4 for folder /home/cabbey/local/SavedClips/2019-09-02_14-31-00 with duration 0:10:02 is ready.
    Creating movie /home/cabbey/local/Processed/Processed.mp4, please be patient.
Movie /home/cabbey/local/Processed/Processed.mp4 with duration 0:10:02 has been created, enjoy.
Total processing time: 0:02:22

cabbey@cheshire:~/local$ ls -lrt Processed/2019-09-02T14-20-45_2019-09-02T14-30-56.mp4 
-rw-r--r-- 1 cabbey cabbey 63491253 Sep  4 22:28 Processed/2019-09-02T14-20-45_2019-09-02T14-30-56.mp4
cabbey@cheshire:~/local$ ffmpeg -i Processed/2019-09-02T14-20-45_2019-09-02T14-30-56.mp4 -hide_banner
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'Processed/2019-09-02T14-20-45_2019-09-02T14-30-56.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.20.100
  Duration: 00:10:02.84, start: 0.000000, bitrate: 842 kb/s
    Chapter #0:0: start 0.000000, end 59.440000
    Metadata:
      title           : 09/02/19 14:20:45
    Chapter #0:1: start 59.440000, end 118.720000
    Metadata:
      title           : 09/02/19 14:21:46
    Chapter #0:2: start 118.720000, end 178.160000
    Metadata:
      title           : 09/02/19 14:22:46
    Chapter #0:3: start 178.160000, end 237.680000
    Metadata:
      title           : 09/02/19 14:23:46
    Chapter #0:4: start 237.680000, end 297.160000
    Metadata:
      title           : 09/02/19 14:24:46
    Chapter #0:5: start 297.160000, end 356.520000
    Metadata:
      title           : 09/02/19 14:25:47
    Chapter #0:6: start 356.520000, end 415.800000
    Metadata:
      title           : 09/02/19 14:26:47
    Chapter #0:7: start 415.800000, end 475.480000
    Metadata:
      title           : 09/02/19 14:27:47
    Chapter #0:8: start 475.480000, end 534.800000
    Metadata:
      title           : 09/02/19 14:28:48
    Chapter #0:9: start 534.800000, end 594.160000
    Metadata:
      title           : 09/02/19 14:29:48
    Chapter #0:10: start 594.160000, end 602.840000
    Metadata:
      title           : 09/02/19 14:30:48
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1280x960 [SAR 1:1 DAR 4:3], 840 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(eng): Data: bin_data (text / 0x74786574), 0 kb/s
    Metadata:
      handler_name    : SubtitleHandler
At least one output file must be specified

That's bizarre. The interim files with the three cameras stitched together didn't have that issue. And copying the file locally doesn't fix it... it seems to have gotten corrupted at generation.

Or if you run without --merge you shouldn't see the error.

Hmm... perhaps I didn't understand the point of that command then. I thought that was needed to merge the several 1 minute timestamp files into single file for the entire saved clip?

On another note, not 100% sure if ffmpeg would be using the GPU.

Yeah, it isn't yet... but it's on my todo list to get setup eventually, in the mean time, the CPU alone is plenty fast. :)

ehendrix23 commented 5 years ago

That's bizarre. The interim files with the three cameras stitched together didn't have that issue. And copying the file locally doesn't fix it... it seems to have gotten corrupted at generation.

OK, I might have an idea. If you edit the tesla_dashcam.py file and search for movflags. Now put a # in front of that one and the one below it which is +faststart

You would then end up with something like: # "-movflags", # "+faststart",

See if that works. Wondering if this is resulting in corrupt file due to it going over a network share.

Hmm... perhaps I didn't understand the point of that command then. I thought that was needed to merge the several 1 minute timestamp files into single file for the entire saved clip? The 1 minute timestamp files are always merged into 1 entire saved video. What the --merge option does it then merge saved videos from multiple folders into 1. Let's say there are 5 folders, without --merge you would have 5 video files and that's it. With --merge it would then merge those 5 video files into 1 large video file.

Yeah, it isn't yet... but it's on my todo list to get setup eventually, in the mean time, the CPU alone is plenty fast. :) Next version will include support for Intel GPU (if the Intel Hardware encoder supports that GPU). Currently NVIDIA is supported unless you provide the encoder using --enc. Yeah, sounds like you got yourself a pretty big system there. :-)

cabbey commented 5 years ago

OK, I might have an idea. If you edit the tesla_dashcam.py file and search for movflags. Now put a # in front of that one and the one below it which is +faststart

You would then end up with something like:

"-movflags",

"+faststart",

See if that works. Wondering if this is resulting in corrupt file due to it going over a network share.

I try not to edit "installed" apps... so I pulled git master and commented those two lines out, the resulting file appears to be correct.

abbey@cheshire:~/src/tesla_dashcam$ python3 tesla_dashcam/tesla_dashcam.py --compression veryslow --quality HIGH --output ~/test/Processed/ ~/test/SavedClips/2019-09-02_14-31-00/
Discovered 1 folders containing total of 33 files, retrieving clip data.
There are 1 folders with 11 clips to process.
    Processing 11 clips in folder /home/cabbey/test/SavedClips/2019-09-02_14-31-00 (1/1)
        Processing clip 1/11 from 09/02/19 14:20:45 and 59 seconds long.
        Processing clip 2/11 from 09/02/19 14:21:46 and 59 seconds long.
        Processing clip 3/11 from 09/02/19 14:22:46 and 59 seconds long.
        Processing clip 4/11 from 09/02/19 14:23:46 and 59 seconds long.
        Processing clip 5/11 from 09/02/19 14:24:46 and 59 seconds long.
        Processing clip 6/11 from 09/02/19 14:25:47 and 59 seconds long.
        Processing clip 7/11 from 09/02/19 14:26:47 and 59 seconds long.
        Processing clip 8/11 from 09/02/19 14:27:47 and 59 seconds long.
        Processing clip 9/11 from 09/02/19 14:28:48 and 59 seconds long.
        Processing clip 10/11 from 09/02/19 14:29:48 and 59 seconds long.
        Processing clip 11/11 from 09/02/19 14:30:48 and 8 seconds long.
        Creating movie /home/cabbey/test/Processed/2019-09-02T14-20-45_2019-09-02T14-30-56.mp4, please be patient.
    Movie /home/cabbey/test/Processed/2019-09-02T14-20-45_2019-09-02T14-30-56.mp4 for folder /home/cabbey/test/SavedClips/2019-09-02_14-31-00 with duration 0:10:02 is ready.
All folders have been processed, resulting movie files are located in /home/cabbey/test/Processed/
Total processing time: 0:02:29
cabbey@cheshire:~/src/tesla_dashcam$ ffmpeg -i ~/test/Processed/2019-09-02T14-20-45_2019-09-02T14-30-56.mp4 -hide_banner
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/home/cabbey/test/Processed/2019-09-02T14-20-45_2019-09-02T14-30-56.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.20.100
  Duration: 00:10:02.84, start: 0.000000, bitrate: 842 kb/s
    Chapter #0:0: start 0.000000, end 59.440000
    Metadata:
      title           : 09/02/19 14:20:45
    Chapter #0:1: start 59.440000, end 118.720000
    Metadata:
      title           : 09/02/19 14:21:46
    Chapter #0:2: start 118.720000, end 178.160000
    Metadata:
      title           : 09/02/19 14:22:46
    Chapter #0:3: start 178.160000, end 237.680000
    Metadata:
      title           : 09/02/19 14:23:46
    Chapter #0:4: start 237.680000, end 297.160000
    Metadata:
      title           : 09/02/19 14:24:46
    Chapter #0:5: start 297.160000, end 356.520000
    Metadata:
      title           : 09/02/19 14:25:47
    Chapter #0:6: start 356.520000, end 415.800000
    Metadata:
      title           : 09/02/19 14:26:47
    Chapter #0:7: start 415.800000, end 475.480000
    Metadata:
      title           : 09/02/19 14:27:47
    Chapter #0:8: start 475.480000, end 534.800000
    Metadata:
      title           : 09/02/19 14:28:48
    Chapter #0:9: start 534.800000, end 594.160000
    Metadata:
      title           : 09/02/19 14:29:48
    Chapter #0:10: start 594.160000, end 602.840000
    Metadata:
      title           : 09/02/19 14:30:48
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1280x960 [SAR 1:1 DAR 4:3], 840 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(eng): Data: bin_data (text / 0x74786574), 0 kb/s
    Metadata:
      handler_name    : SubtitleHandler
At least one output file must be specified

So unless there is something else on master that isn't in 0.1.12 that might have impacted this, it does look like that option may have an adverse impact on network mounted files.

Yeah, sounds like you got yourself a pretty big system there. :-)

I normally use it for fluid dynamic simulations and 3D modeling... so yeah, I built it as big as I could realistically afford. :)

ehendrix23 commented 5 years ago

So unless there is something else on master that isn't in 0.1.12 that might have impacted this, it does look like that option may have an adverse impact on network mounted files.

The master is exactly same as 0.1.12; all my development happens in the dev branch and I only update the master for a new release. Hence that is the issue.

I'll be adding a parameter (most likely --no_faststart) which would then not add that parameter when creating the files. That way I keep same behavior as in 0.1.12, but if someone has an issue with it due to using a network share or so then using that parameter it can be fixed.

I normally use it for fluid dynamic simulations and 3D modeling... so yeah, I built it as big as I could realistically afford. :)

K, now I have to ask. What do you do? :-)

cabbey commented 5 years ago

I'll be adding a parameter (most likely --no_faststart) which would then not add that parameter when creating the files. That way I keep same behavior as in 0.1.12, but if someone has an issue with it due to using a network share or so then using that parameter it can be fixed.

K, so if I'm keeping count correctly we've identified two changes from this issue then:

  1. use full paths when assembling commands for ffmpeg
  2. skip fast start on network devices... easiest to just provide a flag and some guidance to users to use it in this scenario.

K, now I have to ask. What do you do? :-)

heh, I'm a Sr. Engineer/Architect at a web company... but this is my hobby machine... high power rocketry and 3D printing. (often both at the same time.)

ehendrix23 commented 5 years ago

Issue 1 was already fixed in my dev branch. Just added the option --no-start for issue 2 in my dev branch as well.

cabbey commented 5 years ago

Just pulled the dev branch and validated both fixes are working great for me. Thanks!

ehendrix23 commented 4 years ago

Thank you.

Release 0.1.14 is out now including this enhancement

hitnrun30 commented 4 years ago

I'm still getting this error. here are my options in my preferences file

--temp_dir "E:\TeslaCam\temp" --skip_existing

--perspective

--delete_source

--no-swap --rear --motion_only --layout FULLSCREEN --no-faststart --swap_frontrear --gpu --gpu_type nvidia --quality MEDIUM --ffmpeg "C:\HTPC\ffmpeg\bin\ffmpeg.exe" --encoding x265

Here is my command line: tesla_dashcam.exe --output "\backup\TeslaCam\SentryClips\2020-02-04_16-23-08\2020-02-04_16-23-08.mp4" "@C:\HTPC\Tesla DashCam\preferences.txt" "\backup\TeslaCam\SentryClips\2020-02-04_16-23-08"

And here is the error: Error trying to create movie \backup\TeslaCam\SentryClips\2020-02-04_16-23-08\2020-02-04T16-12-562020-02-04T16-22-57.mp4. RC: 1 Command: ['C:\HTPC\ffmpeg\bin\ffmpeg.exe', '-loglevel', 'error', '-f', 'concat', '-safe', '0', '-i', 'C:\Users\Phoenix\AppData\Local\Temp\tmpfvco494.txt', '-i', 'C:\Users\Phoenix\AppData\Local\Temp\tmp6xlimggx.txt', '-map_metadata', '1', '-map_chapters', '1', '-c', 'copy', '-metadata', 'description=Created by tesla_dashcam v0.1.16', '-y', '\\backup\TeslaCam\SentryClips\2020-02-04_16-23-08\2020-02-04T16-12-56_2020-02-04T16-22-57.mp4'] Error: b'C:\Users\Phoenix\AppData\Local\Temp\tmp6xlimggx.txt: Invalid data found when processing input\r\n'

ehendrix23 commented 4 years ago

Can you open a new issue for this?

Also run it with parameter --loglevel DEBUG and provide the output.

Thx.