Compiled ffmpeg is 2.5x faster than avconv

[NelsonPJ]

Running with the latest Raspbian Stretch on an RPi3, I was having issues with makevideo.sh occasionally creating video files that were unplayable. Running makevideo.sh manually, I noticed that avconv would sometimes crash after all frames were imported and the movie was being made - sometimes with "illegal instruction", sometimes with "segmentation fault", and other errors I don't remember. This leaves you with an mp4 file that is garbage.

I started investigating alternatives, and discovered that you can compile a GPU accelerated version of ffmpeg that is much faster than avconv included with Raspbian. Doing back-to-back runs on 2000 images of 1024x768, it took avconv 8 min 45 seconds to make a movie - but my compiled ffmpeg took only 3 min 20 seconds (over 2.5 times faster!)

I found instructions here - it took about a half hour to compile on my Raspberry Pi 3. https://github.com/legotheboss/YouTube-files/wiki/(RPi)-Compile-FFmpeg-with-the-OpenMAX-H.264-GPU-acceleration I'm not sure if this is the best way to do it - but it works for me.

I then went into makevideo and replaced this line: /usr/bin/avconv -y -f image2 -r $tl_fps -i $tl_folder_working/%5d.$tl_files_ext -aspect $tl_a_ratio -s $tl_vid_size $tl_folder_destination/$tl_videoname

...with this line ffmpeg -y -f image2 -r $tl_fps -i $tl_folder_working/%5d.$tl_files_ext -aspect $tl_a_ratio -s $tl_vid_size -vf format=yuv420p -vcodec mpeg4 -b:v 1500k $tl_folder_destination/$tl_videoname

The options I added were to select the format of yuv420p (which makes it more compatible with devices and browsers), the mpeg4 codec, and a bitrate of 1500 (this can be adjusted - higher numbers make bigger files - I found less than 1000 got blocky)

[pageauc]

Not sure this would work if camera is running since GPU would be busy with camera operation. Does ffmpeg with GPU support work when camera is running video or images?

[NelsonPJ]

I dug into this a bit more, and it turns out when I used the "mpeg4" codec with ffmpeg, that was not GPU accelerated. I ran another test using the "h264_omx" codec, and it only took 2 min 20 seconds (which is 3.7x faster than avconv!)

ffmpeg -y -f image2 -r $tl_fps -i $tl_folder_working/%5d.$tl_files_ext -aspect $tl_a_ratio -s $tl_vid_size -vf format=yuv420p -vcodec h264_omx -b:v 1500k $tl_folder_destination/$tl_videoname

I still need to do some tests walking in front of the camera while a video is encoding to see if motion detection still works - but then again, if the encode only takes 2 minutes, it may not be a big deal?

EDIT: I should point out that the "h264_omx" file is an h264 encoded video file (not MPEG-4) even though both files have mp4 file extensions. I'm able to play both files normally with both Windows and Mac computers - and also on my iPhone after syncing to Google Drive.

[NelsonPJ]

From a quick test of starting makemovie.sh with ffmpeg using the h264_omx codec, and then walking in front of the camera, motion detection activated and took a burst of photos similar to before.

I'm going to run with ffmpeg for the next few days and see if there are any issues.

[pageauc]

Give me the results of your testing and details about compiling source and I can certainly change pi-timolo to use something other than avconv. I was somewhat forced to use avconv when support for mencoder was stopped. Personally if I have a large timelapse I use avidmux on my PC after transferring files using memory stick or filezilla. The only problem is it needs consecutive numbered images so I use a file renaming utility that works OK once you figure it out.

Thanks for your testing. Also I have release version 10.95 that uses symlinks for recent and added motion video and quicktimelapse to recent. You can use menubox.sh to UPGRADE.

[NelsonPJ]

Here's my notes on compiling ffmpeg on a fresh install of Raspbian Stretch Lite on an RPi3 B+ I just got today, following instructions from: https://github.com/legotheboss/YouTube-files/wiki/(RPi)-Compile-FFmpeg-with-the-OpenMAX-H.264-GPU-acceleration

The entire process took just over 30 minutes on an RPi3 B+.

First step is to change to the home directory cd /home/pi/

Then install the OpenMAX package that allows access to hardware acceleration: sudo apt-get install libomxil-bellagio-dev -y

For my installation of Raspbian Stretch Lite, I needed to install git. It's possible other Raspbian "full" installations will already have git installed. sudo apt install git

Then download the latest FFmpeg release from GitHub. This will create a folder named "FFmpeg" git clone https://github.com/FFmpeg/FFmpeg.git

I suppose the above step could be replaced with a "wget" command, similar to how Pi-Timolo is installed, but I didn't try that.

Change into the new "FFmpeg" directory that was created above. cd FFmpeg

This next step configures the ffmpeg build, including options for hardware acceleration. This step can take a few minutes without anything changing on screen - just let it run. You will eventually see text feedback that it worked. sudo ./configure --arch=armel --target-os=linux --enable-gpl --enable-omx --enable-omx-rpi --enable-nonfree

Then if you have an RPi2 or RPi3, compile the configuration with this command. The "-j4" option enables all 4 cores for the compilation process, so it will take less time. You will see a lot of text scroll by, including a lot of warning messages, but you can ignore them. sudo make -j4 If you have anything else (The original RPi or an RPi Zero) run this command (since you only have a single core CPU): sudo make

The final step is to install ffmpeg with this command: sudo make install

After installation, you can check what version you have by typing "ffmpeg" at the command line:

pi@raspberry:~/FFmpeg $ ffmpeg
ffmpeg version N-90403-g4f2ff3a53e Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 6.3.0 (Raspbian 6.3.0-18+rpi1+deb9u1) 20170516
  configuration: --arch=armel --target-os=linux --enable-gpl --enable-omx --enable-omx-rpi --enable-nonfree
  libavutil      56. 11.100 / 56. 11.100
  libavcodec     58. 15.100 / 58. 15.100
  libavformat    58. 10.100 / 58. 10.100
  libavdevice    58.  2.100 / 58.  2.100
  libavfilter     7. 13.100 /  7. 13.100
  libswscale      5.  0.102 /  5.  0.102
  libswresample   3.  0.101 /  3.  0.101
  libpostproc    55.  0.100 / 55.  0.100
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...

[pageauc]

Thanks

I am thinking of allowing ffmpeg as an option as well as avconv. Thinking that will try to pre compile the setup as part of install script so users do not have to compile.

Claude ...

On Thu, Mar 22, 2018 at 3:40 PM, Peter Nelson notifications@github.com wrote:

Here's my notes on compiling ffmpeg on a fresh install of Raspbian Stretch Lite on an RPi3 B+ I just got today, following instructions from: https://github.com/legotheboss/YouTube-files/wiki/(RPi)-Compile-FFmpeg- with-the-OpenMAX-H.264-GPU-acceleration

The entire process took about 40 minutes on an RPi3 B+.

First step is to change to the home directory cd /home/pi/

Then install the OpenMAX package that allows access to hardware acceleration: sudo apt-get install libomxil-bellagio-dev -y

For my installation of Raspbian Stretch Lite, I needed to install git. It's possible other Raspbian "full" installations will already have git installed. sudo apt install git

Then download the latest FFmpeg release from GitHub. This will create a folder named "FFmpeg" git clone https://github.com/FFmpeg/FFmpeg.git

I suppose the above step could be replaced with a "wget" command, similar to how Pi-Timolo is installed, but I didn't try that.

Change into the new "FFmpeg" directory that was created above. cd FFmpeg

This next step configures the ffmpeg build, including options for hardware acceleration. This step can take a few minutes without anything changing on screen - just let it run. You will eventually see text feedback that it worked. sudo ./configure --arch=armel --target-os=linux --enable-gpl --enable-omx --enable-omx-rpi --enable-nonfree

Then if you have an RPi2 or RPi3, compile the configuration with this command. The "-j4" option enables all 4 cores for the compilation process, so it will take less time. You will see a lot of text scroll by, including a lot of warning messages, but you can ignore them. sudo make -j4 If you have anything else (The original RPi or an RPi Zero) run this command (since you only have a single core CPU): sudo make

The final step is to install ffmpeg with this command: sudo make install

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/pageauc/pi-timolo/issues/93#issuecomment-375450179, or mute the thread https://github.com/notifications/unsubscribe-auth/AFr1ZMKRQ4FgLLkM8LOAmU9rbyLl8SHbks5thAxFgaJpZM4Syu0m .

-- See my YouTube Channel at http://www.youtube.com/user/pageaucp

[mehair]

Now may be the time to implement that option of using ffmpeg. Yesterday I built two Pi Zero W, running the latest Raspbian Buster Lite, I had no issues other than I had to install "python-pil" in place of "python-imaging". The remaining issues I'm having is with the makevideo.sh script, it's unable to find /usr/bin/avconv.

Attempting to install it using "sudo apt-get install libav-tools" returns the following:

Reading package lists... Done Building dependency tree Reading state information... Done Package libav-tools is not available, but is referred to by another package. This may mean that the package is missing, has been obsoleted, or is only available from another source However the following packages replace it: ffmpeg

E: Package 'libav-tools' has no installation candidate

Installing ffmpeg doesn't contain avconv.

I'm new to this and I'm not sure the best way to fix the error.

Thanks, Michael

[pageauc]

I originally changed from ffmpeg to avconv when Raspbian changed over to avconv due to maintenance issues with ffmpeg. Since then ffmpeg has come back and is now better supported and looks like it is now replacing avconv. both have similar syntax. These are the problems maintaining code when the distro and underlying codebase and libraries change.

I will look at fixing the issues with PIL and change back to ffmpeg and also try to maintain some backward compatibility. I only have a few machines on Buster and did not experience install issues but then again I don't usually test for every option in pi-timolo.

Up until recently I still had a few RPI;s running Wheezy that is no longer supported. In fact I have some code to fix some pi-timolo Wheezy install issues like updating encryption for pip to get pi-timolo to install successfully. But life moves on ...... Thanks for the update. Claude ....

On Mon, Aug 26, 2019 at 10:14 PM mehair notifications@github.com wrote:

Now may be the time to implement that option of using ffmpeg. Yesterday I built two Pi Zero W, running the latest Raspbian Buster Lite, I had no issues other than I had to install "python-pil" in place of "python-imaging". The remaining issues I'm having is with the makevideo.sh script, it's unable to find /usr/bin/avconv.

Attempting to install it using "sudo apt-get install libav-tools" returns the following:

Reading package lists... Done Building dependency tree Reading state information... Done Package libav-tools is not available, but is referred to by another package. This may mean that the package is missing, has been obsoleted, or is only available from another source However the following packages replace it: ffmpeg

E: Package 'libav-tools' has no installation candidate

Installing ffmpeg doesn't contain avconv.

I'm new to this and I'm not sure the best way to fix the error.

Thanks, Michael

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/pageauc/pi-timolo/issues/93?email_source=notifications&email_token=ABNPKZGBHBL6UJ2257GNZZDQGSEZ5A5CNFSM4EWK5UTKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5GHXQQ#issuecomment-525106114, or mute the thread https://github.com/notifications/unsubscribe-auth/ABNPKZADX5736YFFLNRHCJLQGSEZ5ANCNFSM4EWK5UTA .

-- See my YouTube Channel at http://www.youtube.com/user/pageaucp

[pageauc]

I have update the pi-timolo-install.sh to add ffmpeg install. This will work for Stretch and Buster but not Jessie. Also I noticed ffmpeg is already installed on Buster by default but not sure if this was installed with one of the Raspbian upgrades. I will investigate detecting versions and add checks but in the mean time install will still show an error message for avconv otherwise should work OK

I have Updated makevideo.sh to look for ffmpeg and use it if available. Otherwise avconv is used. Would appreciate if you test and let me know if there are any problems since I left parameters the same and did no test since I was hoping you could assist

Installing ffmpeg on Jessie takes quite a few steps to compile but avconv should still work OK.

I have also included installing py3exiv2 in the install script. This is required for python3 to transfer exif data after image text is added. This is only relevant if you use exif image data.

You can test the new install script using the menubox.sh UPGRADE menu pick. and also test ffmpeg makevideo.sh

Thanks Claude ....

On Tue, Aug 27, 2019 at 6:17 AM Claude Pageau pageauc@gmail.com wrote:

I originally changed from ffmpeg to avconv when Raspbian changed over to avconv due to maintenance issues with ffmpeg. Since then ffmpeg has come back and is now better supported and looks like it is now replacing avconv. both have similar syntax. These are the problems maintaining code when the distro and underlying codebase and libraries change.

I will look at fixing the issues with PIL and change back to ffmpeg and also try to maintain some backward compatibility. I only have a few machines on Buster and did not experience install issues but then again I don't usually test for every option in pi-timolo.

Up until recently I still had a few RPI;s running Wheezy that is no longer supported. In fact I have some code to fix some pi-timolo Wheezy install issues like updating encryption for pip to get pi-timolo to install successfully. But life moves on ...... Thanks for the update. Claude ....

On Mon, Aug 26, 2019 at 10:14 PM mehair notifications@github.com wrote:

Now may be the time to implement that option of using ffmpeg. Yesterday I built two Pi Zero W, running the latest Raspbian Buster Lite, I had no issues other than I had to install "python-pil" in place of "python-imaging". The remaining issues I'm having is with the makevideo.sh script, it's unable to find /usr/bin/avconv.

Attempting to install it using "sudo apt-get install libav-tools" returns the following:

Reading package lists... Done Building dependency tree Reading state information... Done Package libav-tools is not available, but is referred to by another package. This may mean that the package is missing, has been obsoleted, or is only available from another source However the following packages replace it: ffmpeg

E: Package 'libav-tools' has no installation candidate

Installing ffmpeg doesn't contain avconv.

I'm new to this and I'm not sure the best way to fix the error.

Thanks, Michael

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/pageauc/pi-timolo/issues/93?email_source=notifications&email_token=ABNPKZGBHBL6UJ2257GNZZDQGSEZ5A5CNFSM4EWK5UTKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5GHXQQ#issuecomment-525106114, or mute the thread https://github.com/notifications/unsubscribe-auth/ABNPKZADX5736YFFLNRHCJLQGSEZ5ANCNFSM4EWK5UTA .

-- See my YouTube Channel at http://www.youtube.com/user/pageaucp

-- See my YouTube Channel at http://www.youtube.com/user/pageaucp

[mehair]

Thank you for the quick update. I used the UPGRADE option from menubox.sh as you suggested. It ran without error up until the py3exiv2 installation, which it's still sitting at (2 hours so far), no error or any other indications, so I'll let continue to run for now. Guess the Pi Zero W is a little under powered. I've ordered a Pi 4, which should be here by next week, I'll let you know how that works.

In the meantime here is that the screen shows:

Processing triggers for man-db (2.8.5-2) ... Reading package lists... Building dependency tree... Reading state information... g++ is already the newest version (4:8.3.0-1+rpi2). g++ set to manually installed. 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple Collecting py3exiv2 Downloading https://files.pythonhosted.org/packages/49/e0/f8b22cc5061559345a340747feb6f60369f5ba576155ef73009fe898278d/py3exiv2-0.7.1.tar.gz Building wheels for collected packages: py3exiv2 Running setup.py bdist_wheel for py3exiv2 ... /

Thanks Michael

[pageauc]

I compiles from source and it did run on my machine but did take a while 5-10 minutes. I put this in but will look at taking it back out since I have a separate script that installs this here https://github.com/pageauc/pi-timolo/wiki/Prerequisites#quick-install-py3exiv2

So if it does not finish try running the install from link above. Hopefully this library will be added as a python3 library in future since python2 is near end of life.

Let me know status

On Tue, Aug 27, 2019 at 11:03 PM mehair notifications@github.com wrote:

Thank you for the quick update. I used the UPGRADE option from menubox.sh as you suggested. It ran without error up until the py3exiv2 installation, which it's still sitting at (2 hours so far), no error or any other indications, so I'll let continue to run for now. Guess the Pi Zero W is a little under powered. I've ordered a Pi 4, which should be here by next week, I'll let you know how that works.

In the meantime here is that the screen shows:

Processing triggers for man-db (2.8.5-2) ... Reading package lists... Building dependency tree... Reading state information... g++ is already the newest version (4:8.3.0-1+rpi2). g++ set to manually installed. 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple Collecting py3exiv2 Downloading https://files.pythonhosted.org/packages/49/e0/f8b22cc5061559345a340747feb6f60369f5ba576155ef73009fe898278d/py3exiv2-0.7.1.tar.gz Building wheels for collected packages: py3exiv2 Running setup.py bdist_wheel for py3exiv2 ... /

Thanks Michael

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/pageauc/pi-timolo/issues/93?email_source=notifications&email_token=ABNPKZG2YMDNGYODVPGVRMTQGXTIJA5CNFSM4EWK5UTKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5JXTNI#issuecomment-525564341, or mute the thread https://github.com/notifications/unsubscribe-auth/ABNPKZEG6CDFGEU3CHRXAWTQGXTIJANCNFSM4EWK5UTA .

-- See my YouTube Channel at http://www.youtube.com/user/pageaucp

[mehair]

I continued to let it run however it never finished. I'll try the script you mentioned, but that might not happen until the weekend. Thanks for all your assistance.

[mehair]

I forgot to mention that I ran the makevideo.sh I got an error at line 208.

========================================================================== INFO : Video Saved to media/videos/TL-20190828-0613.mp4 INFO : Deleting Working Folder /home/pi/pi-timolo/makevideo_tmp ./makevideo.sh: line 208: syntax error near unexpected token fi' ./makevideo.sh: line 208:fi'

[pageauc]

I have removed extra fi problem. Have not tested but should be OK. Can you upgrade and retest makevideo.sh version 5.6. and let me know. Sorry for not testing but doing other things and appreciate your help Let me know how this works. Not sure if ffmpeg will give problem but hoping it will run without issue. BTW I think the zero has some issues that other pi models don't seem to have. Also might depend on minimal distro or full and weather aprt-get upgrade was run or not since it installs more packages... You should also make sure to run apt-get update before installing new software.

Claude ...

On Wed, Aug 28, 2019 at 9:59 AM mehair notifications@github.com wrote:

I forgot to mention that I ran the makevideo.sh I got an error at line 208.

========================================================================== INFO : Video Saved to media/videos/TL-20190828-0613.mp4 INFO : Deleting Working Folder /home/pi/pi-timolo/makevideo_tmp ./makevideo.sh: line 208: syntax error near unexpected token fi' ./makevideo.sh: line 208: fi'

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/pageauc/pi-timolo/issues/93?email_source=notifications&email_token=ABNPKZFRKFQQMBLDM3XOINDQG2ADHA5CNFSM4EWK5UTKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5LG6II#issuecomment-525758241, or mute the thread https://github.com/notifications/unsubscribe-auth/ABNPKZGX3CM7FGX6XPUH5XDQG2ADHANCNFSM4EWK5UTA .

-- See my YouTube Channel at http://www.youtube.com/user/pageaucp

[mehair]

Today before I started testing I did what you suggested and ran "sudo apt-get update" and "sudo apt-get upgrade" then rebooted the devices. On the Pi Zero W I still have the same issue while using the the UPGRADE options from menubox.sh it runs fine until the py3exiv2 process.

I decided to reboot the device and try the "install-py3exiv2.sh" script, it also never completes.

Opening up a few more terminal windows to troubleshoot, I rebooted and restarted the script again. This time I think I found the issues. The Pi Zero W runs out of memory and begins swapping. The Pi Zero W has half the memory as the Pi 2 and Pi 3.

pi@raspberrypi:~ $ free -m total used free shared buff/cache available Mem: 369 335 24 0 9 4 Swap: 99 99 0

pi@raspberrypi:~ $ vmstat procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu----- r b swpd free buff cache si so bi bo in cs us sy id wa st 0 3 102396 23780 156 10424 0 4 42 7 279 730 74 8 15 3 0

While the build.sh runs, and the temp file grows in size the system gets slower and slower until it stops processing, before it did so I was able to look at ps and watch the /tmp directory.

pi@raspberrypi:~ $ ps -ef | grep cc1 pi 18732 18731 11 10:28 pts/0 00:03:27 /usr/lib/gcc/arm-linux-gnueabihf/8/cc1plus -quiet -I /usr/include/python3.7m -imultilib . -imultiarch arm-linux-gnueabihf -D_GNU_SOURCE src/exiv2wrapper_python.cpp -quiet -dumpbase exiv2wrapper_python.cpp -mfloat-abi=hard -mfpu=vfp -mtls-dialect=gnu -marm -march=armv6+fp -auxbase-strip build/exiv2wrapper_python.os -fPIC -o /tmp/cccf7FBd.s

pi@raspberrypi:~ $ ls -l /tmp/cccf7FBd.s total 3416 -rw------- 1 pi pi 3493888 Sep 2 11:13 cccf7FBd.s

But eventually it uses all available memory and swap space until the system says:

-bash: fork: Cannot allocate memory

I'll go test the makevideo.sh script now and see if it the issues has been fixed.

Thanks Michael

[pageauc]

OK thanks for the update.

I could increase temporary swap space if memory is less than 1 gig just prior to the compile in both scripts. I have done this with my opencv3-isetup https://github.com/pageauc/opencv3-setup

see https://github.com/pageauc/opencv3-setup/blob/master/cv3-install-menu.sh

function do_Initialize () function do_swap_check () function do_swap_back ()

I don't have any zero's but I believe I have a few older RPI's with 512 memory and can do some testing on those. Glad you were able to trouble shoot. Not every user can do that. It stumped me because I could not reproduce your issue. Thanks a lot for your work.

Will let you know what I decide after I do some testing.

Claude ....

On Mon, Sep 2, 2019 at 3:17 PM mehair notifications@github.com wrote:

Today before I started testing I did what you suggested and ran "sudo apt-get update" and "sudo apt-get upgrade" then rebooted the devices. On the Pi Zero W I still have the same issue while using the the UPGRADE options from menubox.sh it runs fine until the py3exiv2 process.

I decided to reboot the device and try the "install-py3exiv2.sh" script, it also never completes.

Opening up a few more terminal windows to troubleshoot, I rebooted and restarted the script again. This time I think I found the issues. The Pi Zero W runs out of memory and begins swapping. The Pi Zero W has half the memory as the Pi 2 and Pi 3.

pi@raspberrypi:~ $ free -m total used free shared buff/cache available Mem: 369 335 24 0 9 4 Swap: 99 99 0

pi@raspberrypi:~ $ vmstat procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu----- r b swpd free buff cache si so bi bo in cs us sy id wa st 0 3 102396 23780 156 10424 0 4 42 7 279 730 74 8 15 3 0

While the build.sh runs, and the temp file grows in size the system gets slower and slower until it stops processing, before it did so I was able to look at ps and watch the /tmp directory.

pi@raspberrypi:~ $ ps -ef | grep cc1 pi 18732 18731 11 10:28 pts/0 00:03:27 /usr/lib/gcc/arm-linux-gnueabihf/8/cc1plus -quiet -I /usr/include/python3.7m -imultilib . -imultiarch arm-linux-gnueabihf -D_GNU_SOURCE src/exiv2wrapper_python.cpp -quiet -dumpbase exiv2wrapper_python.cpp -mfloat-abi=hard -mfpu=vfp -mtls-dialect=gnu -marm -march=armv6+fp -auxbase-strip build/exiv2wrapper_python.os -fPIC -o /tmp/cccf7FBd.s

pi@raspberrypi:~ $ ls -l /tmp/cccf7FBd.s total 3416 -rw------- 1 pi pi 3493888 Sep 2 11:13 cccf7FBd.s

But eventually it uses all available memory and swap space until the system says:

-bash: fork: Cannot allocate memory

I'll go test the makevideo.sh script now and see if it the issues has been fixed.

Thanks Michael

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/pageauc/pi-timolo/issues/93?email_source=notifications&email_token=ABNPKZE27X37YLWHJW4FUCTQHVRC5A5CNFSM4EWK5UTKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5WNWHQ#issuecomment-527227678, or mute the thread https://github.com/notifications/unsubscribe-auth/ABNPKZHCQ7NWJ3YP4BVQPJDQHVRC5ANCNFSM4EWK5UTA .

-- See my YouTube Channel at http://www.youtube.com/user/pageaucp

[mehair]

I finished testing the "makevideo.sh" script on the Pi Zero W, although extremely slow it finished without error. It literately takes hours to accomplish what a Pi 3 B can do in under 15 minutes, therefore I'll probably just use the Pi Zero W to just take the images, which it seems to do nicely.

I'm thinking of using a Pi 3 B as a central hub to manage multiple Pi Zero W remote camera's. That way I can off-load the "makevideo.sh" script as we'll as the web services. I'll have to look and see if I can modify your rclone functionality to allow me to do that easily, until then I might just use scp or rsync and a cron.

Using that same concept would it be possible to run the "install-py3exiv2.sh" on another device and then move Python 3 modules over?

Michael

[pageauc]

Yes you can compile py3exiv2 on another machine and move with same cpu architecture. Files should be

/usr/lib/python3/dist-packages/libexiv2python.so and directory /usr/lib/python3/dist-packages/pyexiv2

Make sure permissions and ownership are the same eg -rw-r--r-- :root:root

permissions would be 644

Let me know how this works out

Claude ...

Claude ...

On Mon, Sep 2, 2019 at 10:49 PM mehair notifications@github.com wrote:

I finished testing the "makevideo.sh" script on the Pi Zero W, although extremely slow it finished without error. It literately takes hours to accomplish what a Pi 3 B can do in under 15 minutes, therefore I'll probably just use the Pi Zero W to just take the images, which it seems to do nicely.

I'm thinking of using a Pi 3 B as a central hub to manage multiple Pi Zero W remote camera's. That way I can off-load the "makevideo.sh" script as we'll as the web services. I'll have to look and see if I can modify your rclone functionality to allow me to do that easily, until then I might just use scp or rsync and a cron.

Using that same concept would it be possible to run the "install-py3exiv2.sh" on another device and then move Python 3 modules over?

Michael

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/pageauc/pi-timolo/issues/93?email_source=notifications&email_token=ABNPKZDW3JVV7K6VYJUN4ETQHXGDLA5CNFSM4EWK5UTKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5W3LRI#issuecomment-527283653, or mute the thread https://github.com/notifications/unsubscribe-auth/ABNPKZEIIPKZM2OI725GEJLQHXGDLANCNFSM4EWK5UTA .

-- See my YouTube Channel at http://www.youtube.com/user/pageaucp

[pageauc]

Please Note I have upgrade the pi-timolo-install.sh to do a memory check prior to py3exiv2 pip install and dependencies Tested this on an older 512 system and works OK. Will do some testing on copying files from another RPI if files cannot be compiled.

Anyway this should solve the lockup issue when there is not enough memory. Also pyexiv2 is only used to transfer exif data to images when text is added.

Please retest upgrade and let me know if you have any problems Claude ...

On Tue, Sep 3, 2019 at 5:49 AM Claude Pageau pageauc@gmail.com wrote:

Yes you can compile py3exiv2 on another machine and move with same cpu architecture. Files should be

/usr/lib/python3/dist-packages/libexiv2python.so and directory /usr/lib/python3/dist-packages/pyexiv2

Make sure permissions and ownership are the same eg -rw-r--r-- :root:root

permissions would be 644

Let me know how this works out

Claude ...

Claude ...

On Mon, Sep 2, 2019 at 10:49 PM mehair notifications@github.com wrote:

I finished testing the "makevideo.sh" script on the Pi Zero W, although extremely slow it finished without error. It literately takes hours to accomplish what a Pi 3 B can do in under 15 minutes, therefore I'll probably just use the Pi Zero W to just take the images, which it seems to do nicely.

I'm thinking of using a Pi 3 B as a central hub to manage multiple Pi Zero W remote camera's. That way I can off-load the "makevideo.sh" script as we'll as the web services. I'll have to look and see if I can modify your rclone functionality to allow me to do that easily, until then I might just use scp or rsync and a cron.

Using that same concept would it be possible to run the "install-py3exiv2.sh" on another device and then move Python 3 modules over?

Michael

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/pageauc/pi-timolo/issues/93?email_source=notifications&email_token=ABNPKZDW3JVV7K6VYJUN4ETQHXGDLA5CNFSM4EWK5UTKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5W3LRI#issuecomment-527283653, or mute the thread https://github.com/notifications/unsubscribe-auth/ABNPKZEIIPKZM2OI725GEJLQHXGDLANCNFSM4EWK5UTA .

-- See my YouTube Channel at http://www.youtube.com/user/pageaucp

-- See my YouTube Channel at http://www.youtube.com/user/pageaucp

[pageauc]

Note: Did some testing on a Raspbian Jessie with 512 memory and could NOT get py3exiv2 to compile even with swap increased to 1024. It runs OK at good speed on RPI's with 1024 memory. I have therefore given up Compile/Install of py3exiv2 is removed from pi-timolo-install.sh including dependencies. The install-py3exiv2.sh is copied down when p-timolo-install.sh curl is run. install-py3exiv2.sh now checks for memory before running dependencies.

I tried copying files from one machine to another. Eg sudo cp libexiv2python.so /usr/lib/python3/dist-packages/libexiv2python.so sudo chmod 755 /usr/lib/python3/dist-packages/libexiv2python.so and mkdir -p /usr/lib/python3/dist-packages/pyexiv2 sudo cp pyexiv2/* /usr/lib/python3/dist-packages/pyexiv2

python3 Python 3.4.2 (default, Oct 19 2014, 13:31:11) [GCC 4.9.1] on linux Type "help", "copyright", "credits" or "license" for more information.

import pyexiv2 Traceback (most recent call last): File "", line 1, in File "/usr/lib/python3/dist-packages/pyexiv2/init.py", line 60, in

import libexiv2python ImportError: libboost_python37.so.1.67.0: cannot open shared object file: No such file or directory

Turns out the Jessie version of python I was using was 3.4.2 the libboost is a 34 version rather than a 37 version.

/usr/lib/arm-linux-gnueabihf/libboost_python-py34.so.1.55.0

So if both machines have the same version of python I think it will work otherwise the compiled so file is looking for different libraries for a different version this must be done during compile. Never tried copying libboost 34 to 37 file name to see if it would work. But I am thinking most of the libraries know what version of dependencies the need to fine.

Interesting. I will give up on this approach and since pyexiv2 is only used for transferring image exif data it is not critical

I did learn something new today. Your comments are appreciated Regards Claude ...

On Tue, Sep 3, 2019 at 6:48 AM Claude Pageau pageauc@gmail.com wrote:

Please Note I have upgrade the pi-timolo-install.sh to do a memory check prior to py3exiv2 pip install and dependencies Tested this on an older 512 system and works OK. Will do some testing on copying files from another RPI if files cannot be compiled.

Anyway this should solve the lockup issue when there is not enough memory. Also pyexiv2 is only used to transfer exif data to images when text is added.

Please retest upgrade and let me know if you have any problems Claude ...

On Tue, Sep 3, 2019 at 5:49 AM Claude Pageau pageauc@gmail.com wrote:

Yes you can compile py3exiv2 on another machine and move with same cpu architecture. Files should be

/usr/lib/python3/dist-packages/libexiv2python.so and directory /usr/lib/python3/dist-packages/pyexiv2

Make sure permissions and ownership are the same eg -rw-r--r-- :root:root

permissions would be 644

Let me know how this works out

Claude ...

Claude ...

On Mon, Sep 2, 2019 at 10:49 PM mehair notifications@github.com wrote:

I finished testing the "makevideo.sh" script on the Pi Zero W, although extremely slow it finished without error. It literately takes hours to accomplish what a Pi 3 B can do in under 15 minutes, therefore I'll probably just use the Pi Zero W to just take the images, which it seems to do nicely.

I'm thinking of using a Pi 3 B as a central hub to manage multiple Pi Zero W remote camera's. That way I can off-load the "makevideo.sh" script as we'll as the web services. I'll have to look and see if I can modify your rclone functionality to allow me to do that easily, until then I might just use scp or rsync and a cron.

Using that same concept would it be possible to run the "install-py3exiv2.sh" on another device and then move Python 3 modules over?

Michael

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/pageauc/pi-timolo/issues/93?email_source=notifications&email_token=ABNPKZDW3JVV7K6VYJUN4ETQHXGDLA5CNFSM4EWK5UTKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5W3LRI#issuecomment-527283653, or mute the thread https://github.com/notifications/unsubscribe-auth/ABNPKZEIIPKZM2OI725GEJLQHXGDLANCNFSM4EWK5UTA .

-- See my YouTube Channel at http://www.youtube.com/user/pageaucp

-- See my YouTube Channel at http://www.youtube.com/user/pageaucp

-- See my YouTube Channel at http://www.youtube.com/user/pageaucp

[mehair]

Thank you for attempting to get it to compile properly, I do appreciate all the time and work you've put into the project. As I said previously I think I'll just use the Pi Zero W to just take the images, and leave all the heavy lifting to a Pi 3 or Pi 4.

Michael