convert h264 to mp4 right after recording.

[ozett]

hi, as i can browse only mp4-files without any hasse in chrome, i would be pleased to save all motion-recordings directly as mp4. as this seems not possible, what about directly call mp4box after recording to convert every h264-recording directly to an mp4-file. not to wait for a cron-time or any other trigger.

maybe here in the code, after "camera.stop.recording():

what do you think?

i looked into "inotifywait" (http://stackoverflow.com/questions/10533200/processing-data-with-inotify-tools-as-a-daemon), which seems a reasonably option.

but that would put another system across the lean pi-timolo.py solution. so i thought to ask, for the developers opinion. 👐

i would test something.. 😃

regards, tozett

[pageauc]

This would not be possible from within pi-timolo since it would slow things down and delay return to motion detection. This could be a big issue with a single core cpu. Doing it through a crontab means it can be done on a different schedule and more than one can be done at one time thus reducing load and pi-timolo can continue to take video's or motion detection.

Claude ...

On Thu, Jan 5, 2017 at 3:35 PM, ozett notifications@github.com wrote:

hi, as i can browse only mp4-files without any hasse in chrome, i would be pleased to save all motion-recordings directly as mp4. as this seems not possible, what about directly call mp4box after recording to convert every h264-recording directly to an mp4-file. not to wait for a cron-time or any other trigger.

maybe here in the code, after "camera.stop.recording(): [image: pi-timolo-h264] https://cloud.githubusercontent.com/assets/5937811/21698314/a96df41a-d396-11e6-9039-ea54ae620b5f.png

what do you think?

i looked into "inotifywait" (http://stackoverflow.com/ questions/10533200/processing-data-with-inotify-tools-as-a-daemon), which seems a reasonably option.

but that would put another system across the lean pi-timolo.py solution. so i thought to ask, for the developers opinion. 👐

i would test something.. 😃

regards, tozett

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/pageauc/pi-timolo/issues/37, or mute the thread https://github.com/notifications/unsubscribe-auth/AFr1ZGV7VLelteX8EML_ErVuuMbVufVvks5rPWIGgaJpZM4LcJ31 .

-- See my YouTube Channel at http://www.youtube.com/user/pageaucp

[ozett]

thanks for explanation, i will see how i can implement the inotifywait and trigger the convid.sh script on "file-close". thanks a lot, your support is a big help.. 🥇

[ozett]

i look for inotifywait on the web, and found there is a python version. maybe of interest for you, for some kind of further development.....

https://github.com/seb-m/pyinotify

[pageauc]

Took a look at this issue of dynamically converting h264 to MP4 format in pi-timolo. I have a preliminary revision that runs convid.sh as a subprocess thread in pi-timolo.py to convert the h264 in pi-timolo.py immediately after recording is complete. This works OK on a quad core RPI but have not tried on a single core. I an looking at making this an option that can be turned on or off but at this time it is the default. It takes the variables in convid.sh so you can control option to delete h264 after conversion. You should do an update or just do a wget

wget -O pi-timolo.py https://raw.github.com/pageauc/pi-timolo/master/source/pi-timolo.py wget -O convid.sh https://raw.github.com/pageauc/pi-timolo/master/source/convid.sh

Please take a look an test. Your feedback would be appreciated.

Claude ...

On Thu, Jan 5, 2017 at 4:15 PM, Claude Pageau pageauc@gmail.com wrote:

This would not be possible from within pi-timolo since it would slow things down and delay return to motion detection. This could be a big issue with a single core cpu. Doing it through a crontab means it can be done on a different schedule and more than one can be done at one time thus reducing load and pi-timolo can continue to take video's or motion detection.

Claude ...

On Thu, Jan 5, 2017 at 3:35 PM, ozett notifications@github.com wrote:

hi, as i can browse only mp4-files without any hasse in chrome, i would be pleased to save all motion-recordings directly as mp4. as this seems not possible, what about directly call mp4box after recording to convert every h264-recording directly to an mp4-file. not to wait for a cron-time or any other trigger.

maybe here in the code, after "camera.stop.recording(): [image: pi-timolo-h264] https://cloud.githubusercontent.com/assets/5937811/21698314/a96df41a-d396-11e6-9039-ea54ae620b5f.png

what do you think?

i looked into "inotifywait" (http://stackoverflow.com/ques tions/10533200/processing-data-with-inotify-tools-as-a-daemon), which seems a reasonably option.

but that would put another system across the lean pi-timolo.py solution. so i thought to ask, for the developers opinion. 👐

i would test something.. 😃

regards, tozett

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/pageauc/pi-timolo/issues/37, or mute the thread https://github.com/notifications/unsubscribe-auth/AFr1ZGV7VLelteX8EML_ErVuuMbVufVvks5rPWIGgaJpZM4LcJ31 .

-- See my YouTube Channel at http://www.youtube.com/user/pageaucp

-- See my YouTube Channel at http://www.youtube.com/user/pageaucp

[ozett]

i will try this evening - and report back... i will try on my raspi one, a single core model. it it is two slow, maybe the option can check which raspi-model it runs on?...

but first i will try it and report back (takes some minutes..)

regards, tozett

[ozett]

i updated the files, addes executable permission to convid.sh and let it run. i will see tomorrow if there had been problems, but after two recordings with conversion it looked good.

it happend that one of my older raspi-ones had an error on usb while booting. it seems that the network-port is dead and i must replace this one with my actual pi-timolo raspberry. but i will take tomorrow to test an older core-one raspberry before i step into actual multi-core ones with pi-timolo...

[ozett]

it seems, that the deletion is not executed...?? files are still in the folder. a problem with the path?

edit: seems that there had been files left over from testing.....wrong error.....-> everything ok 👍

[pageauc]

I have run my setup with several thousand files created and successfully converted. It may be a permissions problem depending on how things were started. I have /etc/rc.local run under pi permissions

su pi -c "/home/pi/pi-timolo/pi-timolo.sh start > /dev/null" su pi -c "/home/pi/pi-timolo/webserver.sh start > /dev/null"

Also pi-timolo.py will only process the last h264 files and not any previous ones. If you start and stop it is possible to have h264's and stopped before mp4 is created. You can manually clean these up by running convid.sh on i't own from a command prompt. ./convid.sh Then monitor to see if the situation repeats.

Claude ....

[ozett]

runs without errors. wonderful. thanks..

i dont know if i asked it before, but you updated the script to make DAILY movies from images (timelapse or motion). runs fine.

but what about to make one DAILY movie out of the little motion-sequence-videos (and afterwards delete them/or archive them in a daily-date-named folder). so that one has than a clean folder with one daily-video out of (they are now .mp4-) the many single motion sequences files?

[pageauc]

Yes creating a daily movie from a lot of short clips can be done with MP4Box command. See my wiki https://github.com/pageauc/pi-timolo/wiki/How-to-Convert-h264-video-to-MP4 near bottom of page. If I get some time I will look at a script to create this or add it as an option in make video script.

Claude ...

[ozett]

i found the hint in the wiki:

"You will need to write a script to concatenate a series of files"

i hoped i could trigger something like "makevid.sh" 😏 ... but i can wait ... ⏳ 😏

anyway, thanks for reply 🥇

[pageauc]

Ozett

re github issue 38, 39 and 40

See my GitHub issue comments re convid.sh now having join option to connect a specified number of small video into a larger one.

The convert option still works for a single or multiple files. The default is for pi-timolo.py to automatically convert h264 current finished video using a subprocess call to convid.sh.

I have also updated pi-timolo.py to show video start date/time annotated on the video. It uses the image text on and size variables in config.py. This works great if you join video's together. Also bug fixes for makevideo.sh. You might want to do a new install or just download the changed files.'

Thanks for encouraging me to add these features, like convid.sh join and video annotation. Let me know how these work for you.

wget -O pi-timolo.py https://raw.github.com/pageauc/pi-timolo/master/source/pi-timolo.py wget -O convid.sh https://raw.github.com/pageauc/pi-timolo/master/source/convid.sh wget -O makevideo.sh https://raw.github.com/pageauc/pi-timolo/master/source/makevideo.sh

Regards Claude ...

On Wed, Jan 11, 2017 at 7:50 AM, ozett notifications@github.com wrote:

i found the hint in the wiki:

"You will need to write a script to concatenate a series of files"

i hoped i could trigger something like "makevid.sh" 😏 ... but i can wait ... ⏳

anyway, thanks for reply 🥇

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/pageauc/pi-timolo/issues/37#issuecomment-271872934, or mute the thread https://github.com/notifications/unsubscribe-auth/AFr1ZBzrCkCEwHdAnQBEtyTvL8ykeqnmks5rRN4QgaJpZM4LcJ31 .

-- See my YouTube Channel at http://www.youtube.com/user/pageaucp

[ozett]

i just updated everything and let it run. as you have all the path-variables directly in the head of the scipt-file, i have to adjust it with every update. would make more sense to me, if the path-vars for the scripts could also be in a central config(.py). ❔ ❕

in the previous bash-scipts i corrected a litte typo-error of a missing space. i will see if you found it too. 😁

[ozett]

i will report the result. actually it is running... 🏃

[ozett]

it puts some load on the raspi-1, single core. but the raspi is still reasonable responsive ... 😁

[pageauc]

I fixed the space problem so the new version on github should work OK. Sorry for the problem of vars being stored in the script. Will take a look at this but was trying to keep number of files lower. Normally these utilities should not change much in the future. Joining does put a load on the cpu but most of the load is on the SD card or hard drive and can make system laggy. The RPI seems to still handle things OK though. I don't expect joining should be done more than a few times a day in a cron job. I will try with a hard drive since I believe it would .

Was thinking of adding a feature to join/convert a specified number at a time eg 100. That way a cron job can batch for a while and continue later and give the RPI cpu a chance to cool down. Otherwise it seems to do what you were asking and join reduces the number of videos. I set join to 12 so 10 second videos combine to be 2 minutes long. Adding the annotate works better since if the motion sequences are spread over a longer time period eg a few hours, then you can see when the event happened.

Also update the wiki and script comments, help

Raise github issue if you have fixes or suggestions

Thanks Claude ...

[ozett]

Ahhh, the default is 12 recordings of motion are joined per one call of "convid.sh join"....

thats my confusion....

if i want to have all existing motion-recordings joined, i have to call it as many times, as there are nummer-of-recordings / 12. (if the assumption is right, the filename will always be the same: convid__.mp4. Overwrite?)

right?

[pageauc]

wrong

12 is just then default number that are joined into one output video. This can be changed to more or less. 12 represents about a two minute output video from twelve 10 second source videos

The video files in the source folder file listing is read and all those are processed until done. So if there are 120 files you will get 12 output videos each about two minutes if the source file videos are ten seconds each.

Joining can take some time so in the mean time more source video's may get produced. These new video's are not included above processing.

Hope this clarifies.

On Mon, Jan 16, 2017 at 10:23 AM, ozett notifications@github.com wrote:

Ahhh, the default is 12 recordings of motion are joined per one call of "convid.sh join"....

thats my confusion....

if i want to have all existing motion-recordings joined, i have to call it as many times, as there are nummer-of-recordings / 12.

right?

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/pageauc/pi-timolo/issues/37#issuecomment-272905779, or mute the thread https://github.com/notifications/unsubscribe-auth/AFr1ZKbuqcWe0GSIhxDV56UN3edCCXhsks5rS5mIgaJpZM4LcJ31 .

-- See my YouTube Channel at http://www.youtube.com/user/pageaucp

[ozett]

yeah, thanks a lot ...now i can see this... (you sharpend my view for this) 😁

[ozett]

do i make mistakes in couting the overall time? i have 369 files, each 10 sec (right? default recording time) => 3690 sec = 61 min (after join)? but after join the resulting file has a playtime of 1:11 min, means 71sec..

hmmm. i am a little 😕

edit: i could have another mis-understanding, that 1:11 means 1hrs:1min, which would roughly been 3690 seconds, but the movie is "realy" playing only one minute+1sec... i watched it live, and it ended after 1 minute. there seems also no "compression" of time... (and the recordings have (and had) all the default 10sec playtime. i checked this again..)

what can i check? (maybe the resulting movie is only the last joined? with its resulting playtime of only the last 12 movies. means 12x10sec=120sec? so that the resulting movie of every processed 12 are not the joins to an overall-movie?)

[pageauc]

You should get 36 approx two min joined output videos and the last video should be approx 90 seconds for a total of 37 total output videos.

I have done some prelim testing but not verified everything. I can do more testing to see if anything is missed or counting logic has a problem

Claude ....

On Tue, Jan 17, 2017 at 2:18 AM, ozett notifications@github.com wrote:

do i make mistakes in couting the overall time? i have 369 files, each 10 sec (right? default recording time) => 3690 sec = 61 min (after join)? but after join the resulting file has a playtime of 1:11 min, means 71sec..

hmmm. i am a little 😕

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/pageauc/pi-timolo/issues/37#issuecomment-273049398, or mute the thread https://github.com/notifications/unsubscribe-auth/AFr1ZKILx1fDWGgD0ElzUgY5jKmjA19yks5rTHligaJpZM4LcJ31 .

-- See my YouTube Channel at http://www.youtube.com/user/pageaucp

[ozett]

i had 142 motion-recordings in my motion-dir. and started convid.sh join...

while it is running, i only see 1 convid__.mp4 in the working dir (pi-timolo). everthing runs as root...should be no permisson problem..

[ozett]

everthing seems to be joined to the one file: convid__.mp4...

:dizzy_face:

[pageauc]

I would not recommend you run as root. Take a look a pi-timolo github wiki for details. Also it would appear that you are not getting date stamp on output file. Make sure you have downloaded he latest convid.sh ver 1.4

It is trying to put the output file in the root users home folder under pi-timolo with no date stamp on the output file. Not sure how you have things configured but the date/time stamp is dynamically created in the script from the file stamp of the first file to be joined. This is so the creation date of the output file has the same time stamp as the first file to be joined. stat command gives information about the file and I am extracting that data.

date=$(stat $the_first_mp4 | grep Modify | cut -f 2 -d ' ')

Try running with the default settings and verify it is working. Let me know if it is not. I see an nfs folder reference so stat does not work for a file on network drive

Anyway check if default local settings runs OK.

Claude ...

On Tue, Jan 17, 2017 at 11:45 AM, ozett notifications@github.com wrote:

everthing seems to be joined to the one file: convid__.mp4...

😵

[image: convidjoin2] https://cloud.githubusercontent.com/assets/5937811/22032533/04ce9b46-dce5-11e6-8294-fff75cc08d82.png

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/pageauc/pi-timolo/issues/37#issuecomment-273243200, or mute the thread https://github.com/notifications/unsubscribe-auth/AFr1ZLxGj6dul56HoBvk45AQq5_nz8szks5rTP5EgaJpZM4LcJ31 .

-- See my YouTube Channel at http://www.youtube.com/user/pageaucp

[ozett]

Ahh...and the error is: :de: -> german language version reply on "grep Modify"...

:mask: regards, tozett (quick and dirty changed "grep Modify" to "grep Modif" and now trying to run it...)

[ozett]

maybe this could help... http://askubuntu.com/questions/350679/get-system-locale-language-code-in-bash-for-a-dynamic-path

[ozett]

i (was) and being still on ver1.4, my modification of "grep Modif" fixed this language issue . now i get the bunch of motion-recodings in destination directory, but ( for me) these results are still to many single files.

what do you suggest, how to get/add to have only 1 recording per day. increase the count of 12 to unlimited (overload?)

... or do you have a new implementation for adding all motion-recording to one big file (per day/cron) ?

thanks up to here, it a pleasure to record with your scripts... 😀 tozett

[pageauc]

Joining uses quite a bit of RPI resources and can take some time. I picked 12 but it could be increased. It could also be changed to limit by max output size or time period restricted. eg all files within a 4 hour period. Also have not tested but it seems to me that joining gets slower as the output file gets bigger. It would be possible to join after convert but this might slow things down especially if there are a lot to process. You could also reduce motion video recording time from 10 to maybe 6 seconds and then increase join to 40. I depends on what you are trying to capture. Short periods of high motion or long periods of intermittent motion.

I am currently working on having configuration files separate (this is pretty easy) and creating a whiptail menu system similar to raspi-config that would allow running and changing settings. Also looking at changing directory structure

           /tmp
           /mnt
           /data
           /config
           /bin

/pi-timolo/media/motion /timelapse /video

Claude ...

On Wed, Jan 18, 2017 at 2:56 AM, ozett notifications@github.com wrote:

i (was) and being still on ver1.4, my modification of "grep Modif" fixed this language issue . now i get the bunch of motion-recodings in destination directory, but ( for me) these results are still to many single files.

what do you suggest, how to get/add to have only 1 recording per day. increase the count of 12 to unlimited (overload?)

... or do you have a new implementation for adding all motion-recording to one big file (per day/cron) ?

thanks up to here, it a pleasure to record with your scripts... 😀 tozett

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/pageauc/pi-timolo/issues/37#issuecomment-273418694, or mute the thread https://github.com/notifications/unsubscribe-auth/AFr1ZJxFPNKkRlXs7f8uJdgtGP_vfEQWks5rTdO3gaJpZM4LcJ31 .

-- See my YouTube Channel at http://www.youtube.com/user/pageaucp

[ozett]

ok. thanks. i will try to join "max" files and see if the raspberry burns.... 🔥 😁

i loved pi-timolo for been clean and small, i reduced the directory only to the few scripts and a storage dir on my nfs.... maybe too much separation distorts the lean-ness ?....

anyway thanks i lot, i look forward to your changes .. 👀

[ozett]

what was considered to put here in the convid.sh script for joining max files...

> # MP4 Join Variables Settings
> #----------------------------
> max_joins=12   # Number of videos to join before starting New Join
> 

max_joins= 'empty' ?

[ozett]

clean and lean ... 😃

[pageauc]

Ok Thanks for your comment about lean folder structure. Will try to keep things lean. Was thinking of combining scripts into one larger script with parameters

eg pi-timolo (would get rid of .sh since bash scripts don't really need it) with parameters

install upgrade start stop restart config web start web stop web restart web config sync start sync stop sync restart sync config all start all stop all restart video join video convert video timelapse menu (would run whiptail menuing for doing the above plus convenient menu for configuring settings) help

if no parameters then basic instructions on paramenters

That would be a larger file but might not take very long to load and would integrate functionality. Most of the code already exists so it would just be a matter of putting them in functions to be called by parameters.

Your comments would be appreciated. Claude

On Wed, Jan 18, 2017 at 7:51 AM, ozett notifications@github.com wrote:

[image: dire] https://cloud.githubusercontent.com/assets/5937811/22066532/984d8f5a-dd8d-11e6-8d52-c4bc9a343ac5.png

clean and lean ... 😃

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/pageauc/pi-timolo/issues/37#issuecomment-273480087, or mute the thread https://github.com/notifications/unsubscribe-auth/AFr1ZM8GLdKGM0WyqAE9ikDGbQS61oiyks5rThj8gaJpZM4LcJ31 .

-- See my YouTube Channel at http://www.youtube.com/user/pageaucp