-reconnect_delay_max (ffmpeg)

[peterpan192]

Hey, I was just wondering if the above mentioned option is in the pilfer scripts since my vpn connections are dropping from time to time? Thanks in advance, peterpan192

[NapoleonWils0n]

hi mate

no i didnt add that option to the pilfer scripts becuase i had feedback from some users that it was causing issue with some recordings. i removed that option from the main bash script and used a loop to check if ffmepg cuts out

in the bash version i did add an option to check if the ffmpeg cmd cuts out before its finished recoring and use a loop to restart ffmpeg 3 times if it cuts out, but i couldnt work out how to do the same thing with python

if you find the bash version works better for you that fine but i have found pilfer to run alot faster than the bash version, so it will download video files alot quicker

but whatever works best for you

[NapoleonWils0n]

hi mate

you can always fork the git repo and add the ffmpeg code as well

[peterpan192]

Hi I think I might need a little help with this. Where exactly do I have to insert the code?

[NapoleonWils0n]

Hi Mate

What operating system are you using, forgot what you are using

There are a couple of steps you need to do, fork the git repo, clone it it edit a file, push the changes back up to github then install

If you have pilfer already installed you should remove it with this command, check the pip command you used to install pilfer as some operating system use pip-3.6 and not pip

pip3 uninstall --user pilfer

1 - Fork the pilfer git repo and clone

You need to login to you github account and then go to the pilfer github repo page

And then click the fork button as shown in step 2 on this page fork a github project

• Name the forked repo as pilfer

• Create a git folder in your home and then open the terminal and change directory into the git folder

cd ~/git

• Clone the your forked version of pilfer into the git folder in your home, using the new forked pilfer url

replace YOUR-USERNAME with your github username in the command below

git clone https://github.com/YOUR-USERNAME/pilfer

2 - Adding the new code

The file you need to edit is the record.py in the pilfer folder Open you text editor and then open the record.py file in your home folder/git/pilfer/pilfer/record.py

• Select all of the text in the page and delete it

I have added the ffmpeg reconnect code into the the ffmpeg and rtmpdump functions in the record.py for you mate in the code below

• Select all of the code below and copy into the record.py file in your text editor

#!/usr/bin/env python3 

import shlex
import sys, os
import subprocess
from datetime import datetime 

def ffmpeg(**kwargs):
    ''' ffmpeg function

    ffmpeg recording function
    '''
    home = os.path.expanduser('~')
    desktop = 'Desktop'
    time = datetime.now().strftime("%Y-%m-%d-%H-%M-%S")
    ext = 'mkv'
    recordingfile = os.path.join(home, desktop, 'video-{}.{}'.format(time, ext))

    if sys.platform.startswith('linux'):
        ffmpeg = 'ffmpeg'
    elif sys.platform.startswith('freebsd'):
        ffmpeg = 'ffmpeg'
    elif sys.platform.startswith('win32'):
        home = os.path.expanduser('~').replace('\\', '/')
        desktop = '//Desktop//'
        time = datetime.now().strftime("%Y-%m-%d-%H-%M-%S")
        ext = '.mkv'
        recordingfile = home + desktop + 'video-' + time + ext
        ffmpeg = 'C:/pilfer/system/bin/ffmpeg'
    elif sys.platform.startswith('darwin'):
        bin = 'bin'
        ffmpeg = os.path.join(home, bin, 'ffmpeg')

    # get the values from the dictionary passed to the function
    values = list(kwargs.values())
    # url from kwargs which is the dictionary passed to the function
    url = kwargs['url']

    ffcmd = "{0} -hide_banner -stats -v panic -i {1} -c:v copy -c:a copy {2}".format(ffmpeg, url, recordingfile)

    if any(word in kwargs for word in ('user-agent', 'referer', 'cookie')):                 
        # dict minus first time which is the url
        options = values[1:]
        remove_bracket = str(options)[1:-1]
        options_join = ''.join(remove_bracket).replace('"', '')
        ffcmd = "{0} -hide_banner -stats -v panic {2} -i {1} -c:v copy -c:a copy {3}".format(ffmpeg, url, options_join, recordingfile)

    if 'duration' in kwargs:
        tflag = kwargs['tflag']
        duration = kwargs['duration']
        # dict minus first time which is the url, and minus the last 2 items tflag and duration
        options = values[1:-2]
        remove_bracket = str(options)[1:-1]
        options_join = ''.join(remove_bracket).replace('"', '')
        ffcmd = "{0} -reconnect 1 -reconnect_at_eof 1 -reconnect_streamed 1 -reconnect_delay_max 300 -hide_banner -stats -v panic {2} -i {1} -c:v copy -c:a copy {3} {4} {5}".format(ffmpeg, url, options_join, tflag, duration, recordingfile)

    # split the ffmpeg command for subprocess
    ffsplit = shlex.split(ffcmd)

    print("running ffmpeg command:")

    # try ffmpeg function except keyboard interupt if user quits script with control c
    try:
        process = subprocess.run(ffsplit)
    except KeyboardInterrupt:
        print("recording stopped by user")
    except IOError:
        print("input outpur error, is ffmpeg installed")

def rtmp(**kwargs):
    ''' rtmpdump function

    rtmpdump recording function
    '''
    home = os.path.expanduser('~')
    desktop = 'Desktop'
    time = datetime.now().strftime("%Y-%m-%d-%H-%M-%S")
    ext = 'mkv'
    recordingfile = os.path.join(home, desktop, 'video-{}.{}'.format(time, ext))

    if sys.platform.startswith('linux'):
        ffmpeg = 'ffmpeg'
        rtmpdump = 'rtmpdump'
    elif sys.platform.startswith('freebsd'):
        ffmpeg = 'ffmpeg'
        rtmpdump = 'rtmpdump'
    elif sys.platform.startswith('win32'):
        home = os.path.expanduser('~').replace('\\', '/')
        desktop = '//Desktop//'
        time = datetime.now().strftime("%Y-%m-%d-%H-%M-%S")
        ext = '.mkv'
        recordingfile = home + desktop + 'video-' + time + ext
        ffmpeg = 'C:/pilfer/system/bin/ffmpeg'
        rtmpdump = 'C:/pilfer/system/bin/rtmpdump'
    elif sys.platform.startswith('darwin'):
        bin = 'bin'
        ffmpeg = os.path.join(home, bin, 'ffmpeg')
        rtmpdump = '/usr/local/bin/rtmpdump'

    # url from kwargs which is the dictionary passed to the function
    url = kwargs['url']

    rtmpcmd = "{0} -i '{1}' -o {2}".format(rtmpdump, url, recordingfile)

    if 'duration' in kwargs:
        tflag = kwargs['tflag']
        duration = kwargs['duration']
        rtmpcmd = "{0} -q -i '{1}' | {2} -reconnect 1 -reconnect_at_eof 1 -reconnect_streamed 1 -reconnect_delay_max 300 -hide_banner -stats -v panic -i - -c:v copy -c:a copy {3} {4} {5}".format(rtmpdump, url, ffmpeg, tflag, duration, recordingfile)

    print("running rtmp command:")

    # try ffmpeg function except keyboard interupt if user quits script with control c
    try:
        process = subprocess.run(rtmpcmd, shell=True)
    except KeyboardInterrupt:
        print("recording stopped by user")
    except IOError:
        print("input outpur error, is rtmpdump or ffmpeg installed")

• The lines i have changed are numbers 55 and 109

• Save the record.py file and close your text editor

3 - Commit the changes and push back to github

Open your terminal and change into the pilfer git repo

cd ~/git/pilfer

Now you need to commit the changes to the record.py file

run the following command to commit the changes, depending on your git set up this may open the git commit in nano in the terminal or whatever editor you have set up for git

If the terminal open with nano heres what you do

1 - type the commit message like pilfer ffmpeg reconnect

2 - save the file by typing control + o 3 - close the file by typing control + x

Now you need to push the chnages back up to github

Open the terminal and make sure you are in the pilfer directory, then run this command to push the changes you made to the record.py file up to github

git push -u origin master

That should push the changes upto your forked version of pilfer on git hub

4 - Install your forked version of pilfer with python pip

Replace YOUR-USERNAME with your github username in the command below, and then run the command to install your forked version of pilfer with the ffmpeg reconnect code

pip3 install --user git+https://github.com/YOUR-USERNAME/pilfer.git

Let me know if you get stuck on any points, but the steps above should allow you to fork pilfer and create you ow version with the custom ffmpeg code

Hope that helps

[NapoleonWils0n]

hi mate

missed command for step 3

after you have changed direcory to the pilfer folder directory like this

cd ~/git/pilfer

you need to run the git commit command like this

git commit -a

that will open nano in the terminal where you can write you git commit message then follow the rest of step 3 and then step 4

[peterpan192]

Thank you so much! I do not have time to do this until Monday but I'll let you know if it worked on Monday/Tuesday. Awesome!

[NapoleonWils0n]

hi mate

no problem just remember to do a git commit in step 3

git commit -a

i think the ffmpeg code should be fine as its the same code i was using in the bash version

having your own forked version of pilfer will also let you change the recording destination to something else instead of the desktop if you want

but first things first need to fork the pilfer repo and add the ffmpeg code and then install

[peterpan192]

Worked like a charm! I'm running a test encoding right now to see if ffmpeg keeps working through vpn-reconnects. I'll let you know if it worked.

[NapoleonWils0n]

hi mate

good job on forking the repo

fingers crossed the code works for you

i use privateinternetaccess for my vpn only £25 a year and you can use on 5 devices at once and no logs

[peterpan192]

Thanks for the hint. Unfortunately, ffmpeg stopped working again. It only does that when I'm connected to a VPN (I use IPVanish). I assume, it has something to do with inactivity timeouts I get hourly.. However, the reconnect_delay_max-option should work against that, right? If I type

pgrep ffmpeg

it tells me that it's still running but it's not encoding anymore. Maybe I should try privateinternetaccess.

[NapoleonWils0n]

hi mate

ive been using privateinternetaccess for several years and it works great never had a problem

i also use it when im download torrents, some locations are better for torrents than others i think i use the usa mid west server for torrents

there are dozens of placesyou can come out in all over the world

i have also used ffmpeg over vpn and not had any issues

i think you can sign up for a month or year so maybe signup for a month which will cost a couple of quid and try it out, the fork out for a year

what i do is create a split vpn tunnel and the run dante socks 5 proxy and attatch to the vpn tun interface

that way if i want to send stuff over the vpn i can change my browser proxy settings or any program tha can use a socks 5 proxy to use the local socks 5 proxy connected to the vpn

i can also use proxychains to make a program use the sock 5 proxy and vpn as well for programs that dont have sock 5 support

i also run the unbound local caching dns server and set that to send dns queries over the vpn which means i can do dns ad blocking as well over the vpn to block things like javascript bittorrent miners and ads

i have some a bash script i run called vpn-split-tunnel which displays a bash menu with numbers next to all the vpn endpoints i can connect to, then i type a number next to where i want to come out, eg us, canada, france germany

the script creates a second routing table connects to the vpn then starts the socks 5 proxy connected to the tun network interface and changes the outbound interface on the unbound dns server to use tun interface as well

then you control c to kill and it stops the vpn, sock 5 proxy and chnages the unbound dns outbound connect back to normal

advantage is i can pick and choose what to send over the vpn been using the set up for several years and it works on all versions of linux and unix as well

i have all the steps, install instructions on github if you want a look

the cool thing is if you use dnsleaktest.com to check your dns leaks the unbound dns server wont leak your real location it will show up as coming out of the vpn endpoint and you have caching dns to speed things up, adblocking and also helps unblocking sites sometimes

On Tue, 23 Jan 2018 at 14:56, peterpan192 notifications@github.com wrote:

Thanks for the hint. Unfortunately, ffmpeg stopped working again. It only does that when I'm connected to a VPN (I use IPVanish). I assume, it has something to do with inactivity timeouts I get hourly.. However, the reconnect_delay_max-option should work against that, right? If I type

pgrep ffmpeg

it tells me that it's still running but it's not encoding anymore. Maybe I should try privateinternetaccess.

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/NapoleonWils0n/pilfer/issues/4#issuecomment-359815831, or mute the thread https://github.com/notifications/unsubscribe-auth/ACiua5QmCdUkATJfUGhY94zlL_ml4V3xks5tNfM3gaJpZM4Rjomk .

-- Dan

[peterpan192]

Again, thanks for your expertise! I will look into that when I have some extra time. For now, I think I might have figured out the problem. My internet is very slow. Around 100 kbps max. If I use IPVanish, the VPN uses my whole bandwith and the openvpn-log tells me that there are frequent 'inactivity timeouts'. I tried to limit the bandwith the vpn is allowed to use to around 85 kbps with wondershaper. That time, ffmpeg did not stop encoding. There were no inactivity timeouts in the openvpn-log, just 'soft resets' every hour. Maybe, if the VPN uses all of my bandwith, there is no bandwith left for the pings sent and received between my device and IP-Vanish. Not sure, if that makes sense, but it worked last night..