Prevent download if livestream in progress

bryan commented 7 years ago

Before submitting an issue, make sure you have:

[x] Read the README
[x] Read the FAQ
[x] Searched the bugtracker for similar issues including closed ones
[x] Make sure you have the latest version: v0.3.6

Purpose of your issue:

[ ] Bug report (problems/errors)
[x] Question
[ ] Other

Describe your issue

Is there a simple way to prevent a stream download from occurring if the streaming is in progress from a different process? I run into this scenario when my time-based scheduler calls livestream_dl every few minutes, but if the same user is already streaming, it'll just create another folder and create a duplicate download stream with a {tempfoldername-1}.

I can think of a one simple way to prevent this. Check to see if the temporary download folder exists for that live stream, if it does, do not download since the stream will always have the same naming format as long as the user does not intentionally change it through dynamic mean.

I didn't mark it as a bug if in the case it it was made intentional to simply create duplicates (maybe create two files at different start times, one a subset of the other). If that is the case, I'll probably have to figure out a different way to circumvent this problem, but would love any feedback if it is in your domain to fix/improve.

Thanks!

taengstagram commented 7 years ago

I see this as a problem with your scheduler/scheduled task starting multiple concurrent jobs. It should be easy enough to fix by implementing a simple lock file mechanism in a scheduled script instead.

bryan commented 7 years ago

Could you elaborate a little bit on the simple lock file mechanism in a scheduled script? I may be thinking more complexly over what you're thinking.

My workflow goes like this:

I have a python script that spawns subprocesses starting the livestream_dl command for a couple of users (since threads are blocking when using os.system to run a command, otherwise a simple lock file would work here).
I have cron starting the python script every 1 minute (which launches all the subprocesses to be nonblocking).
Through the python script, there is no way for me to know if and when a particular user's livestream_dl has finished or not (if I knew then it'd be an easy add/removal through a shared file among the multiple processes)

From what I'm understanding, you want to me to simply just create a script not launched by cron and starts tasks by an internal timer (endless while loop or something matched to a time)? If that's the case, I'd run into the same problem which will make it blocking if there are concurrently live streamers at one time.

Edited side note: I'm curious as to how you're managing to upload 8+ months of photos/videos to repos without hitting the cap by Github or getting noticed. Are you using git-lfs and do you store backups if the repos close?

I do appreciate your support and super timely response. Thanks again.

taengstagram commented 7 years ago

This is what I mean, in the form of a bash script that can be called by cron at a fixed interval

#!/bin/bash
for acct in 'userA' 'userB' 'userC'
do
    if [ ! -f "$acct.lock" ]; then
        # no lock file found, then generate one
        touch "$acct.lock"
        # start downloader and remove lock file after in separate process
        (livestream_dl "$acct"; rm "$acct.lock") &
    else
        echo "$acct.lock file found"
        # don't start downloader
    fi
    sleep 10
done

bryan commented 7 years ago

Ah, I see what you mean.

Well thanks for getting back to me. I'll probably play around with it to match with my other workflows when I have time.

Thanks again.

taengstagram / instagram-livestream-downloader