Proposal: log video files downloaded by destreamer

[FraH90]

So, as I've written in a previous post I'm creating a python script to automate download with destreamer. In practice I've collected all the group URLs of the class I'm interested in, and I'm just passing commands to destreamer from python.

The sense is that I'm automatically downlading every day every new video recorded, skipping the videos that are already in the folder (the one downloaded previously).

Now I'd like to know which new video has been downloaded every time I launch the python script I've created; I've tried to log the console output directly from python, with something like stuff proposed here:

https://stackoverflow.com/questions/14906764/how-to-redirect-stdout-to-both-file-and-console-with-scripting

However obviously this doesn't work, cause destreamer doesn't write to python console, but to "system terminal" (or maybe NodeJS terminal?)

So I'd like to know how I could capture the "log" from python

Here is the python code I've written:

import os

# LOGIN CREDENTIALS
emLogin = 'EMAIL'

# SET DESTREAMER DIRECTORY
cwd = r'C:\destreamer'
os.chdir(cwd)

# SET BASEPATH (ROOT VIDEO DIRECTORY)
basePath = ''

outputFormat = ' -t \"{publishDate} - {publishTime} - {title} - {author} - {duration} - {uniqueId}\"'

def destream(groupURL, outputFolder):
    os.system('destreamer -u ' + emLogin + ' -i ' + groupURL + ' -o ' +  outputFolder + ' -k --skip -x --vcodec libx265' + outputFormat )

#################################################################################

URL = ''
Folder = os.path.join(basePath, 'SubjectFolder')
destream(URL, Folder)

URL = ''
Folder = os.path.join(basePath, 'SubjectFolder')
destream(URL, Folder)

URL = ''
Folder = os.path.join(basePath, 'SubjectFolder')
destream(URL, Folder)

....................

Basically the destream function is called every time with the proper parameters, so I download the videos of each subject in a single separate folder (I know the way I'm doing it is a little "barbaric", but I'm just starting programming in python). What could I do to capture in a text file the output of "destream"? Is there in destream a function like that? (I don't mean verbose, since I'm not debugging).

PS: The goal is to have the destreamer output, so with python I could find every line that says "Download finished", so I could extrapolate which video have been recorded/downloaded in a certain day

[FraH90]

Ok, as with previous post I'm answering this myself: I've used for the moment output redirection from the shell (cmd, windows), like this:

python python-destreamer.py >> logdestream.txt

Where python-destreamer.py is the name of my python script. I wouldn't want to do this because I was thinking (correctly) that with this I wouldn't have the output to the consolle (I want it bot to txt file and to consolle), but I was partialy right: In fact the nice thing with this trick is that I dont'have the destreamer log output on the consolle (just on the txt file), but the progress bar (only the progress bar) does show up in the shell.

PS: Bad thing: I've just checked the log file, and the txt output is full of stuff like:

Spawning ffmpeg with access token and HLS URL. This may take a few seconds...

[WARNING] Unable to get number of columns from terminal.
This happens sometimes in Cygwin/MSYS.
No progress bar can be rendered, however the download process should not be affected.

Please use PowerShell or cmd.exe to run destreamer on Windows.
--- Speed: 4.7kbits/s, Cursor: 00:00:06.016000
--- Speed: 84.2kbits/s, Cursor: 00:00:10.560000
--- Speed: 67.4kbits/s, Cursor: 00:00:13.184000
--- Speed: 118.3kbits/s, Cursor: 00:00:17.728000
--- Speed: 99.6kbits/s, Cursor: 00:00:21.056000
--- Speed: 174.3kbits/s, Cursor: 00:00:24.064000
--- Speed: 217.5kbits/s, Cursor: 00:00:28.928000
--- Speed: 199.4kbits/s, Cursor: 00:00:31.552000
--- Speed: 176.2kbits/s, Cursor: 00:00:35.712000
--- Speed: 217.0kbits/s, Cursor: 00:00:38.656000
--- Speed: 201.0kbits/s, Cursor: 00:00:41.728000
--- Speed: 189.4kbits/s, Cursor: 00:00:44.288000
--- Speed: 264.6kbits/s, Cursor: 00:00:47.552000
--- Speed: 256.0kbits/s, Cursor: 00:00:49.152000
--- Speed: 278.0kbits/s, Cursor: 00:00:52.800000
--- Speed: 265.5kbits/s, Cursor: 00:00:55.296000
--- Speed: 281.3kbits/s, Cursor: 00:00:59.648000
--- Speed: 271.1kbits/s, Cursor: 00:01:01.888000

So practically the cursor gets "written" to the text file, which makes it very very large in size (cause is written too many times)

If someone has a nicer idea of doing this (maybe directly from python), I would be grateful to have insights Thanks

[lukaarma]

You could use subprocess.run() while piping STDOUT to the father, like this

def destream(groupURL, outputFolder):
    result = subprocess.run(['destreamer', '-u', emLogin, '-i', groupURL, '-o', outputFolder, '-k', '--skip', '-x', '--vcodec', 'libx265', outputFormat], stdout=subprocess.PIPE)
    result.stdout # here should be all Destreamer's output

I'm not 100% sure since my python3 it's a bit rust but it should work

[FraH90]

You could use subprocess.run() while piping STDOUT to the father, like this

def destream(groupURL, outputFolder):
    result = subprocess.run(['destreamer', '-u', emLogin, '-i', groupURL, '-o', outputFolder, '-k', '--skip', '-x', '--vcodec', 'libx265', outputFormat], stdout=subprocess.PIPE)
    result.stdout # here should be all Destreamer's output

I'm not 100% sure since my python3 it's a bit rust but it should work

Thank you! I'll try to implement this when I'll have the time. For now I've resolved using the output redirection with powershell, and then with grep (no matter how big is the txt file) I've extracted all the lines where it says "finished" (which gives me the video downloaded).

Thank you, you can close this ;)

[lukaarma]

Thank you, you can close this ;)

You can too :P