search

warren-bank / node-hls-downloader-tubitv

Command-line utility for downloading an offline copy of TubiTV HLS video streams.
GNU General Public License v2.0
64 stars 8 forks source link

Add -metadata title="episode title" to ffmpeg conversion command #6

Closed gstalnaker closed 1 year ago

gstalnaker commented 1 year ago

Warren - thanks for a terrific tool. It works very well. I have one suggestion. Right now the tool downloads a video to a folder that is named according to the TubiTV meta data, e.g., "E01S01 - Name of Episode" in which tubidl puts the ts video data, then creates a mp4 folder in which it puts the captioning doc and the video file named video.mp4. These mp4 files have no metadata in them so a streaming server application like Gerbera is forced to use the filename in its listing - which, because all of the files for a series are named video.mp4 cannot be distinguished one from the other (meaning users have to manually rename the video.mp4 files, which I have been doing). Hoping that you easily modify the ffmpeg command to add metadata, the most useful being '-metadata title="Episode Title"' where "Movie Title" equals the folder name, in the example above "E01S01 - Name of Episode" - or - modify the ffmpeg command so that the output mp4 filename is the folder name, e.g., "E01S01 - Name of episode.mp4" ? Or maybe both of these? I had a go looking at your code thinking perhaps I could do this, but my JavaScript skills didn't include node.js being limited to a vendor's API js implementation and thus the learning curve to attempt this is a bit more than I wish to tackle.

warren-bank commented 1 year ago

Hi. I'm glad you like this utility. I wrote it such a long ago, I'm actually kinda amazed that it still works.

You make a very valid point/request. I'll need to glance at the code to refresh my memory about what it's doing. I'm fairly certain that any such update.. to add this feature.. would be done in an upstream repo:

I'll let you know what I find..

warren-bank commented 1 year ago

notes to self:

warren-bank commented 1 year ago

notes to self:

gstalnaker commented 1 year ago

Warren - I've been having fun this afternoon. I confess I did a bad thing by creating "var episodeTitle" at the global script scope so it can be set in one function and be available in another, but it's only my computer I'm dealing with and ... I got it to work for naming the output mp4 videow with the episode name!! I just did a test and downloaded an episode of Babylon 5 from TubiTV and the output mp4 file is named with the episode name :-) Not sure I should be proud of that or not LOL This is not metadata, which to be honest is likely the better route. But it's better than renaming all of those video.mp4 files one at a time for a series of 110 episodes!. Just so you can see what I did, I've attached the js file to this reply (with appended .txt so I could add it). My mods are prefaced with "// NAK - 20230301" NO worries at all about doing this or not. I've had a fun day figuring out your code and how it worked. VERY well written BTW--really clean (I just had to get my JS thoughts in order then follow your structural logic).

And thanks again for a very useful tool - the best thing about FOSS.

process_cli.js.txt

gstalnaker commented 1 year ago

P.S. that fact that your examples from an hour ago use Connections - The Trigger Effect means we would get along VERY well LOL

warren-bank commented 1 year ago

you found my Easter egg :smiley: imho, that's probably the single best series ever made

ok.. I've pushed updates.. though (admittedly) I haven't actually tested them yet:

warren-bank commented 1 year ago

update:

I'm debating about adding a new (command line) option.. to allow the user to change the resulting .mp4 and .srt filenames. If so, I'm leaning toward just offering a selection of options:

warren-bank commented 1 year ago

update:

gstalnaker commented 1 year ago

You are a scholar and a gentleman :-) Have updated my install and will run it through its paces. And you stayed up WAY too late.

gstalnaker commented 1 year ago

Reporting in - on 110 Babylon 5 episodes, re-run through tubidl to "fix" the tagging, as far as I can see, they are all tagged with "show" and "title" :-)

Much thanks for this tool (which I'm using extensively as it does indeed work - I just found out about TubiTV and that it had the remastered Babylong 5 episodes; otherwise it was completely off my radar) :-) and for your willingness to take it back up after so long. I added one additionaly console.log(cmd) line so that it outputs the full ffmpeg command to the console as it starts the conversion to mp4.

warren-bank commented 1 year ago

glad to hear that these changes are working for you as requested.

just as an fyi.. there is already a way to log the full fmpeg command.. by using: --log-level 2 (or 3 for even more detail).. though, since the console appears to be cleared when each new episode begins, this output would best be piped to a log file:

tubidl ${options[@]} -ll 3 >log.txt 2>&1

where (to be clear):

# options is just a generic array that was used to encapsulate
# (for the purpose of clarity in the above example)
# whatever cli options would normally be used, for example:
options=(-mf 2 --url "https://tubitv.com/series/300005298/zo-zo-zombie")
gstalnaker commented 1 year ago

LOL I had JUST saw that and intended to use it for all my future runs :-)

gstalnaker commented 1 year ago

Downloading using a file with URLs can be frustrating if the URL has issues. tubidl aborts processing of the file on error. I had to check the log/files to see what was successful, then edit the file to remove those already downloaded and the broken URL. So, I wrote a shell script as a frontend for tubidl. It accepts as params a URL or filepath and a boolean ISURL yes/no. Then it processes the file, if ISURL=no, using while/do to read the file line by line and call tubild -u ${LINE}

#!/usr/bin/bash

# This script runs the tubild application with either a single URL
# as a parameter, or a file containing URLs, one per line, as a
# parameter. It will read the file, line by line, and call the 
# tubidl application for each line so that it will fail gracefully
# if a URL cannot be downloaded and continue processing the file

# Usage

usage() {
    echo""
    echo "Usage:"
    echo ""
    echo "~/bin/tubi-dl.sh -i URL/FILEPATH -u YES/NO [-h]"
    echo ""
    echo "This script will call the tubidl application to download tubiTV movies and TV shows."
    echo "It require the tubidl node js application from Warren Bank."
    echo "It requires a tubiTV URL, passed as an argument -i URL"
    echo "or it requires a path to a file containing only URLs, one per line."
    echo "A second argument is a boolean -u YES/NO declaring whether"
    echo "the passed -i argument is a URL or not. That is, if the passed"
    echo "argument is a FILEPATH, the boolean argument is NO."
    echo "The script creates a logfile named with the datetime at run time"
    echo "and logs processing output."
    exit 1
}

# Get arguments

while getopts ":i:u:h" FLAG ; do
    case "${FLAG}" in
        i)
            INPUT=${OPTARG}
            ;;
        u)
            ISURL=${OPTARG}
            ;;
        h)
            usage
            exit 1
            ;;
        \?)
            echo "Unknown option: -$OPTARG" >&2
            usage
            exit 1
            ;;
        :)
            echo "Missing option argument for -$OPTARG" >&2
            usage
            exit 1
            ;;
        *)
            echo "Unimplemented option: -$OPTARG" >&2
            usage
            exit 1
            ;;
    esac
done

shift $((OPTIND-1))

# Missing arguments

if [ -z "${INPUT}" ] || [ -z "${ISURL}" ] ; then
    echo "Command argument is missing."
    echo ""
    echo "Option -i 'input'"
    echo "Option -u 'yes/no'"
    echo ""
    usage
fi

# Set variables

CMD='/usr/local/bin/tubidl'
TIMESTAMP=`(date +%Y%m%dT%H%M%S_%3N)`
LOGFILE=${TIMESTAMP}.log

# Begin logging

echo "`(date +%Y-%m-%d" "%H:%M:%S)` - Starting download and conversion to mp4 file."
echo "`(date +%Y-%m-%d" "%H:%M:%S)` - Starting download and conversion to mp4 file." > ${LOGFILE} 2>&1
echo "`(date +%Y-%m-%d" "%H:%M:%S)` - Variables are: Command: ${CMD}; INPUT: ${INPUT}; ISURL: ${ISURL}"
echo "`(date +%Y-%m-%d" "%H:%M:%S)` - Variables are: Command: ${CMD}; INPUT: ${INPUT}; ISURL: ${ISURL}" >> ${LOGFILE} 2>&1

# Run command

if [ ${ISURL} = 'YES' ] || [ ${ISURL} = 'yes' ]; then
    echo "`(date +%Y-%m-%d" "%H:%M:%S)` - Beginning download of URL."
    echo ""  >> ${LOGFILE} 2>&1

    echo "`(date +%Y-%m-%d" "%H:%M:%S)` - Beginning input download of URL." >> ${LOGFILE} 2>&1
    echo ""  >> ${LOGFILE} 2>&1

    echo "`(date +%Y-%m-%d" "%H:%M:%S)` - Command string: ${CMD} --log-level 3 --max-concurrency 5 --mp4-filename 2 -u ${INPUT} > ${LOGFILE} 2>&1"
    echo "`(date +%Y-%m-%d" "%H:%M:%S)` - Command string: ${CMD} --log-level 3 --max-concurrency 5 --mp4-filename 2 -u ${INPUT} > ${LOGFILE} 2>&1" >> ${LOGFILE} 2>&1
    echo "" >> ${LOGFILE} 2>&1

    ${CMD} --log-level 3 --max-concurrency 5 --mp4-filename 2 -u ${INPUT} >> ${LOGFILE} 2>&1

else # ${ISURL} equals 'NO' or 'no'
    echo "`(date +%Y-%m-%d" "%H:%M:%S)` - Beginning input file parsing."
    echo "" >> ${LOGFILE} 2>&1

    echo "`(date +%Y-%m-%d" "%H:%M:%S)` - Beginning input file parsing." >> ${LOGFILE} 2>&1
    echo "" >> ${LOGFILE} 2>&1

    echo "`(date +%Y-%m-%d" "%H:%M:%S)` - URLs in file to process: "
    echo "`(date +%Y-%m-%d" "%H:%M:%S)` - URLs in file to process: " >> ${LOGFILE} 2>&1

    # Log lines in file

    while IFS= read -r LINE
    do
        echo "`(date +%Y-%m-%d" "%H:%M:%S)` - ${LINE}"
        echo "`(date +%Y-%m-%d" "%H:%M:%S)` - ${LINE}" >> ${LOGFILE} 2>&1
    done < "${INPUT}"

    # Process lines in file, downloading as we go

    while IFS= read -r LINE
    do
        echo "`(date +%Y-%m-%d" "%H:%M:%S)` - Beginning download of $LINE"
        echo "`(date +%Y-%m-%d" "%H:%M:%S)` - Beginning download of $LINE" >> ${LOGFILE} 2>&1

        echo "`(date +%Y-%m-%d" "%H:%M:%S)` - Command string: ${CMD} --log-level 3 --max-concurrency 7 --mp4-filename 2 -u ${LINE} >> ${LOGFILE} 2>&1"
        echo "`(date +%Y-%m-%d" "%H:%M:%S)` - Command string: ${CMD} --log-level 3 --max-concurrency 7 --mp4-filename 2 -u ${LINE} >> ${LOGFILE} 2>&1" >> ${LOGFILE} 2>&1
        echo "" >> ${LOGFILE} 2>&1

        ${CMD} --log-level 3 --max-concurrency 7 --mp4-filename 2 -u "${LINE}" >> ${LOGFILE} 2>&1

        # 2. Get exist status and store into '${STATUS}' var

        STATUS=$?

        if [ ${STATUS} -eq 0 ] ; then
            echo "`(date +%Y-%m-%d" "%H:%M:%S)` - ${CMD} command was successful for ${LINE}"
            echo "`(date +%Y-%m-%d" "%H:%M:%S)` - ${CMD} command was successful for ${LINE}" >> ${LOGFILE} 2>&1
        else
            echo "`(date +%Y-%m-%d" "%H:%M:%S)` - Non-zero exit state: ${CMD} failed for ${LINE}"
            echo "`(date +%Y-%m-%d" "%H:%M:%S)` - Non-zero exit state: ${CMD} failed for ${LINE}" >> ${LOGFILE} 2>&1
        fi
    done < "${INPUT}"
fi

echo "`(date +%Y-%m-%d" "%H:%M:%S)` - Processing concluded." >> ${LOGFILE} 2>&1