How did you capture all the output from streamlink?

[veso266]

Hi there

I was wondering on how you managed to capture all the output that streamlink outputs as it is using some wierd tricks I was never able to capture all the output

for instance, when running it like this

streamlink "hls://https://multiplatform-f.akamaihd.net/i/multi/will/bunny/big_buck_bunny_,640x360_400,640x360_700,640x360_1000,950x540_1500,.f4v.csmil/master.m3u8" live -o MSHSAA_1.mp4

I am not able to capture

[download][MSHSAA_1.mp4] Written 204.0 MB (23s @ 8.8 MB/s)

I am onlyl able to capture this

[cli][info] Available streams: live (worst, best)
[cli][info] Opening stream: live (hls)

no matter what I do my OutputDataReceived and ErrorDataReceived are never fired

here is a simple example to show you how I do it

using System;
using System.Diagnostics;

namespace aaaaa
{
    class Program
    {
        void LetsGo()
        {

            var psi = new System.Diagnostics.ProcessStartInfo("streamlink\\bin\\streamlink.exe");
            psi.Arguments = "\"hls://https://multiplatform-f.akamaihd.net/i/multi/will/bunny/big_buck_bunny_,640x360_400,640x360_700,640x360_1000,950x540_1500,.f4v.csmil/master.m3u8\" live -o \"MSHSAA_1.mp4\"";
            psi.UseShellExecute = false;
            psi.RedirectStandardError = true;
            psi.RedirectStandardOutput = true;

            Process call = Process.Start(psi);
            call.OutputDataReceived += new DataReceivedEventHandler(call_OutputDataReceived);
            call.ErrorDataReceived += new DataReceivedEventHandler(call_ErrDataReceived);

            call.EnableRaisingEvents = true;

            call.BeginOutputReadLine();
            call.BeginErrorReadLine();

            call.WaitForExit();
        }

        private void call_ErrDataReceived(object sender, DataReceivedEventArgs e)
        {
            Console.WriteLine("Err: " + e.Data);
        }

        private void call_OutputDataReceived(object sender, DataReceivedEventArgs e)
        {
            Console.WriteLine("Out: " + e.Data);
        }

        static void Main(string[] args)
        {
            Console.WriteLine("Hello World!");
            Program prg = new Program();
            prg.LetsGo();
        }
    }
}

so was wondering how do you get around that to get all the data out of streamlink?

[laurencee]

It's probably that you start the process before hooking up the handlers and enable raising events call.

See my code ordering here: https://github.com/laurencee/Livestream.Monitor/blob/master/Livestream.Monitor/Model/StreamLauncher.cs#L216

[veso266]

nope its not that (copied your code) :cry:

I guess C# has problems: https://stackoverflow.com/questions/7612524/parse-output-from-a-process-that-updates-a-single-console-line

that cannot be fixed easly :smile:

[laurencee]

The example you posted you are starting the process before setting EnableRaisingEvents and hooking up the output event handlers.

Is the example not the code you're using?

[veso266]

meh, didn't change much in an updated example I guess C# cannot capture a single line (line that doesn't use \n but only \r)

using System;
using System.Diagnostics;

namespace aaaaa
{
    class Program
    {
        void LetsGo()
        {

            var proc = new Process
            {
                StartInfo =
                    {
                        FileName = "streamlink\\bin\\streamlink.exe",
                        Arguments = "\"hls://https://multiplatform-f.akamaihd.net/i/multi/will/bunny/big_buck_bunny_,640x360_400,640x360_700,640x360_1000,950x540_1500,.f4v.csmil/master.m3u8\" best -o \"MSHSAA_1.mp4\"",
                        RedirectStandardOutput = true,
                        RedirectStandardError = true,
                        CreateNoWindow = true,
                        UseShellExecute = false
                    },
                EnableRaisingEvents = true
            };

            bool preventClose = false;

            // see below for output handler
            proc.ErrorDataReceived +=
                (sender, args) =>
                {
                    if (args.Data == null) return;

                    preventClose = true;
                    Console.Write(Environment.NewLine + args.Data);
                };
            proc.OutputDataReceived +=
                (sender, args) =>
                {
                    Console.Write(Environment.NewLine + args.Data);
                };

            try
            {
                proc.Start();

                proc.BeginErrorReadLine();
                proc.BeginOutputReadLine();

                proc.WaitForExit();
                if (proc.ExitCode != 0)
                {
                    preventClose = true;
                }
            }
            catch (Exception ex)
            {
                preventClose = true;
                Console.Write(Environment.NewLine + ex);
            }
        }

        static void Main(string[] args)
        {
            Console.WriteLine("Hello World!");
            Program prg = new Program();
            prg.LetsGo();
        }
    }
}

as I said after Opening stream no event is ever fired again, not realy sure if I even need LiveStreamer (since LiveStreamer is probably using ffmpeg to download video) to only download HLS video, I started using it because for some reason ffmpeg command I was using to download before

ffmpeg -i  "hls://https://multiplatform-f.akamaihd.net/i/multi/will/bunny/big_buck_bunny_,640x360_400,640x360_700,640x360_1000,950x540_1500,.f4v.csmil/master.m3u8" -c copy -bsf:a aac_adtstoasc "output.mp4"

started to give me choppy video when downloaded so I looked at Livestreamer and it downloaded video like it should but capturing its output is a pain :smile:

[laurencee]

Sorry, I misunderstood the question the first time. I can see it misses that 1 particular line from the output about the downloaded amount.

I'll have a play around to see if I can figure something out since this interests me too :)

When I ran the code you sent above from the exe (not from the debugger) it outputs the following which misses that 1 line:

What it should look like:

Modified code

using System;
using System.Diagnostics;
using System.IO;

namespace ConsoleApplication1
{
    internal class Program
    {
        static void LetsGo()
        {
            var proc = new Process
            {
                StartInfo =
                {
                    FileName = "streamlink.exe",
                    Arguments =
                        "\"hls://https://multiplatform-f.akamaihd.net/i/multi/will/bunny/big_buck_bunny_,640x360_400,640x360_700,640x360_1000,950x540_1500,.f4v.csmil/master.m3u8\" best -o \"MSHSAA_1.mp4\"",
                    RedirectStandardOutput = true,
                    RedirectStandardError = true,
                    CreateNoWindow = true,
                    UseShellExecute = false
                },
                EnableRaisingEvents = true
            };

            proc.ErrorDataReceived +=
                (sender, args) =>
                {
                    if (args.Data == null) return;

                    Console.WriteLine($"{DateTime.Now}: {args.Data}");
                };
            proc.OutputDataReceived +=
                (sender, args) =>
                {
                    Console.WriteLine($"{DateTime.Now}: {args.Data}");
                };

            try
            {
                proc.Start();

                proc.BeginErrorReadLine();
                proc.BeginOutputReadLine();

                proc.WaitForExit();
            }
            catch (Exception ex)
            {
                Console.WriteLine($"{DateTime.Now}: {ex}");
            }
        }

        public static void Main(string[] args)
        {
            Console.WriteLine("Hello World!");
            LetsGo();

            Console.WriteLine("Press any key to exit...");
            Console.ReadKey();
        }
    }
}

[laurencee]

From a brief investigation it seems like that particular line is being written to standard error (which in itself seems wrong) though it never comes through when I read the error stream so I'm unsure what's going on there.

Streamlink source code for that line: https://github.com/streamlink/streamlink/blob/44d4afaa931e0e3f525ae9ec7adcb91f1020acb2/src/streamlink_cli/utils/progress.py#L55

[laurencee]

Ok I figured it out after finding this issue: https://github.com/streamlink/streamlink/issues/2353

You need to add a --force-progress flag to the argument as streamlink wont output the progress information unless either that flag is set or the output is a tty (which would require a shell execute = true in our case).

The data comes in on the StandardError stream instead like I figured out above.

Final output:

Updated code (including some code I wrote for testing more manual processing of the streams)

using System;
using System.Diagnostics;
using System.IO;

namespace ConsoleApplication1
{
    internal class Program
    {
        static void LetsGo()
        {
            var proc = new Process
            {
                StartInfo =
                {
                    FileName = "streamlink.exe",
                    Arguments =
                        "\"hls://https://multiplatform-f.akamaihd.net/i/multi/will/bunny/big_buck_bunny_,640x360_400,640x360_700,640x360_1000,950x540_1500,.f4v.csmil/master.m3u8\" best -o \"MSHSAA_1.mp4\" --force-progress",
                    RedirectStandardOutput = true,
                    RedirectStandardError = true,
                    CreateNoWindow = true,
                    UseShellExecute = false
                },
                EnableRaisingEvents = true
            };

            proc.ErrorDataReceived +=
                (sender, args) =>
                {
                    if (args.Data == null) return;

                    Console.WriteLine($"{DateTime.Now}: {args.Data}");
                };
            proc.OutputDataReceived +=
                (sender, args) =>
                {
                    Console.WriteLine($"{DateTime.Now}: {args.Data}");
                };

            try
            {
                proc.Start();

                proc.BeginErrorReadLine();
                proc.BeginOutputReadLine();

                // ReadStdOut(proc.StandardOutput);
                // ReadStdOut(proc.StandardError);

                proc.WaitForExit();
            }
            catch (Exception ex)
            {
                Console.WriteLine($"{DateTime.Now}: {ex}");
            }
        }

        static void ReadStdOut(StreamReader stdOutStream)
        {
            var buffer = new char[4096];
            bool stopReading = false;
            while (!stopReading)
            {
                var bytesRead = stdOutStream.Read(buffer, 0, buffer.Length);
                var content = new string(buffer, 0, bytesRead);
                Console.WriteLine($"{DateTime.Now}: {content}");

                Array.Clear(buffer, 0, bytesRead);
                stopReading = bytesRead == 0;
            }
        }

        public static void Main(string[] args)
        {
            Console.WriteLine("Hello World!");
            LetsGo();

            Console.WriteLine("Press any key to exit...");
            Console.ReadKey();
        }
    }
}