search

segator / proxylive

This application allow you to transcode and broadcast on demand a http mpeg ts to transcoded mpegts/hls
MIT License
59 stars 21 forks source link

FFMPEG arguments are added by default #9

Closed noobi0 closed 4 years ago

noobi0 commented 5 years ago

During testing, it seems like even if in application.yml:

    profiles:
        -
            alias: "h264"
            parameters: "-i {input} {channelParameters}"
    hls:
        tempPath: "/tmp"
        parameters: ""
        timeout: 30

as well as in channels.json:

      "ffmpegParameters": "",

are left completely empty, some ffmpeg switches are still added by default:

[test_h264] Start Transcode
Transcoding Commandffmpeg -i http://localhost:8080/view/raw/test?user=internal&token=DH3FM5MH2L62DMHTZVSJQ6FE9X80HPZRILT5MXD8  -threads 0 -f mpegts -mpegts_m2ts_mode 1 -mpegts_copyts 1 -mpegts_flags +resend_headers   -

Is it possible to completely control ffmpeg arguments, meaning exclude these default ones, and only add whatever was defined? Thank you for any input.

segator commented 5 years ago

Hey Sorry I saw your issue a month ago but I forgot to answer :( There are some parameter by default depending of the transport you are using, check this. https://github.com/segator/proxylive/blob/master/src/main/resources/application.yml#L55

Anyway those parameters are default, is causing problems in your stream? I'm not an expert on ffmpeg so if you have any issue with this parameters let me know please ;)

noobi0 commented 4 years ago

Ok, mpegTS parameters were removed at all from application.yml file, once I returned that part with empty values, there are no default values inserted.

However, as much as I would want to start using this application, as it perfectly fits my required scenario where I have couple of dozen of streams what are unplayable unless are correctly remuxed with ffmpeg), where proxylive could simplify this a lot.

Source streams are h264 HLS m3u8 playlists.

If you want this in separate issue, and close this one, please tell, will move then.

As to minimize any misconfig, I removed / disabled everything just to keep minimal essentials. I just want it to work for now, then gradually add prometheus / grafana monitoring, epg, and other extras. Only basic ffmpeg arguments (copy) were used while testing.

$ cat channels.json

[ 
    {
      "number" : 1,
      "name" : "test",
      "id" : "test",
      "epgID" : "test",
      "logoFile":"",
      "logoURL": "file:///opt/proxylive/logo.png",
      "categories" : [ "General", "TEST" ],
      "ffmpegParameters": "",
      "sources" : [ 
          {
            "url" : "https://bitdash-a.akamaihd.net/content/sintel/hls/playlist.m3u8",
            "priority" : 1
          }
      ]

    }
]

$ cat application.yml

spring.jackson.serialization.INDENT_OUTPUT: true
management:
    server:
        port: 8090
    endpoints:
        prometheus:
            enabled: false
        metrics:
            enabled: false
        web:
            exposure:
                include: "*"
    metrics:
        export:
            prometheus:
                enabled: false
    security:
        enabled: false

geoIP:
    enabled: false
    url: https://geolite.maxmind.com/download/geoip/database/GeoLite2-City.tar.gz
source:
    epg:
        refresh: 600
    channels:
        refresh: 60
        type: json
        url: file:///opt/proxylive/channels.json
    reconnectTimeout: 15
streamTimeout: 30
ffmpeg:
    path: '/usr/bin/ffmpeg'
    profiles:
        -
            alias: "aac"
            parameters: "-i {input} {channelParameters}"
        -
            alias: "240p"
            parameters: "-i {input} {channelParameters}"
        -
            alias: "360p"
            parameters: "-i {input} {channelParameters}"
        -
            alias: "480p"
            parameters: "-i {input} {channelParameters}"
        -
            alias: "720p"
            parameters: "-i {input} {channelParameters}"
        -
            alias: "1080p"
            parameters:  "-i {input} {channelParameters} -acodec copy -vcodec copy"
        -
            alias: "vaapi1080p"
            parameters: "-i {input} {channelParameters}"
    mpegTS:
        parameters: ""
    hls:
        enabled: true
        tempPath: "/opt/proxylive/tmp"
        parameters: ""
        timeout: 30

buffers:
    chunkSize: 131072
    broadcastBufferSize: 52428800

$ java -jar proxylive.jar

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v2.1.3.RELEASE)

2019-12-03 05:23:44.849  INFO 3531 --- [           main] c.github.segator.proxylive.Application   : Starting Application v1.0-SNAPSHOT on server with PID 3531 (/opt/proxylive/proxylive.jar started by root in /opt/proxylive)
2019-12-03 05:23:44.853  INFO 3531 --- [           main] c.github.segator.proxylive.Application   : No active profile set, falling back to default profiles: default
2019-12-03 05:23:47.093  INFO 3531 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port(s): 8080 (http)
2019-12-03 05:23:47.155  INFO 3531 --- [           main] o.apache.catalina.core.StandardService   : Starting service [Tomcat]
2019-12-03 05:23:47.156  INFO 3531 --- [           main] org.apache.catalina.core.StandardEngine  : Starting Servlet engine: [Apache Tomcat/9.0.16]
2019-12-03 05:23:47.173  INFO 3531 --- [           main] o.a.catalina.core.AprLifecycleListener   : The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: [/usr/java/packages/lib:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib]
2019-12-03 05:23:47.291  INFO 3531 --- [           main] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
2019-12-03 05:23:47.291  INFO 3531 --- [           main] o.s.web.context.ContextLoader            : Root WebApplicationContext: initialization completed in 2355 ms
Channels loaded
2019-12-03 05:23:48.098  INFO 3531 --- [           main] o.s.s.concurrent.ThreadPoolTaskExecutor  : Initializing ExecutorService 'applicationTaskExecutor'
2019-12-03 05:23:48.604  INFO 3531 --- [           main] o.s.s.c.ThreadPoolTaskScheduler          : Initializing ExecutorService 'taskScheduler'
2019-12-03 05:23:48.782  INFO 3531 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port(s): 8090 (http)
2019-12-03 05:23:48.784  INFO 3531 --- [           main] o.apache.catalina.core.StandardService   : Starting service [Tomcat]
2019-12-03 05:23:48.784  INFO 3531 --- [           main] org.apache.catalina.core.StandardEngine  : Starting Servlet engine: [Apache Tomcat/9.0.16]
2019-12-03 05:23:48.788  INFO 3531 --- [           main] o.a.c.c.C.[Tomcat-1].[localhost].[/]     : Initializing Spring embedded WebApplicationContext
2019-12-03 05:23:48.789  INFO 3531 --- [           main] o.s.web.context.ContextLoader            : Root WebApplicationContext: initialization completed in 129 ms
2019-12-03 05:23:48.836  INFO 3531 --- [           main] o.s.b.a.e.web.EndpointLinksResolver      : Exposing 15 endpoint(s) beneath base path '/actuator'
2019-12-03 05:23:48.955  INFO 3531 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 8090 (http) with context path ''
2019-12-03 05:23:49.028  INFO 3531 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 8080 (http) with context path ''
2019-12-03 05:23:49.030  INFO 3531 --- [           main] c.github.segator.proxylive.Application   : Started Application in 4.953 seconds (JVM running for 5.486)

$ curl http://localhost:8080/channel/list/hls/1080p -o test.m3u8

$ cat test.m3u8

#EXTM3U cache=2000 url-tvg="http://localhost:8080/epg" x-tvg-url="http://localhost:8080/epg" tvg-shift=0

#EXTINF:-1 tvg-chno="1" tvg-logo="http://localhost:8080/channel/test/icon"  group-title="General"   group-title="TEST"  tvg-id="test" tvg-name="test" type="hls",test
http://localhost:8080/view/1080p/test/playlist.m3u8?user=null&pass=null

$ curl "http://localhost:8080/view/1080p/test/playlist.m3u8?user=null&pass=null"

This returns empty because of java errors: (removed ffmpeg compile options for better visibility)

java.lang.NullPointerException
Client require:playlist.m3u8 after 0
[test_1080p_HLS] Start HLS
[HLS Command]ffmpeg -i http://localhost:8080/view/1080p/test?user=internal&token=O3W46DH7DLB2TB91TLSXH4WAOMBKRDGU9BBFXJGR  -c:a copy -c:v copy -sn   /opt/proxylive/tmp/test_1080p_HLS/playlist.m3u8
ffmpeg version 4.1.4-1build2 Copyright (c) 2000-2019 the FFmpeg developers
built with gcc 9 (Ubuntu 9.2.1-4ubuntu1)

java.lang.NullPointerException
require connection from : {user=internal, ip=/127.0.0.1, browserInfo=Lavf/58.20.100, streams=[DirectTranscoderStreamProcessor{channel=com.github.segator.proxylive.entity.Channel@5418723d, profile=1080p}]}
[test_1080p] Start Transcode
Transcoding Command/usr/bin/ffmpeg -i http://localhost:8080/view/raw/test?user=internal&token=O3W46DH7DLB2TB91TLSXH4WAOMBKRDGU9BBFXJGR  -acodec copy -vcodec copy  -
ffmpeg version 4.1.4-1build2 Copyright (c) 2000-2019 the FFmpeg developers
built with gcc 9 (Ubuntu 9.2.1-4ubuntu1)

java.lang.NullPointerException
require connection from : {user=internal, ip=/127.0.0.1, browserInfo=Lavf/58.20.100, streams=[DirectTranscoderStreamProcessor{channel=com.github.segator.proxylive.entity.Channel@5418723d, profile=1080p},HttpSoureStreamProcessor{channel=com.github.segator.proxylive.entity.Channel@5418723d}]}
[test] Get stream
null
[test] Get stream
null
[test] Get stream
null
[test] Get stream
null
User Close:{user=internal, ip=/127.0.0.1, browserInfo=Lavf/58.20.100, streams=[DirectTranscoderStreamProcessor{channel=com.github.segator.proxylive.entity.Channel@5418723d, profile=1080p}]}
http://localhost:8080/view/raw/test?user=internal&token=O3W46DH7DLB2TB91TLSXH4WAOMBKRDGU9BBFXJGR: Invalid data found when processing input
Error:Process is not running
Error:Process is not running
2019-12-03 05:41:22.272 ERROR 3940 --- [     test_1080p] c.g.s.p.tasks.DirectTranscodeTask        : null

java.io.IOException: Process is not running
        at com.github.segator.proxylive.tasks.DirectTranscodeTask.isRunning(DirectTranscodeTask.java:222) ~[classes!/:1.0-SNAPSHOT]
        at com.github.segator.proxylive.tasks.DirectTranscodeTask.run(DirectTranscodeTask.java:140) ~[classes!/:1.0-SNAPSHOT]
        at java.base/java.lang.Thread.run(Thread.java:834) ~[na:na]

2019-12-03 05:41:22.272 ERROR 3940 --- [read:test_1080p] c.g.s.p.tasks.DirectTranscodeTask        : null

java.io.IOException: Process is not running
        at com.github.segator.proxylive.tasks.DirectTranscodeTask.isRunning(DirectTranscodeTask.java:222) ~[classes!/:1.0-SNAPSHOT]
        at com.github.segator.proxylive.tasks.DirectTranscodeTask$1.run(DirectTranscodeTask.java:113) ~[classes!/:1.0-SNAPSHOT]

Direct raw stream, in this case "http://localhost:8080/view/raw/test?user=internal&token=O3W46DH7DLB2TB91TLSXH4WAOMBKRDGU9BBFXJGR" is also not available directly.

To note, that example stream provided in channels.json is happily consumed by ffmpeg using above arguments.

I would be very thankful for any suggestions, to what to check.

segator commented 4 years ago

Hello, it wont work but i can do it working i will have a look next week and add suport for hls sources

El mar., 3 dic. 2019 5:22, noobi0 notifications@github.com escribió:

Ok, mpegTS parameters were removed at all from application.yml file, once I returned that part with empty values, there are no default values inserted.

However, as much as I would want to start using this application, as it perfectly fits my required scenario where I have couple of dozen of streams what are unplayable unless are correctly remuxed with ffmpeg), where proxylive could simplify this a lot.

Source streams are h264 HLS m3u8 playlists.

If you want this in separate issue, and close this one, please tell, will move then.

As to minimize any misconfig, I removed / disabled everything just to keep minimal essentials. I just want it to work for now, then gradually add prometheus / grafana monitoring, epg, and other extras. Only basic ffmpeg arguments (copy) were used while testing.

$ cat channels.json

[ { "number" : 1, "name" : "test", "id" : "test", "epgID" : "test", "logoFile":"", "logoURL": "file:///opt/proxylive/logo.png", "categories" : [ "General", "TEST" ], "ffmpegParameters": "", "sources" : [ { "url" : "https://bitdash-a.akamaihd.net/content/sintel/hls/playlist.m3u8", "priority" : 1 } ]

}

]

$ cat application.yml

spring.jackson.serialization.INDENT_OUTPUT: true management: server: port: 8090 endpoints: prometheus: enabled: false metrics: enabled: false web: exposure: include: "*" metrics: export: prometheus: enabled: false security: enabled: false

geoIP: enabled: false url: https://geolite.maxmind.com/download/geoip/database/GeoLite2-City.tar.gz source: epg: refresh: 600 channels: refresh: 60 type: json url: file:///opt/proxylive/channels.json reconnectTimeout: 15 streamTimeout: 30 ffmpeg: path: '/usr/bin/ffmpeg' profiles:

        alias: "aac"
        parameters: "-i {input} {channelParameters}"
    -
        alias: "240p"
        parameters: "-i {input} {channelParameters}"
    -
        alias: "360p"
        parameters: "-i {input} {channelParameters}"
    -
        alias: "480p"
        parameters: "-i {input} {channelParameters}"
    -
        alias: "720p"
        parameters: "-i {input} {channelParameters}"
    -
        alias: "1080p"
        parameters:  "-i {input} {channelParameters} -acodec copy -vcodec copy"
    -
        alias: "vaapi1080p"
        parameters: "-i {input} {channelParameters}"
mpegTS:
    parameters: ""
hls:
    enabled: true
    tempPath: "/opt/proxylive/tmp"
    parameters: ""
    timeout: 30

buffers: chunkSize: 131072 broadcastBufferSize: 52428800

$ java -jar proxylive.jar

. _ _ /\ / '_ () \ \ \ \ ( ( )\ | ' | '| | ' \/ ` | \ \ \ \ \/ _)| |)| | | | | || (| | ) ) ) ) ' |__| .|| ||| |\, | / / / / =========|_|==============|__/=//// :: Spring Boot :: (v2.1.3.RELEASE)

2019-12-03 05:23:44.849 INFO 3531 --- [ main] c.github.segator.proxylive.Application : Starting Application v1.0-SNAPSHOT on server with PID 3531 (/opt/proxylive/proxylive.jar started by root in /opt/proxylive) 2019-12-03 05:23:44.853 INFO 3531 --- [ main] c.github.segator.proxylive.Application : No active profile set, falling back to default profiles: default 2019-12-03 05:23:47.093 INFO 3531 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat initialized with port(s): 8080 (http) 2019-12-03 05:23:47.155 INFO 3531 --- [ main] o.apache.catalina.core.StandardService : Starting service [Tomcat] 2019-12-03 05:23:47.156 INFO 3531 --- [ main] org.apache.catalina.core.StandardEngine : Starting Servlet engine: [Apache Tomcat/9.0.16] 2019-12-03 05:23:47.173 INFO 3531 --- [ main] o.a.catalina.core.AprLifecycleListener : The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: [/usr/java/packages/lib:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib] 2019-12-03 05:23:47.291 INFO 3531 --- [ main] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring embedded WebApplicationContext 2019-12-03 05:23:47.291 INFO 3531 --- [ main] o.s.web.context.ContextLoader : Root WebApplicationContext: initialization completed in 2355 ms Channels loaded 2019-12-03 05:23:48.098 INFO 3531 --- [ main] o.s.s.concurrent.ThreadPoolTaskExecutor : Initializing ExecutorService 'applicationTaskExecutor' 2019-12-03 05:23:48.604 INFO 3531 --- [ main] o.s.s.c.ThreadPoolTaskScheduler : Initializing ExecutorService 'taskScheduler' 2019-12-03 05:23:48.782 INFO 3531 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat initialized with port(s): 8090 (http) 2019-12-03 05:23:48.784 INFO 3531 --- [ main] o.apache.catalina.core.StandardService : Starting service [Tomcat] 2019-12-03 05:23:48.784 INFO 3531 --- [ main] org.apache.catalina.core.StandardEngine : Starting Servlet engine: [Apache Tomcat/9.0.16] 2019-12-03 05:23:48.788 INFO 3531 --- [ main] o.a.c.c.C.[Tomcat-1].[localhost].[/] : Initializing Spring embedded WebApplicationContext 2019-12-03 05:23:48.789 INFO 3531 --- [ main] o.s.web.context.ContextLoader : Root WebApplicationContext: initialization completed in 129 ms 2019-12-03 05:23:48.836 INFO 3531 --- [ main] o.s.b.a.e.web.EndpointLinksResolver : Exposing 15 endpoint(s) beneath base path '/actuator' 2019-12-03 05:23:48.955 INFO 3531 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8090 (http) with context path '' 2019-12-03 05:23:49.028 INFO 3531 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8080 (http) with context path '' 2019-12-03 05:23:49.030 INFO 3531 --- [ main] c.github.segator.proxylive.Application : Started Application in 4.953 seconds (JVM running for 5.486)

$ curl http://localhost:8080/channel/list/hls/1080p -o test.m3u8

$ cat test.m3u8

EXTM3U cache=2000 url-tvg="http://localhost:8080/epg" x-tvg-url="http://localhost:8080/epg" tvg-shift=0

EXTINF:-1 tvg-chno="1" tvg-logo="http://localhost:8080/channel/test/icon" group-title="General" group-title="TEST" tvg-id="test" tvg-name="test" type="hls",testhttp://localhost:8080/view/1080p/test/playlist.m3u8?user=null&pass=null

$ curl " http://localhost:8080/view/1080p/test/playlist.m3u8?user=null&pass=null"

This returns empty because of java errors: (removed ffmpeg compile options for better visibility)

java.lang.NullPointerException Client require:playlist.m3u8 after 0 [test_1080p_HLS] Start HLS [HLS Command]ffmpeg -i http://localhost:8080/view/1080p/test?user=internal&token=O3W46DH7DLB2TB91TLSXH4WAOMBKRDGU9BBFXJGR -c:a copy -c:v copy -sn /opt/proxylive/tmp/test_1080p_HLS/playlist.m3u8 ffmpeg version 4.1.4-1build2 Copyright (c) 2000-2019 the FFmpeg developers built with gcc 9 (Ubuntu 9.2.1-4ubuntu1)

java.lang.NullPointerException require connection from : {user=internal, ip=/127.0.0.1, browserInfo=Lavf/58.20.100, streams=[DirectTranscoderStreamProcessor{channel=com.github.segator.proxylive.entity.Channel@5418723d, profile=1080p}]} [test_1080p] Start Transcode Transcoding Command/usr/bin/ffmpeg -i http://localhost:8080/view/raw/test?user=internal&token=O3W46DH7DLB2TB91TLSXH4WAOMBKRDGU9BBFXJGR -acodec copy -vcodec copy - ffmpeg version 4.1.4-1build2 Copyright (c) 2000-2019 the FFmpeg developers built with gcc 9 (Ubuntu 9.2.1-4ubuntu1)

java.lang.NullPointerException require connection from : {user=internal, ip=/127.0.0.1, browserInfo=Lavf/58.20.100, streams=[DirectTranscoderStreamProcessor{channel=com.github.segator.proxylive.entity.Channel@5418723d, profile=1080p},HttpSoureStreamProcessor{channel=com.github.segator.proxylive.entity.Channel@5418723d}]} [test] Get stream null [test] Get stream null [test] Get stream null [test] Get stream null User Close:{user=internal, ip=/127.0.0.1, browserInfo=Lavf/58.20.100, streams=[DirectTranscoderStreamProcessor{channel=com.github.segator.proxylive.entity.Channel@5418723d, profile=1080p}]}http://localhost:8080/view/raw/test?user=internal&token=O3W46DH7DLB2TB91TLSXH4WAOMBKRDGU9BBFXJGR: Invalid data found when processing input Error:Process is not running Error:Process is not running 2019-12-03 05:41:22.272 ERROR 3940 --- [ test_1080p] c.g.s.p.tasks.DirectTranscodeTask : null

java.io.IOException: Process is not running at com.github.segator.proxylive.tasks.DirectTranscodeTask.isRunning(DirectTranscodeTask.java:222) ~[classes!/:1.0-SNAPSHOT] at com.github.segator.proxylive.tasks.DirectTranscodeTask.run(DirectTranscodeTask.java:140) ~[classes!/:1.0-SNAPSHOT] at java.base/java.lang.Thread.run(Thread.java:834) ~[na:na]

2019-12-03 05:41:22.272 ERROR 3940 --- [read:test_1080p] c.g.s.p.tasks.DirectTranscodeTask : null

java.io.IOException: Process is not running at com.github.segator.proxylive.tasks.DirectTranscodeTask.isRunning(DirectTranscodeTask.java:222) ~[classes!/:1.0-SNAPSHOT] at com.github.segator.proxylive.tasks.DirectTranscodeTask$1.run(DirectTranscodeTask.java:113) ~[classes!/:1.0-SNAPSHOT]

Direct raw stream, in this case " http://localhost:8080/view/raw/test?user=internal&token=O3W46DH7DLB2TB91TLSXH4WAOMBKRDGU9BBFXJGR" is also not available directly.

To note, that example stream provided in channels.json is happily consumed by ffmpeg using above arguments.

I would be very thankful for any suggestions, to what to check.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/segator/proxylive/issues/9?email_source=notifications&email_token=AAR5IY5DUE3UZDDY56OYWY3QWXNH5A5CNFSM4I2Y4KZKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEFYA5IA#issuecomment-560991904, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAR5IYZINWUQ5URDNECSWC3QWXNH5ANCNFSM4I2Y4KZA .

segator commented 4 years ago

I did a POC and HLS work as source but i need to cleanup the code I will push the changes before end of year

segator commented 4 years ago

I just added HLS/DASh/RTMP/RTSP support for backends please check here https://github.com/segator/proxylive/commit/809365df9f669ec68c290db1724d745294b649aa

and let me know if works for you.

noobi0 commented 4 years ago

Well, great news Proxylive Broadcaster will now support HLS as ingress streams, thank You!

Compiled latest version, and for just testing video stream for now was using "https://bitdash-a.akamaihd.net/content/sintel/hls/video/250kbit.m3u8" what is single video stream, as playlist mentioned above contained multiple audio and video streams, as well as subtitles.

$ java -jar proxylive.jar 
               (      (         )       )       )   (      (
               )\ )   )\ )   ( /(    ( /(    ( /(   )\ )   )\ )
              (()/(  (()/(   )\())   )\())   )\()) (()/(  (()/(   (   (    (
 ___   ___     /(_))  /(_)) ((_)\   ((_)\   ((_)\   /(_))  /(_))  )\  )\   )\      ___   ___
|___| |___|   (_))   (_))     ((_)  __((_) __ ((_) (_))   (_))   ((_)((_) ((_)    |___| |___|
              | _ \  | _ \   / _ \  \ \/ / \ \ / / | |    |_ _|  \ \ / /  | __|
              |  _/  |   /  | (_) |  >  <   \ V /  | |__   | |    \ V /   | _|
              |_|    |_|_\   \___/  /_/\_\   |_|   |____| |___|    \_/    |___|

            (         )              (                         (                     (
       (    )\ )   ( /(      (       )\ )      (       (       )\ )    *   )         )\ )
     ( )\  (()/(   )\())     )\     (()/(      )\      )\     (()/(  ` )  /(   (    (()/(
     )((_)  /(_)) ((_)\   ((((_)(    /(_))   (((_)  ((((_)(    /(_))  ( )(_))  )\    /(_))
    ((_)_  (_))     ((_)   )\ _ )\  (_))_    )\___   )\ _ )\  (_))   (_(_())  ((_)  (_))
     | _ ) | _ \   / _ \   (_)_\(_)  |   \  ((/ __|  (_)_\(_) / __|  |_   _|  | __| | _ \
     | _ \ |   /  | (_) |   / _ \    | |) |  | (__    / _ \   \__ \    | |    | _|  |   /
     |___/ |_|_\   \___/   /_/ \_\   |___/    \___|  /_/ \_\  |___/    |_|    |___| |_|_\

2019-12-09 09:04:42,132 INFO  [main] org.springframework.boot.StartupInfoLogger: Starting Application v1.0-SNAPSHOT on debian with PID 2897 (/opt/proxylive/proxylive.jar started by root in /opt/proxylive)
2019-12-09 09:04:42,148 DEBUG [main] org.springframework.boot.StartupInfoLogger: Running with Spring Boot v2.1.3.RELEASE, Spring v5.1.5.RELEASE
2019-12-09 09:04:42,149 INFO  [main] org.springframework.boot.SpringApplication: No active profile set, falling back to default profiles: default
2019-12-09 09:04:44,394 INFO  [main] org.springframework.boot.web.embedded.tomcat.TomcatWebServer: Tomcat initialized with port(s): 8080 (http)
2019-12-09 09:04:44,464 INFO  [main] org.apache.juli.logging.DirectJDKLog: Initializing ProtocolHandler ["http-nio-8080"]
2019-12-09 09:04:44,480 INFO  [main] org.apache.juli.logging.DirectJDKLog: Starting service [Tomcat]
2019-12-09 09:04:44,480 INFO  [main] org.apache.juli.logging.DirectJDKLog: Starting Servlet engine: [Apache Tomcat/9.0.16]
2019-12-09 09:04:44,498 INFO  [main] org.apache.juli.logging.DirectJDKLog: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: [/usr/java/packages/lib:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib]
2019-12-09 09:04:44,590 INFO  [main] org.apache.juli.logging.DirectJDKLog: Initializing Spring embedded WebApplicationContext
2019-12-09 09:04:44,590 INFO  [main] org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext: Root WebApplicationContext: initialization completed in 2343 ms
2019-12-09 09:04:44,722 INFO  [main] com.github.segator.proxylive.config.ProxyLiveConfiguration: Your internal token is:B25G9WTCZ5VH74Q99WK0IXQPVL4WHWDJ4H8F458D
2019-12-09 09:04:44,999 INFO  [main] com.github.segator.proxylive.service.ChannelURLService: Channels loaded
2019-12-09 09:04:45,233 INFO  [main] org.springframework.scheduling.concurrent.ExecutorConfigurationSupport: Initializing ExecutorService 'applicationTaskExecutor'
2019-12-09 09:04:45,813 INFO  [main] org.springframework.scheduling.concurrent.ExecutorConfigurationSupport: Initializing ExecutorService 'taskScheduler'
2019-12-09 09:04:45,970 INFO  [main] org.springframework.boot.web.embedded.tomcat.TomcatWebServer: Tomcat initialized with port(s): 8090 (http)
2019-12-09 09:04:45,971 INFO  [main] org.apache.juli.logging.DirectJDKLog: Initializing ProtocolHandler ["http-nio-8090"]
2019-12-09 09:04:45,972 INFO  [main] org.apache.juli.logging.DirectJDKLog: Starting service [Tomcat]
2019-12-09 09:04:45,973 INFO  [main] org.apache.juli.logging.DirectJDKLog: Starting Servlet engine: [Apache Tomcat/9.0.16]
2019-12-09 09:04:45,978 INFO  [main] org.apache.juli.logging.DirectJDKLog: Initializing Spring embedded WebApplicationContext
2019-12-09 09:04:45,978 INFO  [main] org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext: Root WebApplicationContext: initialization completed in 121 ms
2019-12-09 09:04:46,024 INFO  [main] org.springframework.boot.actuate.endpoint.web.EndpointLinksResolver: Exposing 15 endpoint(s) beneath base path '/actuator'
2019-12-09 09:04:46,123 INFO  [main] org.apache.juli.logging.DirectJDKLog: Starting ProtocolHandler ["http-nio-8090"]
2019-12-09 09:04:46,155 INFO  [main] org.springframework.boot.web.embedded.tomcat.TomcatWebServer: Tomcat started on port(s): 8090 (http) with context path ''
2019-12-09 09:04:46,175 INFO  [main] org.apache.juli.logging.DirectJDKLog: Starting ProtocolHandler ["http-nio-8080"]
2019-12-09 09:04:46,185 INFO  [main] org.springframework.boot.web.embedded.tomcat.TomcatWebServer: Tomcat started on port(s): 8080 (http) with context path ''
2019-12-09 09:04:46,189 INFO  [main] org.springframework.boot.StartupInfoLogger: Started Application in 5.132 seconds (JVM running for 5.699)

After requesting "http://localhost:8080/view/1080p/test/playlist.m3u8?user=null&pass=null"

2019-12-09 09:06:46,950 INFO  [http-nio-8080-exec-1] org.apache.juli.logging.DirectJDKLog: Initializing Spring DispatcherServlet 'dispatcherServlet'
2019-12-09 09:06:46,951 INFO  [http-nio-8080-exec-1] org.springframework.web.servlet.FrameworkServlet: Initializing Servlet 'dispatcherServlet'
2019-12-09 09:06:46,960 INFO  [http-nio-8080-exec-1] org.springframework.web.servlet.FrameworkServlet: Completed initialization in 9 ms
2019-12-09 09:07:02,214 DEBUG [http-nio-8080-exec-2] com.github.segator.proxylive.controller.StreamController: Client require:playlist.m3u8 after 0
2019-12-09 09:07:02,215 DEBUG [test_1080p_HLS] com.github.segator.proxylive.tasks.HLSDirectTask: [test_1080p_HLS] Start HLS
2019-12-09 09:07:02,221 DEBUG [test_1080p_HLS] com.github.segator.proxylive.tasks.HLSDirectTask: [test_1080p_HLS] HLS ffmpeg command: ffmpeg -i http://localhost:8080/view/1080p/test?user=internal&token=B25G9WTCZ5VH74Q99WK0IXQPVL4WHWDJ4H8F458D  -c:a copy -c:v copy -sn   /opt/proxylive/tmp/test_1080p_HLS/playlist.m3u8
2019-12-09 09:07:02,238 DEBUG [test_1080p_HLS] com.github.segator.proxylive.tasks.HLSDirectTask: [test_1080p_HLS] ffmpeg version 4.2.1-static https://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2019 the FFmpeg developers
2019-12-09 09:07:02,238 DEBUG [test_1080p_HLS] com.github.segator.proxylive.tasks.HLSDirectTask: [test_1080p_HLS]   built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
2019-12-09 09:07:02,239 DEBUG [test_1080p_HLS] com.github.segator.proxylive.tasks.HLSDirectTask: [test_1080p_HLS]   configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc-6 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-libgme --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d --enable-libxvid --enable-libzvbi --enable-libzimg
2019-12-09 09:07:02,239 DEBUG [test_1080p_HLS] com.github.segator.proxylive.tasks.HLSDirectTask: [test_1080p_HLS]   libavutil      56. 31.100 / 56. 31.100
2019-12-09 09:07:02,240 DEBUG [test_1080p_HLS] com.github.segator.proxylive.tasks.HLSDirectTask: [test_1080p_HLS]   libavcodec     58. 54.100 / 58. 54.100
2019-12-09 09:07:02,240 DEBUG [test_1080p_HLS] com.github.segator.proxylive.tasks.HLSDirectTask: [test_1080p_HLS]   libavformat    58. 29.100 / 58. 29.100
2019-12-09 09:07:02,240 DEBUG [test_1080p_HLS] com.github.segator.proxylive.tasks.HLSDirectTask: [test_1080p_HLS]   libavdevice    58.  8.100 / 58.  8.100
2019-12-09 09:07:02,241 DEBUG [test_1080p_HLS] com.github.segator.proxylive.tasks.HLSDirectTask: [test_1080p_HLS]   libavfilter     7. 57.100 /  7. 57.100
2019-12-09 09:07:02,241 DEBUG [test_1080p_HLS] com.github.segator.proxylive.tasks.HLSDirectTask: [test_1080p_HLS]   libswscale      5.  5.100 /  5.  5.100
2019-12-09 09:07:02,241 DEBUG [test_1080p_HLS] com.github.segator.proxylive.tasks.HLSDirectTask: [test_1080p_HLS]   libswresample   3.  5.100 /  3.  5.100
2019-12-09 09:07:02,242 DEBUG [test_1080p_HLS] com.github.segator.proxylive.tasks.HLSDirectTask: [test_1080p_HLS]   libpostproc    55.  5.100 / 55.  5.100
2019-12-09 09:07:02,255 DEBUG [http-nio-8080-exec-3] com.github.segator.proxylive.controller.StreamController: Open Stream test by internal
2019-12-09 09:07:02,282 DEBUG [test_1080p] com.github.segator.proxylive.tasks.DirectTranscodeTask: [test_1080p] Start Transcode
2019-12-09 09:07:02,284 DEBUG [test_1080p] com.github.segator.proxylive.tasks.DirectTranscodeTask: [test_1080p] Transcoding Command: /usr/bin/ffmpeg -i http://localhost:8080/view/raw/test?user=internal&token=B25G9WTCZ5VH74Q99WK0IXQPVL4WHWDJ4H8F458D  -acodec copy -vcodec copy  -
2019-12-09 09:07:02,308 DEBUG [http-nio-8080-exec-4] com.github.segator.proxylive.controller.StreamController: Open Stream test by internal
2019-12-09 09:07:02,309 DEBUG [Transcoding Error Reader Thread:test_1080p] com.github.segator.proxylive.tasks.DirectTranscodeTask$1: [test_1080p] ffmpeg version 4.2.1-static https://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2019 the FFmpeg developers
2019-12-09 09:07:02,316 DEBUG [Transcoding Error Reader Thread:test_1080p] com.github.segator.proxylive.tasks.DirectTranscodeTask$1: [test_1080p]   built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
2019-12-09 09:07:02,316 DEBUG [Transcoding Error Reader Thread:test_1080p] com.github.segator.proxylive.tasks.DirectTranscodeTask$1: [test_1080p]   configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc-6 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-libgme --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d --enable-libxvid --enable-libzvbi --enable-libzimg
2019-12-09 09:07:02,317 DEBUG [Transcoding Error Reader Thread:test_1080p] com.github.segator.proxylive.tasks.DirectTranscodeTask$1: [test_1080p]   libavutil      56. 31.100 / 56. 31.100
2019-12-09 09:07:02,317 DEBUG [Transcoding Error Reader Thread:test_1080p] com.github.segator.proxylive.tasks.DirectTranscodeTask$1: [test_1080p]   libavcodec     58. 54.100 / 58. 54.100
2019-12-09 09:07:02,317 DEBUG [Transcoding Error Reader Thread:test_1080p] com.github.segator.proxylive.tasks.DirectTranscodeTask$1: [test_1080p]   libavformat    58. 29.100 / 58. 29.100
2019-12-09 09:07:02,318 DEBUG [Transcoding Error Reader Thread:test_1080p] com.github.segator.proxylive.tasks.DirectTranscodeTask$1: [test_1080p]   libavdevice    58.  8.100 / 58.  8.100
2019-12-09 09:07:02,318 DEBUG [Transcoding Error Reader Thread:test_1080p] com.github.segator.proxylive.tasks.DirectTranscodeTask$1: [test_1080p]   libavfilter     7. 57.100 /  7. 57.100
2019-12-09 09:07:02,318 DEBUG [Transcoding Error Reader Thread:test_1080p] com.github.segator.proxylive.tasks.DirectTranscodeTask$1: [test_1080p]   libswscale      5.  5.100 /  5.  5.100
2019-12-09 09:07:02,318 DEBUG [Transcoding Error Reader Thread:test_1080p] com.github.segator.proxylive.tasks.DirectTranscodeTask$1: [test_1080p]   libswresample   3.  5.100 /  3.  5.100
2019-12-09 09:07:02,319 DEBUG [Transcoding Error Reader Thread:test_1080p] com.github.segator.proxylive.tasks.DirectTranscodeTask$1: [test_1080p]   libpostproc    55.  5.100 / 55.  5.100
2019-12-09 09:07:02,334 DEBUG [test] com.github.segator.proxylive.tasks.HttpDownloaderTask: [id:test,priority:1] Get Stream -- https://bitdash-a.akamaihd.net/content/sintel/hls/video/250kbit.m3u8
2019-12-09 09:07:32,242 DEBUG [http-nio-8080-exec-2] com.github.segator.proxylive.controller.StreamController: Client get Stream:playlist.m3u8 after 30
2019-12-09 09:07:32,247 DEBUG [http-nio-8080-exec-2] com.github.segator.proxylive.controller.StreamController: Client exit(404) request:playlist.m3u8 after 30
2019-12-09 09:07:36,303 DEBUG [scheduling-1] com.github.segator.proxylive.tasks.HLSDirectTask: [test_1080p_HLS] Required Terminate HLS
2019-12-09 09:07:36,305 DEBUG [test_1080p_HLS] com.github.segator.proxylive.tasks.HLSDirectTask: [test_1080p_HLS] Terminated HLS
2019-12-09 09:08:02,294 DEBUG [http-nio-8080-exec-3] com.github.segator.proxylive.tasks.DirectTranscodeTask: [test_1080p] Required Terminate Transcode
2019-12-09 09:08:02,445 DEBUG [test_1080p] com.github.segator.proxylive.tasks.DirectTranscodeTask: [test_1080p] Terminated Transcode
2019-12-09 09:08:02,448 DEBUG [http-nio-8080-exec-3] com.github.segator.proxylive.controller.StreamController: Close Stream test by internal
2019-12-09 09:08:02,956 ERROR [test] com.github.segator.proxylive.tasks.HttpDownloaderTask: [id:test,priority:1] no data received on 60 seconds -- https://bitdash-a.akamaihd.net/content/sintel/hls/video/250kbit.m3u8
2019-12-09 09:08:02,957 DEBUG [test] com.github.segator.proxylive.tasks.HttpDownloaderTask: [id:test,priority:1] Get Stream -- https://bitdash-a.akamaihd.net/content/sintel/hls/video/250kbit.m3u8
2019-12-09 09:08:03,013 DEBUG [http-nio-8080-exec-4] com.github.segator.proxylive.controller.StreamController: Close Stream test by internal
segator commented 4 years ago

Ok don't worry I will have a look late :) By the way HLS is not prod ready, I implemented but I'm not using it so can have bugs. But happy to fix them as soon you detect then.

segator commented 4 years ago

Hello I tried it doesn't work because this link you are trying is not live TV. Try with this one. hlss://hlsliveamdgl0-lh.akamaihd.net/i/hlsdvrlive_1@60531/index_1500_av-p.m3u8?sd=10&rebase=on

remember to change http or https to hls hlss (this is what I use to recognize is a hls or dash stream, this need to be improved in the future and will be auto detected.

noobi0 commented 4 years ago

Hi, thank you for following up, reporting my findings so far. Good news is that I finally managed to get video output, what is great, however it seems to me, that even when using HLS in URL, mpegTS is still used. Let me explain, used your provided source. Original source:

Stream #0:0: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p, 1024x576 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc
Stream #0:1: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp

HLS section in application.yml

    hls:
        enabled: true
        tempPath: "/opt/proxylive/tmp"
        parameters: ""
        timeout: 30

Trying to play http://dev:8080/channel/list/hls/1080p (what calls http://dev:8080/view/1080p/test/playlist.m3u8), DEBUG message from Proxylive Broadcaster:

DEBUG [test_1080p] com.github.segator.proxylive.tasks.DirectTranscodeTask: [test_1080p] Transcoding Command: /usr/bin/ffmpeg -i http://localhost:8080/view/raw/test?user=internal&token=P0Y7RGXGRT91ME1R3R00D64X2PFDQ2TJDI3YDZ9J  -threads 0 -f mpegts -mpegts_m2ts_mode 1 -mpegts_copyts 1 -mpegts_flags +resend_headers  -nostats -hide_banner -

Output video format now is mpeg2video:

Stream #0:0: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p(tv), 1024x576 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc
Stream #0:1: Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, s16p, 384 kb/s

As obviously mpegTS parameters were used, even when called for HLS.

segator commented 4 years ago

Yes this is because we work on layers for better stream reusability , source loop, transcoding loop and hls/dash loop, anyway those it cause you any issue? Im not an expert using ffmpeg so maybe i skip something important on my design? Matbe you can give more feedback about how a proper hls should be

El mié., 11 dic. 2019 6:51, noobi0 notifications@github.com escribió:

Hi, thank you for following up, reporting my findings so far. Good news is that I finally managed to get video output, what is great, however it seems to me, that even when using HLS in URL, mpegTS is still used. Let me explain, used your provided source. Original source:

Stream #0:0: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p, 1024x576 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc Stream #0:1: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp

HLS section in application.yml

hls:
    enabled: true
    tempPath: "/opt/proxylive/tmp"
    parameters: ""
    timeout: 30

Trying to play http://dev:8080/channel/list/hls/1080p (what calls http://dev:8080/view/1080p/test/playlist.m3u8), DEBUG message from Proxylive Broadcaster:

DEBUG [test_1080p] com.github.segator.proxylive.tasks.DirectTranscodeTask: [test_1080p] Transcoding Command: /usr/bin/ffmpeg -i http://localhost:8080/view/raw/test?user=internal&token=P0Y7RGXGRT91ME1R3R00D64X2PFDQ2TJDI3YDZ9J -threads 0 -f mpegts -mpegts_m2ts_mode 1 -mpegts_copyts 1 -mpegts_flags +resend_headers -nostats -hide_banner -

Output video format now is mpeg2video:

Stream #0:0: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p(tv), 1024x576 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc Stream #0:1: Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, s16p, 384 kb/s

As obviously mpegTS parameters were used, even when called for HLS.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/segator/proxylive/issues/9?email_source=notifications&email_token=AAR5IY2AN32YYZOS4DZX5T3QYB5XRA5CNFSM4I2Y4KZKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGR62SA#issuecomment-564391240, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAR5IY6IGZXEP4563DQ5CN3QYB5XRANCNFSM4I2Y4KZA .

segator commented 4 years ago

any new on that?