LMS-Community / slimserver

Server for Squeezebox and compatible players. This server is also called Lyrion Music Server.
https://lyrion.org
Other
1.16k stars 293 forks source link

Is it time to move to http/1.1 by default for outbound connections? #1121

Open PaulWebster opened 3 months ago

PaulWebster commented 3 months ago

Occasionally (admittedly rarely) I have had issues when trying to play radio streams and have traced one problem to LMS using http/1.0 In one case the block is being done by Cloudflare .. with the response "This website is using a security service to protect itself from online attacks. The action you just performed triggered the security solution. There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data."

To show in action .... try using curl to access h t t p s : / / station.thecheese.co.nz/hls/the_cheese/stream_high.m3u8 then try again with --http1.0 added to the curl command

I override the version when fetching metadata for this station (although their metadata is currently broken) - but I do not get involved with the stream playing so that now fails since they changed their stream URL to the one above.

https://github.com/LMS-Community/slimserver/blob/8c759394ecbbd37941ca7bf773682af561a41078/Slim/Networking/Async/HTTP.pm#L305

michaelherger commented 3 months ago

@philippe44 you had done some important work to support http 1.1 a few years back (see eg. the line @PaulWebster is pointing to). I honestly don't understand what the impact might be moving all requests to 1.1 by default. Would you have any guess?

I believe the above line isn't what is causing the issue. But the scanner would request 1.0 by default. If you changed it to request 1.1 instead it would pass the station you listed (I believe):

diff --git a/Slim/Utils/Scanner/Remote.pm b/Slim/Utils/Scanner/Remote.pm
index 6fbad4da8..ee8f29e2c 100644
--- a/Slim/Utils/Scanner/Remote.pm
+++ b/Slim/Utils/Scanner/Remote.pm
@@ -203,6 +203,7 @@ sub scanURL {

        # Connect to the remote URL and figure out what it is
        my $request = HTTP::Request->new( GET => $url );
+       $request->protocol('HTTP/1.1');

        main::DEBUGLOG && $log->is_debug && $log->debug("Scanning remote URL $url");
PaulWebster commented 3 months ago

I came across another one that does not like HTTP/1.0 (recent addition to hisresaudio.online) h t t p s : / / radio.audiomastering.lt/ ogg/flac stream at h t t p s : / / transliacija.audiomastering.lt/pure Accessing using 1.0 gives a 403 Forbidden

I then changed Utils/Scanner/Remote.pm as earlier suggested to request 1.1 and it then gets a 200 OK.

However, it stops almost immediately

[24-06-19 22:30:51.1812] Slim::Networking::Async::HTTP::_http_read_body (530) Read body: [387] bytes

[24-06-19 22:30:51.1819] Slim::Networking::Async::disconnect (252) Close Slim::Networking::Async::Socket::HTTPS=GLOB(0xacea2f0) => 85

[24-06-19 22:30:51.1834] Slim::Networking::Async::HTTP::_http_read_body (579) Body read (stopped after 387 bytes)

I see that they use chunked encoding but I have seen other sources using chunked encoding with LMS and it seems to work.

Same stream URL does play in vlc

michaelherger commented 3 months ago

I'm sorry, this is nothing I'm familiar with... maybe @bpa-code has a better understanding?

bpa-code commented 3 months ago

I only used started using HTTP 1.1 for HLS via a plugin lib module to have persistent connections. Philippe implemented fully within LMS possibly for UPNP or bridge stuff.

HTTP 1.1 has other additions besides persistent connection not sure of their status with LMS implementation. If we enable HTTP 1.1 by default, what HTTP 1.1 features will be requested by LMS or LMS plugins ?
If none then I think it is unlikely to be a problem as most sites have implemented 1.1 as far LMS applications usage.

awy commented 3 months ago

You'll get chunked encoding responses which won't be unpacked for you if using AnyEvent::HTTP's want_steam_handle.

On Sat, 22 Jun 2024, 16:40 bpa, @.***> wrote:

I only used started using HTTP 1.1 for HLS via a plugin lib module to have persistent connections. Philippe implemented fully within LMS possibly for UPNP or bridge stuff.

HTTP 1.1 has other additions besides persistent connection not sure of their status with LMS implementation. If we enable HTTP 1.1 by default, what HTTP 1.1 features will be requested by LMS or LMS plugins ? If none then I think it is unlikely to be a problem as most sites have implemented 1.1 as far LMS applications usage.

— Reply to this email directly, view it on GitHub https://github.com/LMS-Community/slimserver/issues/1121#issuecomment-2184076892, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAL3JV23Y5MYNW4ZUMAS453ZIWLGXAVCNFSM6AAAAABJMM2E36VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBUGA3TMOBZGI . You are receiving this because you are subscribed to this thread.Message ID: @.***>

bpa-code commented 3 months ago

Not clear what point the AnyEvent::HTTP comment is addressing.

Referring to initial comment - I think any HLS (i.e.m3u8) or DASH stream will expect to use HTTP 1.1 - HLS/DASH has a performance hit if TCP connection is not persistent - it is more significant to broadcasters/CDNs that to LMS users.

Chunked encoding seems to be handled within LMS at certain places so I'm guessing changing default will not an issues for existing LMS application.

If the default was changed to HTTP 1.1, should calls like SimpleAsynchHTTP have "$response->header( 'Transfer-Encoding' => 'chunked' )" added and if so would there be any noticeable behaviour change for plugins etc.

awy commented 3 months ago

Not clear what point the AnyEvent::HTTP comment is addressing.

I was referring to the point that, when AnyEvent::HTTP is used to fetch a stream, typically using want_body_handle => 1, then the code that takes the resulting body handle has to be able to deal with chunked encoding itself, if HTTP1.1 was used. That small complication does not occur with HTTP/1.0. I have not checked but it may be that this is already handled everywhere that it needs to be.

philippe44 commented 3 months ago

Are you sure ? When I made modifications to the LMS http code a long while ago for 1.1, I think I did what was necessary to handle chunked encoding and persistent connections, i.e not close when whole body has been received. The lower level code which is some Net::XXX AFAIR is in fact already handles mostly the chunk aggregation over Async.

awy commented 3 months ago

When I made modifications to the LMS http code a long while ago for 1.1, I think I did what was necessary to handle chunked encoding

Great! Like I said, I had not actually checked current use. I'll shut up now.

philippe44 commented 3 months ago

But I'm not 100% sure, that was a while ago ...

bpa-code commented 3 months ago

Looking though LMS code - I think all the chunked support is as LMS as a recipient of a request (e.g. a JSONRPC or a UPNP request) requiring chunked encoding.

The issues is about LMS making HTTP 1.1 requests.

There is a comment in routine send_request in Slim::Networking::Async

    # the used class is NET::HTTP::Method which now supports HTTP 1.1
    # XXX until we support chunked encoding, force 1.0
    # $self->request->protocol('HTTP/1.0');

I think we need to get clarification whether this comment is out of date now.

I'm not sure whether LMS as a client supports sending HTTP 1.1. request asking for chunked encoding. The pertinent question may be whether LMS as a client needs to support chunked-encoding.

About 2 yrs ago, I did a test version of my PlayHLS using LMS routines which essentially did

    $headers = [ 'Connection' , 'keep-alive'];
    $chunkrequest = HTTP::Request->new( GET => $url, $headers);
    $chunkrequest->protocol('HTTP/1.1');
        $hlsconnect->send_request(
            {
                request => $chunkrequest,

So they are HTTP 1.1 requests but with no headers to have chunked encoding. It worked OK probably because HLS chunks have definite length (i.e. not streams).

I think the Paul's original request is similar. The expected replies are definite length so "stream" is not required.

If there is no need to support "chunked encoding" in HTTP 1.1. GET request - then it is probably OK to enable HTTP 1.1 by default.

michaelherger commented 3 months ago

When Andy commented out that line he left the following commit message:

Force HTTP/1.0, we don't support chunked encoding in the HTTP client and this can break some sites

I don't know how or if this is relevant. Just wanted to mention it.

Moonbase59 commented 3 months ago

@PaulWebster asked me in the forum to add to this if my results were different.

So I used above https://station.thecheese.co.nz/hls/the_cheese/stream_high.m3u8 link and curl 7.81.0 on Linux:

$ curl -v --http1.0 https://station.thecheese.co.nz/hls/the_cheese/stream_high.m3u8
*   Trying 2a06:98c1:3120::7:443...
* Connected to station.thecheese.co.nz (2a06:98c1:3120::7) port 443 (#0)
* ALPN, offering http/1.1
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS header, Finished (20):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.2 (OUT), TLS header, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: CN=thecheese.co.nz
*  start date: Jun 23 08:30:27 2024 GMT
*  expire date: Sep 21 08:30:26 2024 GMT
*  subjectAltName: host "station.thecheese.co.nz" matched cert's "*.thecheese.co.nz"
*  issuer: C=US; O=Google Trust Services; CN=WE1
*  SSL certificate verify ok.
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
> GET /hls/the_cheese/stream_high.m3u8 HTTP/1.0
> Host: station.thecheese.co.nz
> User-Agent: curl/7.81.0
> Accept: */*
> 
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Tue, 02 Jul 2024 21:51:50 GMT
< Content-Type: application/x-mpegurl
< Content-Length: 381
< Connection: close
< accept-ranges: bytes
< access-control-allow-origin: *
< Cache-Control: no-cache
< etag: "66847673-17d"
< last-modified: Tue, 02 Jul 2024 21:51:47 GMT
< CF-Cache-Status: DYNAMIC
< Report-To: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v4?s=EkgacnSj6z0x2tqtVN2vSguKB0YIauNVejViBsax3pSLkXyplLLvNMZhP7UreSOEHods05MtnNYgDhJww25XAURW1KDsSLqZPWzXr3iBP4iVf6QgapkfDC38mjdJKfwzSSqDy8Z6%2FA%2FHiULbOrYnIBdL1B98jA%3D%3D"}],"group":"cf-nel","max_age":604800}
< NEL: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
< Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
< X-Content-Type-Options: nosniff
< Server: cloudflare
< CF-RAY: 89d1dc02dd9b1fe2-IAD
< alt-svc: h3=":443"; ma=86400
< 
#EXTM3U
#EXT-X-TARGETDURATION:4
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:1137127
#EXT-X-DISCONTINUITY-SEQUENCE:0
#EXTINF:4.040,
stream_high_4_1719957087_1137128.ts
#EXTINF:4.040,
stream_high_4_1719957091_1137129.ts
#EXTINF:4.040,
stream_high_4_1719957095_1137130.ts
#EXTINF:4.040,
stream_high_4_1719957099_1137131.ts
#EXTINF:4.040,
stream_high_4_1719957103_1137132.ts
* Closing connection 0
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS alert, close notify (256):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.3 (OUT), TLS alert, close notify (256):
$ curl -v --http1.1 https://station.thecheese.co.nz/hls/the_cheese/stream_high.m3u8
*   Trying 2a06:98c1:3121::7:443...
* Connected to station.thecheese.co.nz (2a06:98c1:3121::7) port 443 (#0)
* ALPN, offering http/1.1
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS header, Finished (20):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.2 (OUT), TLS header, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: CN=thecheese.co.nz
*  start date: Jun 23 08:30:27 2024 GMT
*  expire date: Sep 21 08:30:26 2024 GMT
*  subjectAltName: host "station.thecheese.co.nz" matched cert's "*.thecheese.co.nz"
*  issuer: C=US; O=Google Trust Services; CN=WE1
*  SSL certificate verify ok.
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
> GET /hls/the_cheese/stream_high.m3u8 HTTP/1.1
> Host: station.thecheese.co.nz
> User-Agent: curl/7.81.0
> Accept: */*
> 
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Tue, 02 Jul 2024 21:52:51 GMT
< Content-Type: application/x-mpegurl
< Content-Length: 381
< Connection: keep-alive
< accept-ranges: bytes
< access-control-allow-origin: *
< Cache-Control: no-cache
< etag: "668476b1-17d"
< last-modified: Tue, 02 Jul 2024 21:52:49 GMT
< CF-Cache-Status: DYNAMIC
< Report-To: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v4?s=AlKIaeZk0q%2B8GaUlZrwgyewx80HU6WKwQm%2FkB8LuVfDFxfZQmaD5cFRleOfmrnfKZb7rnztF5nxvNYqbzq6NtxRRUuJqc0kyHvGLMaNFho5UU4V7i78gUeyUgaVG9kPbbcxyRve3LpqLXhSgPvS8n1D71cfpBw%3D%3D"}],"group":"cf-nel","max_age":604800}
< NEL: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
< Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
< X-Content-Type-Options: nosniff
< Server: cloudflare
< CF-RAY: 89d1dd7daa398003-IAD
< alt-svc: h3=":443"; ma=86400
< 
#EXTM3U
#EXT-X-TARGETDURATION:4
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:1137143
#EXT-X-DISCONTINUITY-SEQUENCE:0
#EXTINF:4.040,
stream_high_4_1719957148_1137144.ts
#EXTINF:4.040,
stream_high_4_1719957152_1137145.ts
#EXTINF:4.040,
stream_high_4_1719957156_1137146.ts
#EXTINF:4.040,
stream_high_4_1719957160_1137147.ts
#EXTINF:4.040,
stream_high_4_1719957164_1137148.ts
* Connection #0 to host station.thecheese.co.nz left intact
$ curl -v --http2 https://station.thecheese.co.nz/hls/the_cheese/stream_high.m3u8
*   Trying 2a06:98c1:3120::7:443...
* Connected to station.thecheese.co.nz (2a06:98c1:3120::7) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS header, Finished (20):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.2 (OUT), TLS header, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=thecheese.co.nz
*  start date: Jun 23 08:30:27 2024 GMT
*  expire date: Sep 21 08:30:26 2024 GMT
*  subjectAltName: host "station.thecheese.co.nz" matched cert's "*.thecheese.co.nz"
*  issuer: C=US; O=Google Trust Services; CN=WE1
*  SSL certificate verify ok.
* Using HTTP2, server supports multiplexing
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* Using Stream ID: 1 (easy handle 0x55d096e98c90)
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
> GET /hls/the_cheese/stream_high.m3u8 HTTP/2
> Host: station.thecheese.co.nz
> user-agent: curl/7.81.0
> accept: */*
> 
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
< HTTP/2 200 
< date: Tue, 02 Jul 2024 21:53:23 GMT
< content-type: application/x-mpegurl
< content-length: 381
< accept-ranges: bytes
< access-control-allow-origin: *
< cache-control: no-cache
< etag: "668476d1-17d"
< last-modified: Tue, 02 Jul 2024 21:53:21 GMT
< cf-cache-status: DYNAMIC
< report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v4?s=1XLTPcONNFNeQuFhzb4c%2BS%2B%2FVH8M5cCQPPCFY%2B9ggZUKr9gEM6rI8g4h2u70PrXF7Hlyqk7xMa%2Fdd1DOeNlfVyqJVj2DXuFEQ16IrZzLFvbo2fXY%2FbW14B3fHcnuQGqECWApdqjcZ89rcSmieAtclhIEcL3jQA%3D%3D"}],"group":"cf-nel","max_age":604800}
< nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
< strict-transport-security: max-age=31536000; includeSubDomains; preload
< x-content-type-options: nosniff
< server: cloudflare
< cf-ray: 89d1de466fb1060e-IAD
< alt-svc: h3=":443"; ma=86400
< 
* TLSv1.2 (IN), TLS header, Supplemental data (23):
#EXTM3U
#EXT-X-TARGETDURATION:4
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:1137151
#EXT-X-DISCONTINUITY-SEQUENCE:0
#EXTINF:4.040,
stream_high_4_1719957181_1137152.ts
#EXTINF:4.040,
stream_high_4_1719957185_1137153.ts
#EXTINF:4.040,
stream_high_4_1719957189_1137154.ts
#EXTINF:4.040,
stream_high_4_1719957193_1137155.ts
#EXTINF:4.040,
stream_high_4_1719957197_1137156.ts
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* Connection #0 to host station.thecheese.co.nz left intact
PaulWebster commented 3 months ago

Update from TheCheese - they have heard of our problem and so have removed their block of HTTP/1.0 via Cloudflare. So ... great that the station can now be played but will need other examples of HTTP/1.0 blocks for testing.

Also @Moonbase59 - I think, based on the headers shown, that your http/1.0 test used http/1.1 anyway. Perhaps that version of curl does not honour the command line request or determines that the target system does not say it accepts it so it moves to 1.1

Moonbase59 commented 3 months ago

Yeah, I wondered, because it said HTTP/1.1 200 OK but still closed connection. Who knows…

bpa-code commented 3 months ago

From what I read the discussion about TheCheese, the issue about not supporting HTTP 1.0 seemed to be tied into TLS 1.0 and 1.1 but disabling HTTP 1.0 is collateral damage not the reason itself.

The dev's assumption was that old devices that use HTTP 1.0 probably only support TLS 1.0 and 1.1 and so since TLS 1.2 or higher is recommended for security,it is probably OK to limit connections to HTTP 1.1.

That said, it is probably better that LMS support HTTP 1.1 and chunked encoding and so some investigation is needed. While delving into that area, and if significant changes are needed, then perhaps HTTP/2 (aka SPDY) may need to be considered as part of those changes.

bpa-code commented 3 months ago

For completeness. I have a dev version of PlayHLS which supports fMP4/Flac and uses internal LMS HTTP 1.1 support unlike PlayHLS 2.* .

On LMS 8.5.2 and the dev plugin can play the HLS/Flac stream https://station.thecheese.co.nz/hls/the_cheese/stream_high.m3u8

PaulWebster commented 3 months ago

On LMS 8.5.2 and the dev plugin can play the HLS/Flac stream https://station.thecheese.co.nz/hls/the_cheese/stream_high.m3u8

That URL works for me on LMS 9 with regular PlayHLSv2 - but since they removed the block on http/1.0 I was not surprised. Should I have been? Is it because that PlayHLS uses you own http?

bpa-code commented 3 months ago

LMS does an initial HTTP1.0 GET of the m3u8 file before handling over the PlayHLS based on MIME/suffix returned - if that initial GET is blocked then stream can't play even with PlayHLS own support of HTTP 1.1

PlayHLSV2 was developed for LMS 7.8/7.9 - has its own version of AnyEvent HTTP to support HTTP 1.1 as LMS 7.9 didn't support HTTP 1.1. HTTP 1.1. is used for persistent connection - the load on small device was high with HLS on HTTP 1.0

I think HLS Flac is only supported in fMPEG4. The Cheese "HLS" URL is a MPEG-2 stream and looks like to be AAC.

Input #0, hls, from 'https://station.thecheese.co.nz/hls/the_cheese/stream_high.m3u8':
  Duration: N/A, start: 38244.561911, bitrate: N/A
  Program 0 
    Metadata:
      variant_bitrate : 0
  Stream #0:0: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp
    Metadata:
      variant_bitrate : 0
  Stream #0:1: Data: timed_id3 (ID3  / 0x20334449)
    Metadata:
      variant_bitrate : 0
michaelmurfy commented 3 months ago

Just a note as already mentioned above The Cheese now allows HTTP/1.0 based on feedback and I am monitoring usage here.

If you however wish to conduct testing then you're welcome to do so via a sister station who has a FLAC stream also (@PaulWebster) where I've just deployed the exact same rules that were on The Cheese (also blocking HTTP/1.0) and only allowing TLS 1.2/1.3 - h t t p s : / / station.huttcityfm.co.nz/listen/hutt_city_fm/flac

Note: there is no station data for this (yet) as we're still working on migrating it over to newer infrastructure.

bpa-code commented 3 months ago

Looking further into HTTP/1.1 and chunked encoding.

AFAICT LMS does not use AnyEvent::HTTP - so any handling of chunked data is within LMS routines.

There are two places where HTTP/1.0 request are sent by LMS. (i) Slim/Player/Protocols/HTTP.pm (ii) Slim/Networking/Async/HTTP.pm

I did tests where I changed The LMS "HTTP 1.0" requests into HTTP 1.1 to observe effects. In many cases header "Transfer-Encoding: chunked" was returned.

Slim/Player/Protocols/HTTP.pm is used to proxy request from a players GET when streaming audio. I think the chunked encoding can be handled by adding an appropriate "processor" on detection of the encoding header.

Slim/Networking/Async/HTTP.pm is used for "other" HTTP request (e.g. SimpleAsyncHTTP) - I think the chunked encoding can be handled by updating the _http_read_body and associated routines.

Currently a player playing "direct" uses a HTTP 1.0 request. Is there something that needs to be done ? Given that many stream already require https is it an issue at all ?

LMS proxies players for "https" - should it also be done for HTTP 1.1 ? Is there a need to add another capability to player to indicate HTTP 1.1 native support like https ?

I'll try to update routines to explore "chunked" support.