Proxy Server Unhandled Exception (Null check operator used on a null value)

[mt633]

Which API doesn't behave as documented, and how does it misbehave? Line 2002 in just_audio.dart (I was trying out the treadmill branch in which it is line 1951 in the error log below, which by the way seems to work great to me). https://github.com/ryanheise/just_audio/blob/8ad81998a2086ac0a8638f00fb7a1f7c7392bf0b/just_audio/lib/just_audio.dart#L2002

This can apparently sometimes be null so a null check should be used before calling the handle method.

To Reproduce

1. Change the example audio source to HlsAudioSource (or use a HLS source)
2. Add a header

For instance, update the example with the following code:

 await _player.setAudioSource(HlsAudioSource(Uri.parse(
          "https://bitdash-a.akamaihd.net/content/MI201109210084_1/m3u8s/f08e80da-bf1d-4e3d-8899-f0f6155f6efa.m3u8"), headers: {'Accept-Encoding': 'identity'}));

Error messages

Unhandled Exception: Null check operator used on a null value
#0      _ProxyHttpServer.start.<anonymous closure>
package:just_audio/just_audio.dart:1951
#1      _RootZone.runUnaryGuarded (dart:async/zone.dart:1586:10)
#2      _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:339:11)
#3      _BufferingStreamSubscription._add (dart:async/stream_impl.dart:271:7)
#4      _SyncStreamControllerDispatch._sendData (dart:async/stream_controller.dart:774:19)
#5      _StreamController._add (dart:async/stream_controller.dart:648:7)
#6      _StreamController.add (dart:async/stream_controller.dart:596:5)
#7      _HttpServer._handleRequest (dart:_http/http_impl.dart:3303:19)
#8      new _HttpConnection.<anonymous closure> (dart:_http/http_impl.dart:3053:19)
#9      _RootZone.runUnaryGuarded (dart:async/zone.dart:1586:10)
#10     _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:339:11)
#11     _BufferingStreamSubscription._add (dart:asy<…>

Expected behavior Null exceptions should be handled/avoided properly.

Additional information The headers only partially work with the new way to send headers (see https://github.com/ryanheise/just_audio/issues/858#issuecomment-1297309817). It does not work fully with 'Accept-Encoding': 'identity' (causing a problem for us when the server is auto-compressing the response, see https://github.com/ryanheise/just_audio/issues/470#issuecomment-1187323425), but the other headers seem to work.

[github-actions[bot]]

Oops, it appears that your issue did not follow the template and is missing one or more required sections. Please open a new issue, and provide all required sections and information.

FAQ:

1. Do I really need to submit a minimal reproduction project for a bug? A: Yes. I prioritise bugs secondarily on how many people are affected, and primarily on whether the bug report is complete, in the sense that it enables me to immediately reproduce it and start working on a fix. If a bug is important to you, the best thing you can do is to provide all requested information ASAP so that I can start looking into it ASAP.

2. I think I supplied all required information, so did the bot make a mistake? A: The bot only checks the section headings, so when you post a new issue, make sure you leave the section headings intact. (Note that because of this, it is even possible to trick the bot by including only the section headings, and then not providing the requested information under each heading. This is frowned upon, and the issue will be closed manually.)

[ryanheise]

I have reopened and will try to investigate the stack trace but I will still eventually need a way to reproduce it to verify whether the fix is working.

Thanks for trying out this branch.

[mt633]

Thanks, I'll let you know if I manage to find a way to reproduce it.

[mt633]

I also managed to get this log from just_audio if I use try catch on load/play:

(-1008) resource unavailable
#0      AudioPlayer._load (package:just_audio/just_audio.dart:800:9)
<asynchronous suspension>
#1      AudioPlayer._setPlatformActive.setPlatform (package:just_audio/just_audio.dart:1370:28)
<asynchronous suspension>

I was trying to play a new HLS item. If you need me to test anything, I've right now stumbled upon the same error again. I think I can change to another branch through hot reload, so I can keep it running for now before I restart the app.

[mt633]

After some further digging, this seems to be an issue caused by the headers refactoring commit https://github.com/ryanheise/just_audio/commit/27ab201c18d75f299643615661f87e5ebd34b459. If I use HLS without headers it works, but there seems to be some kind of issue with the new code when it comes to headers with HlsAudioSource. I'll update the issue with steps to reproduce it.

[Mikhail-Ivanou]

Got the same issue everything works with direct stream URL and I get

I/flutter ( 8644): Null check operator used on a null value
I/flutter ( 8644): #0      _ProxyHttpServer.start.<anonymous closure> (package:just_audio/just_audio.dart:2002:45)

if set base url with auth header

I see correct redirect value if (originResponse.redirects.isNotEmpty) { redirectedUri = originResponse.redirects.last.location; } in _ProxyHandler _proxyHandlerForUri()

One more update redirected url is https://5eb15a56ce2c9.streamsource.net/Products/mp4:track-1-0.m4a/playlist.m3u8?tstarttime=1&tendtime=1&tp1=val&thash=X= (and it works if I use it directly)

and than it tries to find handler for /api/v4/chunklist_w771808981_tkdH.....R0k9.m3u8?

[ryanheise]

After some further digging, this seems to be an issue caused by the headers refactoring commit 27ab201. If I use HLS without headers it works, but there seems to be some kind of issue with the new code when it comes to headers with HlsAudioSource. I'll update the issue with steps to reproduce it.

Are you saying that this worked before that commit and broke after that commit?

[mt633]

Exactly, it did work before that commit. Not entirely sure if there are several different issues here or not, but at least one issue was caused by that commit.

[os5633]

The reason for the null error is that the request with the header clears the line with audio.m3u8 as a regular expression.

        for (var line in const LineSplitter().convert(m3u8)) {

          //this line
          line = line.replaceAll(RegExp(r'#.*$'), '').trim();

          if (line.isEmpty) continue;
          try {
            final rawNestedUri = Uri.parse(line);
            if (rawNestedUri.hasScheme) {
              // Don't propagate headers
              server.addUriAudioSource(AudioSource.uri(rawNestedUri));
            } else {
              // This is a resource on the same server, so propagate the headers.
              final basePath = rawNestedUri.path.startsWith('/')
                  ? ''
                  : uri.path.replaceAll(RegExp(r'/[^/]*$'), '/');
              final nestedUri =
                  uri.replace(path: '$basePath${rawNestedUri.path}');
              server.addUriAudioSource(
                  AudioSource.uri(nestedUri, headers: headers));
            }
          } catch (e) {
            // ignore malformed lines
          }
        }```

[ryanheise]

Thanks for the lead, @os5633 .

Try adding this as the first line inside the loop, and leaving all else the same:

line = line.replaceAllMapped(RegExp(r'#EXT-X-MEDIA:.*?URI="(.*?)".*'), (m) => m[1]!);

I can publish a fix if that works.

[ryanheise]

I've implemented this fix on the branch fix/hls_nested_uri. Please let me know how it goes, and then I can merge it for release and also into treadmill.

[os5633]

Null error completely disappeared Thank you :) However, this error is still not resolved. If you set up a proxy in Ubuntu and connect the vpn, it will be received normally, but if you turn it off, the error The network connection was lost. will occur based on ios.

[ryanheise]

Would you be able to create a separate bug report for that? I think the null check issue at least can be closed.

[os5633]

I solved this error. I erased the part that set the header every time I made a request.

// this line
 request.response.headers.clear();
 originResponse.headers.forEach((name, value) {
     final filteredValue = value
            .map((e) => e.replaceAll(RegExp(r'[^\x09\x20-\x7F]'), '?'))
            .toList();
      request.response.headers.set(name, filteredValue);
  });

[ryanheise]

I'm not sure if that is the correct solution, so I'd like to understand which specific headers need to be inserted. Hence, if you are able to create a bug report by listing the steps to reproduce, I'll be able to look at it further.

[os5633]

What's certain is that the header didn't go in as I intended. I'll write a report that can reproduce the this problem.

[ryanheise]

I have released my fix from earlier and also merged it into treadmill.