[QUESTION] Struggling with KVS streams on iOS

[djhanove]

A one liner description about the use case and what you are trying to achieve Using HLS streams on a web front end, iphone users are having trouble viewing the KVS livestream. How can I adjust my pipeline configuration to accomodate for these devices? It is only iOS that has problems. Logging 2024-02-27 09:33:58 [547348386176] DEBUG - logStreamInfo(): Kinesis Video Stream Info 2024-02-27 09:33:58 [547348386176] DEBUG - logStreamInfo(): Stream name: sparkcam_192.168.192.77_thermal 2024-02-27 09:33:58 [547348386176] DEBUG - logStreamInfo(): Streaming type: STREAMING_TYPE_REALTIME 2024-02-27 09:33:58 [547348386176] DEBUG - logStreamInfo(): Content type: video/h264 2024-02-27 09:33:58 [547348386176] DEBUG - logStreamInfo(): Max latency (100ns): 600000000 2024-02-27 09:33:58 [547348386176] DEBUG - logStreamInfo(): Fragment duration (100ns): 20000000 2024-02-27 09:33:58 [547348386176] DEBUG - logStreamInfo(): Key frame fragmentation: Yes 2024-02-27 09:33:58 [547348386176] DEBUG - logStreamInfo(): Use frame timecodes: Yes 2024-02-27 09:33:58 [547348386176] DEBUG - logStreamInfo(): Absolute frame timecodes: Yes 2024-02-27 09:33:58 [547348386176] DEBUG - logStreamInfo(): Nal adaptation flags: 0 2024-02-27 09:33:58 [547348386176] DEBUG - logStreamInfo(): Average bandwidth (bps): 4194304 2024-02-27 09:33:58 [547348386176] DEBUG - logStreamInfo(): Framerate: 25 2024-02-27 09:33:58 [547348386176] DEBUG - logStreamInfo(): Buffer duration (100ns): 1200000000 2024-02-27 09:33:58 [547348386176] DEBUG - logStreamInfo(): Replay duration (100ns): 400000000 2024-02-27 09:33:58 [547348386176] DEBUG - logStreamInfo(): Connection Staleness duration (100ns): 600000000 2024-02-27 09:33:58 [547348386176] DEBUG - logStreamInfo(): Store Pressure Policy: 1 2024-02-27 09:33:58 [547348386176] DEBUG - logStreamInfo(): View Overflow Policy: 1 2024-02-27 09:33:58 [547348386176] DEBUG - logStreamInfo(): Allow stream creation: Yes 2024-02-27 09:33:58 [547348386176] DEBUG - logStreamInfo(): Segment UUID: NULL 2024-02-27 09:33:58 [547348386176] DEBUG - logStreamInfo(): Frame ordering mode: 0 2024-02-27 09:33:58 [547348386176] DEBUG - logStreamInfo(): Track list 2024-02-27 09:33:58 [547348386176] DEBUG - logStreamInfo(): Track id: 1 2024-02-27 09:33:58 [547348386176] DEBUG - logStreamInfo(): Track name: kinesis_video 2024-02-27 09:33:58 [547348386176] DEBUG - logStreamInfo(): Codec id: V_MPEG4/ISO/AVC 2024-02-27 09:33:58 [547348386176] DEBUG - logStreamInfo(): Track type: TRACK_INFO_TYPE_VIDEO 2024-02-27 09:33:58 [547348386176] DEBUG - logStreamInfo(): Track cpd: NULL

Any design considerations/constraints gstreamer configuration

        appsrc name={self.cam_type} ! video/x-raw, format=BGR, width=640, height=480, framerate={self.frame_rate}/1 ! 
        videoconvert ! x264enc bitrate=1000 speed-preset=ultrafast tune=zerolatency bframes=0 key-int-max=45 vbv-buf-capacity=0 ! 
        video/x-h264,profile=baseline,stream-format=avc,alignment=au ! h264parse ! 
        kvssink stream-name="{self.stream_name}" access-key="{self.access_key}" 
        secret-key="{self.secret_key}" aws-region="{self.region}" storage-size=128

js configuration:

import Hls from "hls.js";
import React, { useEffect, useRef } from "react";

type VideoPlayerProps = {
  streamUrl: string;
};

export const VideoPlayer: React.FC<VideoPlayerProps> = ({ streamUrl }) => {
  const videoRef = useRef<HTMLVideoElement>(null);
  const sourceRef = useRef<HTMLSourceElement>(null);

  useEffect(() => {
    if (Hls.isSupported()) {
      var hls = new Hls();
      hls.loadSource(streamUrl);
      hls.attachMedia(videoRef.current);
    } else if (videoRef.current.canPlayType("application/vnd.apple.mpegurl")) {
      sourceRef.current.type = "application/x-mpegURL";
      videoRef.current.src = streamUrl;
      videoRef.current.setAttribute("playsinline", "true");
    }
  }, [streamUrl]);

  return (
    <video ref={videoRef} controls autoPlay className="h-full w-full">
      <source ref={sourceRef} />
    </video>
  );
};

Are there any obvious issues here that I am missing? Thanks!!

[djhanove]

A workaround I found was to increase the number of frames between keyframes. It seems iOS likes a longer fragment length Can modify with the key-int-max param in your pipeline

[sirknightj]

HI @djhanove, thanks for sharing. Increasing the number of frames between keyframes will increase the fragment duration.

See the documentation for more information and troubleshooting tips: https://docs.aws.amazon.com/kinesisvideostreams/latest/dg/troubleshooting.html#troubleshooting-hls

[stefankiesz]

Hi @djhanove, can you please provide some more details as to how HLS playback is failing (are there any errors, is the HLS URL fetching successful, video player logs if possible), and what is the GetHLSStreamingSessionURL command properties are for the streams?

[djhanove]

Hi there, The HLS URL is fetched successfully and playback tries to start but then results in the following black image.

I am using the baseline h-264 profile which is read correctly in the kinesis media player. No issues on android or web with the same pipeline.

The GetHLSStreamingSessionURL is configured below:

export async function GET(request: NextRequest, { params }: { params: { device_id: string } }) {
  const deviceId = params.device_id;
  const { searchParams } = new URL(request.url);
  const livestreamMode = searchParams.get("mode");
  const streamName = `xxxxxxx_${deviceId}_${livestreamMode === "thermal" ? "thermal" : "rgb"}`;

  try {
    const result = await new Promise((resolve, reject) => {
      kvClient.getDataEndpoint(
        {
          StreamName: streamName,
          APIName: "GET_HLS_STREAMING_SESSION_URL",
        },
        (err, data) => {
          if (err) {
            reject(err);
          }

          const kvArchived = new AWS.KinesisVideoArchivedMedia({
            endpoint: data.DataEndpoint,
          });

          kvArchived.getHLSStreamingSessionURL(
            {
              StreamName: streamName,
              PlaybackMode: "LIVE",
            },
            (err, data) => {
              if (err || !data?.HLSStreamingSessionURL) {
                reject(err);
              } else {
                resolve(data.HLSStreamingSessionURL);
              }
            },
          );
        },
      );
    });

    return NextResponse.json({ url: result });
  } catch (e) {
    console.log("ERROR SERVER", e);
    return NextResponse.json({ error: e.message }, { status: 400 });
  }
}

[stefankiesz]

Can you please include what you were able to try from the above linked trouble shooting guide, and do you have any success using DASH instead of HLS on ios?

[niyatim23]

Closing due to no response