aws / amazon-kinesis-video-streams-parser-library

Amazon Kinesis Video Streams parser library is for developers to include in their applications that makes it easy to work with the output of video streams such as retrieving frame-level objects, metadata for fragments, and more.
Apache License 2.0
103 stars 52 forks source link

Keep consuming stream even if no video available #110

Closed FlorianRuen closed 3 years ago

FlorianRuen commented 4 years ago

Hi there,

I'm using the AWS KVS Parser Library to consume video and to extract each image available on the stream, and save it to a folder for another usage.

But due to network issues, sometime, the stream is not correctly send to the KVS service (the application that send the stream is still trying but it can due to network issue). But when the network issues are resolved, everything go back to normal.

Today my consumer which extract frame works, but if no stream available, it stop, and I need to run it again manually. My question is, there is a way to keep alive on a stream even if no video, so when everything go back to normal, it can automacailly continue the extraction process ?

Maybe there is a timeout on the parser library, that force the thread to stop ? If it's the case, can we remove it safely ?

Thanks for your help,

Kind regards, Florian

MushMal commented 4 years ago

@FlorianRuen what you are referring to is the Notification capability which we currently do not have but it's something we are looking at. Basically, the application can listen to various types of notifications and perform different tasks. Even, to the level of fragment processing or get notified when the new streaming session gets established.

Without notification, in your case you can resort to two options.

1) Listen to CW events for PutMedia and run your workload from there 2) Do polling for the available fragment.

Perhaps, the option 2) is the easiest to implement if you have continuous streaming with some "hiccups" during the streaming.

FlorianRuen commented 4 years ago

@MushMal What I'm using is the GetMediaWorker today. We are ok, that is no video available, it stop automatically ? Because it was happend on my current application

//Start a GetMedia worker to read and process data from the Kinesis Video Stream. val getMediaWorker = GetMediaWorker.create( Regions.EU_CENTRAL_1, awsService.generateCredentials(), streamName, StartSelector().withStartSelectorType(StartSelectorType.NOW), amazonKinesisVideo, getMediaProcessingArguments )

So what you are saying, is to run something like "infinite loop", that check if fragment are available to run the extraction process automatically ?

MushMal commented 4 years ago

Unfortunately, I won't be able to give details as it would be application specific. The suggestion is to perhaps have some heuristics that will understand the reason why the backend GetMedia API returns. The GetMedia API has a default 30 seconds timeout after which the backend will close the connection if no activity is detected. On a timeout, your application could try some sort of progressive back-off logic to certain period - say up to 5 seconds and keep retrying at 5 seconds apart until the producer recovers. This is very end-to-end application specific. Some customers might alarm if the GetMedia doesn't recover within certain time and it would require an external process to kick the communication off again.

disa6302 commented 3 years ago

Closing based on the tag. Feel free to reach out if you have any more questions.