Closed mattfeury closed 2 years ago
note that if i run without opencv (and consequently without the "tee" split to appsink), and manually trigger wifi on and off, the stream does not gracefully recover:
gst-launch-1.0 rtspsrc location="rtsp://IP_ADDRESS:554/axis-media/media.amp?videocodec=h264&fps=5" short-header=TRUE ! rtph264depay ! h264parse ! kvssink stream-name="stream-test-name" storage-size=512 replay-duration =0 access-key="KEY" secret-key="SECRET" aws-region="us-east-1"
works fine for some time, then i trigger wifi on/off to simulate a network hiccup and get:
2021-12-08 19:38:29 [140013148813056] DEBUG - postReadCallback(): Wrote 84 bytes to Kinesis Video. Upload stream handle: 0
2021-12-08 19:38:50 [140013148813056] DEBUG - postReadCallback(): Pausing CURL read for upload handle: 0
2021-12-08 19:38:51 [140013148813056] INFO - stopKinesisVideoStreamSync(): Synchronously stopping Kinesis Video Stream 000055baf2700d18.
2021-12-08 19:38:56 [140013140420352] INFO - getStreamData(): Handle 0 waiting for last persisted ack with ts 16389887082840000
2021-12-08 19:38:56 [140013148813056] DEBUG - postReadCallback(): Pausing CURL read for upload handle: 0 waiting for last ack.
2021-12-08 19:38:56 [140013148813056] WARN - curlCompleteSync(): curl perform failed for url https://s-468461da.kinesisvideo.us-east-1.amazonaws.com/putMedia with result Timeout was reached: Operation too slow. Less than 30 bytes/sec transferred the last 30 seconds
2021-12-08 19:39:05 [140013148813056] WARN - curlCompleteSync(): HTTP Error 0 : Response: (null)
Request URL: https://s-468461da.kinesisvideo.us-east-1.amazonaws.com/putMedia
Request Headers:
Authorization: AWS4-HMAC-SHA256 Credential=ABCD/20211208/us-east-1/kinesisvideo/aws4_request, SignedHeaders=connection;host;transfer-encoding;user-agent;x-amz-date;x-amzn-fragment-acknowledgment-required;x-amzn-fragment-timecode-type;x-amzn-producer-start-timestamp;x-amzn-stream-name, Signature=e698423ffcf87dcbef7a7541609e4909c6c1e01345015ed
2021-12-08 19:39:05 [140013148813056] DEBUG - putStreamCurlHandler(): Network thread for Kinesis Video stream: matt-rtcc-stream-test with upload handle: 0 exited. http status: 0
2021-12-08 19:39:05 [140013148813056] WARN - putStreamCurlHandler(): Stream with streamHandle 94261419707672 uploadHandle 0 has exited without triggering end-of-stream. Service call result: 599
2021-12-08 19:39:05 [140013148813056] INFO - kinesisVideoStreamTerminated(): Stream 0x55baf2700d18 terminated upload handle 0 with service call result 599.
2021-12-08 19:39:05 [140013148813056] DEBUG - stepStateMachine(): State Machine - Current state: 0x0000000000000100, Next state: 0x0000000000000200
2021-12-08 19:39:05 [140013148813056] WARNING: from element /GstPipeline:pipeline0/GstRTSPSrc:rtspsrc0: Could not read from resource.
Additional debug info:
gstrtspsrc.c(5543): gst_rtspsrc_loop_interleaved (): /GstPipeline:pipeline0/GstRTSPSrc:rtspsrc0:
The server closed the connection.
i would expect the pipeline to recover. is this not the case? i'm not sure if these are two different things i'm describing, but my overall goal is to create a pipeline that will withstand poor connections and automatically reconnect as it's able.
note that i tried the addition of the following flags here to no avail: framerate=5 connection-staleness=500 replay-duration=0 restart-on-error=true
Hi @mattfeury, the pipeline does not recover because it receives an EOS which is not treated as an error. It closes the stream and then later it errors out with 0x52000052
which is STATUS_STREAM_HAS_BEEN_STOPPED
. kvssink is behaving as intended as per the logs shared
Closing assuming answered, feel free to reopen / ask further questions if needed
understood. we are now detecting EOS and properly restarting the pipeline as a whole. this was confusing for us because appsink pipelines continue to work even after receiving an EOS, so i wonder if there is opportunity to more gracefully recover to align with how other sinks function. but fine to keep this closed
Hello! I've got an RTSP stream that I am pumping to KVS sink via a Gstreamer pipeline (this is done via OpenCV videoWriter so it can simultaneously pump to an appsink.) Command is:
This works for "some amount of time", but due to what i'm assuming is just bandwidth hiccups, at times, the stream will explode with the logging below.
Logging
Any design considerations/constraints I have the kvssink built into this image. Though it's been instantiated via opencv, the gstreamer pipeline should be pretty bare bones. Though I don't have
restart-on-error
explicitly defined, it seems to be defaulted to true, which i expected it should gracefully restart.Note that the app-sink in this case continues to work even after the kvs-sink throws the above error and stops working, so it seems like that portion of the pipeline properly recovers.
I know this is likely due to bandwidth/network issues, so i'm less concerned with how do i make sure it doesn't happen and moreso how can i make sure i detect this issue / gracefully recover.