Closed jeff-wursta closed 1 year ago
Hey!
The output stream that you're trying to read from is actually the analytics results and not a video. To get those out, you can try
# This will print packets from a stream to stdout.
# This will work for *any* stream, independent of the data type.
vaictl -p PROJECT_ID \
-l LOCATION_ID \
-c application-cluster-0 \
--service-endpoint visionai.googleapis.com \
receive streams packets STREAM_ID
c.f. https://cloud.google.com/vision-ai/docs/read-stream#read-outputs
Let me know how that goes.
The output stream that you're trying to read from is actually the analytics results and not a video.
I'm interpreting this as "the output stream we want to read from should be the analytics results, which we can get from the receive streams packets
command", is that correct?
If so, why is the receive streams video-file
crashing? And is there a way to ingest the processed video output?
Edit: to answer your question the receive streams packets
command does work like a charm, and allows us to get all of the captured metadata for the stream. However, it would be nice to get a visual representation of the output, or at least an easy way to get the individual segments the metadata refers to.
Edit 2: Maybe there isn't a way to get processed video output from the model (eg video w/ bounding boxes), and instead we may need to manufacture those manually?
I'm interpreting this as "the output stream we want to read from should be the analytics results, which we can get from the receive streams packets command", is that correct?
Yep!
If so, why is the receive streams video-file crashing?
This needs a better error message... what it really should've said is that you were attempting to save a video from a non video stream, rather than burying that in the log printout (you actually see this in your trace). Error messages can definitely be improved.
And is there a way to ingest the processed video output?
Currently there isn't, because the application doesn't contain a "merger" node that overlays the results. In practice, you can in fact merge/overlay these results on your own, but admittedly, this can be somewhat of a tedious process. We will have a visualization tool published around May/June, which will do some of this for you.
What kind of use case are you currently thinking about in your application? Any features that you would like to see that could help with that?
A use case we are looking into is real-time alerting when certain events/actions within a stream are identified, including the video segment or screenshot (bonus if it included bounding boxes already) of when the action took place. Since Vision AI already has the stream info, along with any and all metadata during the stream, it would be really cool if we could enable some "Enable Processed Video" feature where we could pull down processed video segments that met certain criteria, including any bounding boxes already overlayed.
As a side note, I think the Google documentation could really use some updating as well, since it even specifically calls out using the application-output-*
(non-video stream) stream for saving video segments.
Thanks a lot for clarifying though, this was very insightful.
I will let you close out the ticket in case you have any other comments or questions.
Thanks so much for the feedback!
Will report back here if there is any new developments on this front.
We are currently sending a looping MP4 video up to a Vertex Vision AI stream and it is streaming properly. We deployed the Application and enabled the
Enable output streaming
feature so that we could ingest the processed stream and analyze the results.However, when ingesting the stream using the following:
we run into the following error:
May be worth noting that we are running both
vaictl
commands in Docker containers for testing.Any idea(s) on what is going on here? Thanks in advance!