search

NVIDIA-AI-IOT / deepstream-occupancy-analytics

This is a sample application for counting people entering/leaving in a building using NVIDIA Deepstream SDK, Transfer Learning Toolkit (TLT), and pre-trained models. This application can be used to build real-time occupancy analytics applications for smart buildings, hospitals, retail, etc. The application is based on deepstream-test5 sample application.
MIT License
221 stars 73 forks source link

Is source available for custom libnvds_msgconv.so ? #66

Open RockyPhoenix opened 1 year ago

RockyPhoenix commented 1 year ago

The x86 and arm64/Jetson binaries for libnvds_msgconv.so are provided in bin/ and it appears they are essential to convert the object metadata into the form needed for this test application.

Is the source available and if so, can it be included as part of this application?

avighi commented 11 months ago

Is it possible add informations about object who crossed the line? Like this

"object" : { "id" : 9, "type" : "person", "bbox" : [ 296, 441, 183, 425 ]

RockyPhoenix commented 11 months ago

@avighi It's my understanding this is possible. You would need to read the metadata that gst-nvinfer gst-nvtracker are attaching to the frame and then pack into the message that libnvds_msgconv is preparing for the broker.

We created a custom version of nvmsgconv.cpp that implements the NVDS_PAYLOAD_CUSTOM payload type via custom code in dsmeta_payload.cpp

We did a quick modification to one of the generate_dsmeta_message methods to pull our data from frame_usermeta->user_meta_data

I'm not sure if the raw inference output is in frame_meta (NvDsFrameMeta) or if it's in frame_meta->frame_user_meta_list (NvDsAnalyticsFrameMeta)

RockyPhoenix commented 11 months ago

Check out the other generate_dsmeta_message methods to see where they're pulling the data from. For example generate_dsmeta_message_minimal seems to be packing all of the data you're interested in, just with a different schema.