how to export model resolution and import to custom transform for crowd counting

[yimm0815]

@dahanhan Hi, Dahan,

I tried to get model resolution 128x96 from the model and resize the zonemap bitmask to align with VA pipeline crowd counting output size.

My guess is to change CSRNet_2019R3_model_proc.json file, to add "model_width": 128, and "model_height": 96 inside the "output_postproc" block, then in the crowd_counting.py to call tensor.model_width() and tensor.model_height() to get the resolution.

However, the modification hit an error as below:

{"levelname": "ERROR", "asctime": "2020-01-02 16:32:39,597", "message": "Error on Pipeline 3: element: detection : gst-resource-error-quark: base_inference plugin intitialization failed (2), /home/gst-video-analytics/gstts/base/gva_base_inference.c(438): gva_base_inference_set_caps (): /GstPipeline:pipeline5/GstGvaInference:detection:\n\n[json.exception.parse_error.101] parse error at line 11, column 18: syntax error while parsing objecring literal; expected '}'\n[json.exception.parse_error.101] parse error at line 11, column 18: syntax error while parsing object - unexpected string literal; expected '}'\n[json.exception.parse_error.101] parse error at18: syntax error while parsing object - unexpected string literal; expected '}'\n[json.exception.parse_error.101] parse error at line 11, column 18: syntax error while parsing object - unexpected string literal; expectedtion.parse_error.101] parse error at line 11, column 18: syntax error while parsing object - unexpected string literal; expected '}'\n", "name": "GSTPipeline"} on_created: /tmp/itfQaG8BCrLsWhJDZ3yJ/2020/01/02/1578011559492217514_253003953.mp4 {'id': 3, 'state': 'ERROR', 'avg_fps': 0, 'start_time': 1578011558.9943552, 'elapsed_time': 0.6035614013671875} pineline ended with ERROR exiting va pipeline Exception in connect: VA exited. This should not happen.

so the grammar is not right yet. Do you know where to check the right solution? any thoughts?

thanks, Mo

[yimm0815]

woops, i forget to add a comma before the two new parameters "model_width" and "model_height". plus string is needed too. "layer_name": "y_out/Relu/Transpose", "model_width": "128", "model_height": "96"

now the bug leads to 'TensorMeta' object has no attribute 'model_width'. log as below:

{"levelname": "ERROR", "asctime": "2020-01-02 17:11:01,612", "message": "Error on Pipeline 1: element: crowdcounting : pygi-error: unknown error (0), 'TensorMeta' object has no attribute 'model_width'", "name": "GSTPipeline"} {'id': 1, 'state': 'ERROR', 'avg_fps': 0, 'start_time': 1578013853.0422275, 'elapsed_time': 8.570432901382446} pineline ended with ERROR exiting va pipeline Exception in connect: VA exited. This should not happen.

could you share some thoughts on how to get the model resolution?

[yimm0815]

adding @nnshah1 Neelay too.

guess i cannot define "model_width" and "model_height" by myself. what keywords do gst pipeline support? @nnshah1 could you share some lights here?

CSRNet_2019R3_model_proc.json: { "json_schema_version" : 1.0, "input_preproc": [ { "color_format": "BGR" } ], "output_postproc": [ { "layer_name": "y_out/Relu/Transpose", "model_width": "128", "model_height": "96" } ] }

[dahanhan]

We are using gvainference pipeline, does this element support to use any new word items? @nnshah1

[nnshah1]

If I understand the thread correctly:

1) The model output size is captured in the model IR files (.xml). It can not be changed via the postproc file - the postproc file can only provide details on how to interpret the output layer (labels, etc.). So as long as the model output size is 128 x 96 - that should be the tensor size.

2) The tensor meta object provided to the custom transform code includes a method to return the dimensions of the output tensor (note the first dimension is the batch size, the second is the height, and third is width):

height = tensor.dims()[1] width = tensor.dims()[2]

You can print out the dimensions to make sure it matches what you expect (128 x 96)

[yimm0815]

very cool, @nnshah1 it works just as expected. thanks!

[yimm0815]

code checked in at https://github.com/OpenVisualCloud/Smart-City-Sample/pull/307. close issue