search

TNTWEN / OpenVINO-YOLO-Automatic-Generation

MIT License
25 stars 1 forks source link

yolov4x-mish config file support? #4

Open andreim78 opened 2 years ago

andreim78 commented 2 years ago

Hi @TNTWEN

Are there any plans for supporting yolov4x-mish cfg files? When I try to use the config parser with a default or custom yolov4x-mish.cfg file, I get the following error:

Traceback (most recent call last): File "./yolov4parse/parse_config.py", line 137, in <module> mdefs=parse_model_cfg(path) File "./yolov4parse/parse_config.py", line 49, in parse_model_cfg assert not any(u), "Unsupported fields %s in %s." % (u, path) AssertionError: Unsupported fields ['stopbackward', 'objectness_smooth', 'resize', 'obj_normalizer', 'new_coords', 'max_delta'] in ....

I've tried parse_config.py in your OpenVINO-YOLOV4 repo under yolov4x-mish folder and get this error:

Traceback (most recent call last): File "./yolov4x-mish/parse_config.py", line 127, in <module> mdefs=parse_model_cfg(path) File "./yolov4x-mish/parse_config.py", line 49, in parse_model_cfg assert not any(u), "Unsupported fields %s in %s." % (u, path) AssertionError: Unsupported fields ['stopbackward'] in ....

Both these tests were done inside an OpenVINO 2021.3 Ubuntu 18.04 Official Docker Image with Python 3.6.9 and Tensorflow 1.15.5.

Thanks, Andrei

TNTWEN commented 2 years ago

you could use parse_config.py here: https://github.com/TNTWEN/OpenVINO-YOLOV4/tree/ScaledYOLOv4

andreim78 commented 2 years ago

Thank you for a lightning fast response!

I've previously used the parse_config.pyinside the yolov4x-mish folder of that repo and still got the error, the one in the root worked.

However there's no ************************************ divider line as you had in the README of the https://github.com/TNTWEN/OpenVINO-YOLO-Automatic-Generation, should I just paste the result of the parser in yolov4x-mish/yolo_v4.py?

Thanks, Andrei

TNTWEN commented 2 years ago

yeah ,just paste the result and also remember to set --output when using mo.py

andreim78 commented 2 years ago

Hi @TNTWEN

Thank you for your help and your quick response.

I've updated the parse_config.py to automatically generate the yolo_v4.py, including updating anchors if different anchors are used. I've also preserved the original logic of outputing to the console as well, just added and anchors string at the bottom of output.

Would you like me to contribute these changes?

Regards, Andrei

TNTWEN commented 2 years ago

@andreim78 Recently I have focused on research in other directions. So failed to maintain the repo in time. Feel free to fork and create Pull requests .And thanks for your contributions in advance!

andreim78 commented 2 years ago

@TNTWEN, sure, I'll fork, add my changes and create pull request.

However, I have one more issue that I got now running the mo.py - I've generated a new yolo_v4.py sections for my custom config using the parse_config.py, then did the convert_weights_pb.py with relevant class names, weights file, size, etc. The .pb file is generated successfully. When I try to run the optimization via mo.py, I get the following error:

`root@9395a971271b:/app# mo.py --input_model ./yolov4x-mish.pb --batch 1 --reverse_input_channels --transformations_config ./yolov4x-mish.json --output detector/yolo-v4/Conv_112/BiasAdd,detector/yolo-v4/Conv_124/BiasAdd,detector/yolo-v4/Conv_136/BiasAdd Model Optimizer arguments: Common parameters:

The json file for transofrmations configs is below: [{"id": "TFYOLOV4", "match_kind": "general", "custom_attributes": {"classes": 3, "anchors": [57, 79, 104, 74, 83, 109, 68, 160, 135, 112, 105, 150, 108, 231, 148, 183, 149, 305], "coords": 4, "num": 9, "masks": [[0, 1, 2], [3, 4, 5], [6, 7, 8]], "entry_points": ["detector/yolo-v4/Reshape", "detector/yolo-v4/Reshape_4", "detector/yolo-v4/Reshape_8"]}}]

I've tried running mo.py without specifying the transformations config file and get the same error.

Then I tried the same with the yolov4x-mish.weights for the COCO dataset with the yolo_v4.py file as it came with your repo. Once again the .pb file is generated successfully, however I get a similar error when running optimization:

`root@9c433df2f159:/app# mo.py --input_model ./yolov4x-mish.pb --batch 1 --reverse_input_channels --output detector/yolo-v4/Conv_112/BiasAdd,detector/yolo-v4/Conv_124/BiasAdd,detector/yolo-v4/Conv_136/BiasAdd Model Optimizer arguments: Common parameters:

I've tried changing the id in the JSON file to TFYOLOV3 as per #8 here, however I got the following error:

[ ERROR ] Exception occurred during running replacer "TFYOLOV3" (<class 'extensions.front.YOLO.YoloV3RegionAddon'>): TensorFlow YOLO V3 conversion mechanism was enabled. Entry points "detector/yolo-v4/Reshape, detector/yolo-v4/Reshape_4, detector/yolo-v4/Reshape_8" were provided in the configuration file. Entry points are nodes that feed YOLO Region layers. Node with name detector/yolo-v4/Reshape doesn't exist in the graph. Refer to documentation about converting YOLO models for more information.

Can you, please, assist?

Thanks, Andrei

TNTWEN commented 2 years ago

for scaled-yolov4 ,you don't need to use .json. Only need to set right --output when using mo.py. And also you should use https://github.com/TNTWEN/OpenVINO-YOLOV4/blob/ScaledYOLOv4/yolov4-inference-demo.py for inference, which is different from yolov4 and yolov4-tiny.

And anchors seems to be set in https://github.com/TNTWEN/OpenVINO-YOLOV4/blob/ScaledYOLOv4/yolov4-inference-demo.py too.

andreim78 commented 2 years ago

Thanks for your response, @TNTWEN

I've tried running the mo.py without the JSON file and I get the same error: [ ERROR ] Exception occurred during running replacer "fusing" (<class 'extensions.middle.fusings.Fusing'>): After partial shape inference were found shape collision for node detector/yolo-v4/Conv/BatchNorm/FusedBatchNormV3/beta (old shape: [ 1 32 640 640], new shape: [ 1 32 640 -1])

I've been looking through the code and cannot easily pick up where the issue is.

I'm going to try and create my own OpenVINO docker image and see if there's something to the official image that causes this. I'll keep you posted, however if you could have a look as well, I'd really appreciate it.

Thanks, Andrei

TNTWEN commented 2 years ago

https://github.com/TNTWEN/OpenVINO-YOLOV4/blob/2f7edc4424483f58b488fa6c59b8fdb5f40f1211/yolov4x-mish/convert_weights_pb.py#L22 maybe the size error?

you could also try to test whether https://github.com/TNTWEN/OpenVINO-YOLOV4/tree/ScaledYOLOv4 could run correctly

andreim78 commented 2 years ago

I also thought it might be the size parameter, but the old shape has the size populated correctly: old shape: [ 1 32 640 640], it's the new shape that has incorrect height of -1, the width is correct. I've also played around with other yolov4x-mish config files that I have with different sizes, and I get the same:

old shape: [ 1 32 416 416], new shape: [ 1 32 416 -1]) or old shape: [ 1 32 800 800], new shape: [ 1 32 800 -1]) for --size parameter of 416 or 800 respectively.

I'm now rebuilding the OpenVINO docker image from scratch rather than using the official one - perhaps there's a version mismatch.

andreim78 commented 2 years ago

Hi @TNTWEN

I've tried rebuilding the docker image from scratch with Ubuntu 18.04, OpenVINO 2021.3.394 and Tensorflow 1.15.5 - the versions that I've seen used by you. Unfortunately I'm still getting the same error when running model optimization.

I've tried using your original code with a coco pretrained model for yolov4x-mish, found here, unfortunately I get the same error during the optimization: [ ERROR ] Exception occurred during running replacer "fusing" (<class 'extensions.middle.fusings.Fusing'>): After partial shape inference were found shape collision for node detector/yolo-v4/Conv/BatchNorm/FusedBatchNormV3/beta (old shape: [ 1 32 416 416], new shape: [ 1 32 416 -1])

In this test I haven't modified any of your code in the repo - original anchors, model definitions, etc. were used with the default yolov4x-mish.cfg files.

After I did a bit of searching, I stumbled across this issue for OpenVINO toolkit: #8617, which is still unresolved. I've also tried providing the --input_shape parameter to the model optimizer instead of the --batch 1, but I get the same error.

Seeing that our environments are practically the same, I'm not sure what else I can try as I've moved from our custom models and am working with the coco models and vanilla configs, however still getting the same errors and unable to get passed the mo.py step. Are you able to share the yolov4x-mish.weights and yolov4x-mish.cfg that you've used in your tests? So that I can try these on my side to get to a working state and then retrace to my custom model.

TNTWEN commented 2 years ago

download from https://github.com/AlexeyAB/darknet