object detection demo app not working with image size other than 640

[apanand14]

Hello,

I would like to use YOLOv5 trained model on image size other than 640. For example, (480,480) or (240,240). And I change model input image size in (PrePostprocessor.java)the below code. But produce some error and not able to run for the lower input size. One more question: Do I nees to change "private static int mOutputRow = 25200" for another input image??? and from where this number 25200 came for image 640*640?? Thank you for your answer in advance.

public class PrePostProcessor { // for yolov5 model, no need to apply MEAN and STD static float[] NO_MEAN_RGB = new float[] {0.0f, 0.0f, 0.0f}; static float[] NO_STD_RGB = new float[] {1.0f, 1.0f, 1.0f};

// model input image size
**static int mInputWidth = 480;
static int mInputHeight = 480;**

// model output is of size 25200*(num_of_class+5)
private static int mOutputRow = 25200; // as decided by the YOLOv5 model for input image of size 640*640
private static int mOutputColumn = 7; // left, top, right, bottom, score and 80 class probability
private static float mThreshold = 0.30f; // score above which a detection is generated
private static int mNmsLimit = 15;

static String[] mClasses;

Error for input image size 480::

W/native: [W TensorImpl.h:1408] Warning: Named tensors and all their associated APIs are an experimental feature and subject to change. Please do not use them for anything important until they are released as stable. (function operator()) **E/AndroidRuntime: FATAL EXCEPTION: Thread-2 Process: org.pytorch.demo.objectdetection, PID: 16491 com.facebook.jni.CppException: The size of tensor a (60) must match the size of tensor b (80) at non-singleton dimension 3

  Debug info for handle(s): -1, was not found.

Exception raised from infer_size_impl at ../aten/src/ATen/ExpandUtils.cpp:28 (most recent call first):
(no backtrace available)
    at org.pytorch.LiteNativePeer.forward(Native Method)
    at org.pytorch.Module.forward(Module.java:52)
    at org.pytorch.demo.objectdetection.MainActivity.run(MainActivity.java:242)
    at java.lang.Thread.run(Thread.java:923)**

I/Process: Sending signal. PID: 16491 SIG: 9

[Piplebobble]

Did you solve your problem? When I change the input image to 416*416 I also have an ERROR, do you know how to solve it? “ E/AndroidRuntime: FATAL EXCEPTION: Thread-3 Process: org.pytorch.demo.objectdetection, PID: 22733 java.lang.RuntimeException: shape '[1, 2, 8, 160, 160]' is invalid for input of size 173056 at org.pytorch.LiteNativePeer.forward(Native Method) at org.pytorch.Module.forward(Module.java:52) at org.pytorch.demo.objectdetection.MainActivity.run(MainActivity.java:244) at java.lang.Thread.run(Thread.java:919) ”

[apanand14]

hello, No, I didn't solve yet. If you find anything then please let me know. Thank you!!

[masc-it]

for my yolov5n I've set mOutputRow=6300. with input image=320 it works just fine. I have found this number analysing my model using netron.ai

[atultiwari]

for my yolov5n I've set mOutputRow=6300. with input image=320 it works just fine. I have found this number analysing my model using netron.ai

Sir, how are you able to decide mOutputRow from Netron. I am stuck since days in customizing the app for 5 classes, input image size 256. I am able to open the model Netron. But I don't know how to interpret it. Can you please help.

[masc-it]

I found the formula I used to find the mOutputRow:

ratio=640/new_shape

25200 : ratio*x = 640 : new_shape

x = (25200 * new_shape / 640) / ratio

EXAMPLES

new_shape = 320
ratio=2
x = (25200 * 320 / 640) / 2 = 6300

new_shape = 1280
ratio=640/1280=0.5
x = (25200 * 1280 / 640) / 0.5 = 100800

Of course take this formula with a grain of salt, it might be just wrong (even if it works).

[atultiwari]

I found the formula I used to find the mOutputRow:

ratio=640/new_shape

25200 : ratio*x = 640 : new_shape

x = (25200 * new_shape / 640) / ratio

EXAMPLES

new_shape = 320
ratio=2
x = (25200 * 320 / 640) / 2 = 6300

new_shape = 1280
ratio=640/1280=0.5
x = (25200 * 1280 / 640) / 0.5 = 100800

Of course take this formula with a grain of salt, it might be just wrong (even if it works).

I tried my best, with image size 320 and mOutputRow=6300, but the image crashed with error message like mentioned above.

This error only occurs when we change mOutputRow. I am still not able to figure out what's the criteria to set this value.

[atultiwari]

I followed the following comment -

https://github.com/pytorch/android-demo-app/issues/102#issuecomment-789953343

and updated my export.py as mentioned.

As you can see in the attached screenshot, when i try to print the shape of a is - torch.Size([1, 25200, 10])

From my understanding from the mentioned comment is that the 2nd number decides mOutputRow (25200) and 3rd number decides mOutputColumn. (Please correct me if I interpreted it wrong)

I don't know what does first number decides from this shape (value of which is 1 in my case, while it was 16 in the mentioned comment). If you help me understand it, that would be a great help.

[MetinDereli]

you need a different model for the android app because you have set your model to imgsize 640

change the imgsize when you start export.py if you don't specify the imgsize, it sets the size to 640.

example: python export.py --weights best.pt --img 1280

then you see the value what you need. paste this document "torchscript.ptl" you created into the android app and change the values

[pdeubel]

For future reference: I had a similar issue because I changed the image size and used a custom dataset. If the previous answer does not work for you try the following which worked for me.

Change export.py as described in the Readme of this project (use fl = file.with_suffix('.torchscript.ptl' etc.), then add the YAML configuration for your custom dataset and the custom image size when exporting the model:

python export.py --weights best.pt --include torchscript --optimize --data PATH_TO_DATASET.yaml --imgsz CUSTOM_HEIGHT CUSTOM_WIDTH

Then of course change the following values in your android app: mInputWidth and mInputHeight to your custom image size, mOutputColumn to the number of classes in your custom dataset plus five and mOutputRow to whichever dimensions your model outputs. An easy way to see the output dimensions is described in https://github.com/pytorch/android-demo-app/issues/102#issuecomment-789953343 but the exact syntax does not work anymore. This should do it:

Search for y = model(im) in export.py, then add the following in the line below: print(f"Model output shape {y[0].shape}"). This then prints the output shape, where the first dimension is the batch size, the second dimension is the correct value for mOutputRow and the third dimension is the correct value for mOutputColumn.

In the last step you have to add the class names to the android app. To do this you need to create a .txt file in which the class names are listed, one per row. The order must be the same as in the dataset YAML file so that the class ids can be mapped to the labels. Then add the .txt to app/src/main/assets (same place where the model file is located) and change in MainActivity.java:

BufferedReader br = new BufferedReader(new InputStreamReader(getAssets().open("YOUR_FILE.txt")));

Then the class names of your custom dataset will be loaded and displayed.