No tensor arguments when running Batch Image Detection on own photos and output image color mismatch.

[JaimyvS]

First of all, thanks for building the MegaDetector and keeping it free to use for everyone.

In the past, I've successfully used the previous version of the MegaDetector but decided to install the new PyTorch Wildlife package.

I've tested the demo python scripts which all seem to work, but when I try to run the pw_data.DetectionImageFolder function on a folder of custom photos I get the following error:

Traceback (most recent call last): File "batch_giraf_detect.py", line 23, in results = detection_model.batch_image_detection(loader) File "/home/jaimy/CameraTraps/PytorchWildlife/models/detection/yolov5/base_detector.py", line 150, in batch_image_detection for pred in torch.cat(total_preds, dim=0).cpu() NotImplementedError: There were no tensor arguments to this function (e.g., you passed an empty list of Tensors), but no fallback function is registered for schema aten::_cat. This usually means that this function requires a non-empty list of Tensors, or that you (the operator writer) forgot to register a fallback function. Available functions are [CPU, CUDA, QuantizedCPU, BackendSelect, Python, Named, Conjugate, Negative, ADInplaceOrView, AutogradOther, AutogradCPU, AutogradCUDA, AutogradXLA, AutogradLazy, AutogradXPU, AutogradMLC, AutogradHPU, AutogradNestedTensor, AutogradPrivateUse1, AutogradPrivateUse2, AutogradPrivateUse3, Tracer, UNKNOWN_TENSOR_TYPE_ID, Autocast, Batched, VmapMode].

CPU: registered at aten/src/ATen/RegisterCPU.cpp:18433 [kernel] CUDA: registered at aten/src/ATen/RegisterCUDA.cpp:26496 [kernel] QuantizedCPU: registered at aten/src/ATen/RegisterQuantizedCPU.cpp:1068 [kernel] BackendSelect: fallthrough registered at ../aten/src/ATen/core/BackendSelectFallbackKernel.cpp:3 [backend fallback] Python: registered at ../aten/src/ATen/core/PythonFallbackKernel.cpp:47 [backend fallback] Named: registered at ../aten/src/ATen/core/NamedRegistrations.cpp:7 [backend fallback] Conjugate: registered at ../aten/src/ATen/ConjugateFallback.cpp:18 [backend fallback] Negative: registered at ../aten/src/ATen/native/NegateFallback.cpp:18 [backend fallback] ADInplaceOrView: fallthrough registered at ../aten/src/ATen/core/VariableFallbackKernel.cpp:64 [backend fallback] AutogradOther: registered at ../torch/csrc/autograd/generated/VariableType_3.cpp:10141 [autograd kernel] AutogradCPU: registered at ../torch/csrc/autograd/generated/VariableType_3.cpp:10141 [autograd kernel] AutogradCUDA: registered at ../torch/csrc/autograd/generated/VariableType_3.cpp:10141 [autograd kernel] AutogradXLA: registered at ../torch/csrc/autograd/generated/VariableType_3.cpp:10141 [autograd kernel] AutogradLazy: registered at ../torch/csrc/autograd/generated/VariableType_3.cpp:10141 [autograd kernel] AutogradXPU: registered at ../torch/csrc/autograd/generated/VariableType_3.cpp:10141 [autograd kernel] AutogradMLC: registered at ../torch/csrc/autograd/generated/VariableType_3.cpp:10141 [autograd kernel] AutogradHPU: registered at ../torch/csrc/autograd/generated/VariableType_3.cpp:10141 [autograd kernel] AutogradNestedTensor: registered at ../torch/csrc/autograd/generated/VariableType_3.cpp:10141 [autograd kernel] AutogradPrivateUse1: registered at ../torch/csrc/autograd/generated/VariableType_3.cpp:10141 [autograd kernel] AutogradPrivateUse2: registered at ../torch/csrc/autograd/generated/VariableType_3.cpp:10141 [autograd kernel] AutogradPrivateUse3: registered at ../torch/csrc/autograd/generated/VariableType_3.cpp:10141 [autograd kernel] Tracer: registered at ../torch/csrc/autograd/generated/TraceType_3.cpp:11560 [kernel] UNKNOWN_TENSOR_TYPE_ID: fallthrough registered at ../aten/src/ATen/autocast_mode.cpp:466 [backend fallback] Autocast: fallthrough registered at ../aten/src/ATen/autocast_mode.cpp:305 [backend fallback] Batched: registered at ../aten/src/ATen/BatchingRegistrations.cpp:1016 [backend fallback] VmapMode: fallthrough registered at ../aten/src/ATen/VmapModeRegistrations.cpp:33 [backend fallback]

When I use the single image detect code as shown in the ReadTheDocs on a custom photo, it runs properly. Although in the output photo with bounding box, the colors seem to be altered a bit.

My photos are JPG's around 3 MB's in size each and a resolution of 6048 x 4024.

I've already tried to use the CPU instead of the GPU and lower the resolution of the images. But since a single image detect does work, this would probably not be the issue.

Any help would be much appreciated!

[zhmiao]

Hello @JaimyvS, thank you very much for submitting the issue, and thank you for trying Pytorch-Wildlife out! This NotImplementError is kind of new to us. Would you mind sharing your pytorch version with us? Also, what operating system are you using? Is it a mac or linux? Me and @aa-hernandez will take a look at this and see if we could reproduce this error.

Also, since it shows "(e.g., you passed an empty list of Tensors)" on this line 150, in batch_image_detection, I am wondering, whether you would mind to print out total_preds before line 148? Maybe total_preds was an empty list, which means nothing was produced in line 140. In that case, and since you mentioned single image detection worked, would you mind debugging by creating a folder only consist a few worked single images, and run batch detection on this example folder and see if the results come through? If yes, then maybe the model didn't predict anything in your previous batch folder, if no, then there is a chance the dataloader didn't work. Then would you mind print out len(dataloader) before line 139

And yes, it actually doesn't matter with the input image sizes because all images will be down sampled to adapt to the model requirements. And I doubt it is about cpu or gpu.

[JaimyvS]

Thanks for the response!

I'm using Windows Subsystem for Linux 2 on Windows 11 V23H2. Running Ubuntu 22.04.3 LTS with PyTorch version 1.10.1

I've checked total_preds and it does print out an empty list. I also created a folder of 3 images that work when using single image detect. But not with the batch detection. Length of dataloader is 0.

When I remove my own photos from the folder and add your demo photos of the Tamandua in the Amazon. It does work like it should.

[zhmiao]

Hello @JaimyvS, thank you very much for the information. So based on what you said, in the same folder, the Tamandua images work but your own images don't, right? It might be some file format issue. Would you mind print out self.images before this line to make sure if the images are recognized by glob. The default file extension is JPG. I am afraid the images are not recognized automatically. If your images are not correctly picked up by glob, then it must be the file extensions are different. We need to manually specify the extension for pw_data.DetectionImageFolder(). We are sorry we didn't clarify this in our demo notebook. Also please keep in mind that the extension here is case sensitive because it is a string search function. So if your images are x.jpg, then your extension argument should be extension="jpg". Maybe we should make this image extension more universal in future updates so we don't need to manually specify case sensitive extensions.

If your images are correctly picked up by glob, then there must be some other issues. I wonder if you would mind sending us one or two images from your data for us to do some further testing? Thanks again!

[JaimyvS]

Ahh! The issue was in the file extension. Adobe Lightroom exports into lowercase .jpg, so all my photos were formatted like this. After changing the file extensions to .JPG, the batch detection script works. Thanks for the explanation!

Do you also happen to know why the output images are tinted differently than the input images. This is the input: And this is the output:

[zhmiao]

Hello @JaimyvS, this is actually a bug and we have just fixed this in our latest commits https://github.com/microsoft/CameraTraps/commit/0c15bbff4321ac4afc99fa6b197c8cc9ea33abe2. Generally the image saving function we use came from supervision which uses opencv. opencv uses BGR channel rather than RGB channel, while the default image generation functions we use are numpy and PIL, which are RGB images. This channel missmatch is the cause of this weird color output. We have updated the package version to 1.0.2.4. Please update the package using pip install PytorchWildlife --upgrade for the fix, and let us know if the issue still persists!

Also, I want to mention that the argument where you can change the file extensions in pw_data.DetectionImageFolder() instead of the Transform function. I made a mistake. You can also check our updated demo, for example this line.

[JaimyvS]

Yes, it works now! Thank you for the fast help with my issues!

[zhmiao]

@JaimyvS , we are always happy to help!