search

OpenDroneMap / ODM

A command line toolkit to generate maps, point clouds, 3D models and DEMs from drone, balloon or kite images. 📷
https://opendronemap.org
GNU Affero General Public License v3.0
4.94k stars 1.12k forks source link

Possible bug: Different process result using CPU or GPU #1411

Closed markomms closed 2 years ago

markomms commented 2 years ago

How did you install OpenDroneMap? (Docker, natively, ...)?

Docker and native

What's your browser and operating system? (Copy/paste the output of https://www.whatismybrowser.com/)

Chrome 97 on Windows 11

What is the problem?

I am getting way too different results after processing data with GPU or CPU.

What should be the expected behavior? If this is a feature request, please describe in detail the changes you think should be made to the code, citing files and lines where changes should be made, if possible.

Same data should process very similar result no matter what processing type is used.

How can we reproduce this? (What steps did you do to trigger the problem? What parameters are you using for processing? If possible please include a copy of your dataset uploaded on Google Drive or Dropbox. Be detailed)

I have changed WebODM files so i can use GPU processing because its not docker supported on windows. First using nvidia-smi in gitbash so it detects cuda. The files of WebODM which were changed is webodm.sh in line 170 - _export GPUNVIDIA=true docker-compose.nodeodm.gpu.nvidia in whole file _version: '2.1' services: webapp: depends_on:

This way GPU is used for processing when using docker on windows with webodm.sh start --gpu

What happens after is that when dataset is processing with CPU the result is what its suppose to be or something like it. But when its processed with GPU results are basically half of size and result is not the same (basically half of it) Here is the dataset with logs and results: (https://hub.dronedb.app/r/markom/hlevtestdata)

Here im also attaching some other set which data i cant share but i can share logs. TestGPU.txt TestCPU.txt

If there is any consistent/veryfied dataset i can process i can do that too if needed. CPU Intel 12700k GPU Nvidia RTX 3070 RAM 64GB The topic with more insight of how GPU is made to work with docker. I have skipped some steps because its not needed as it looks like. https://community.opendronemap.org/t/installation-of-webodm-gpu-doesnt-detect-nvidia-mx150/10105/35

pierotofy commented 2 years ago

No a bug; the GPU pipeline is different in implementation and the pipeline in general has non-deterministic parts.