Quitting due to error 35072

[clercj]

Hello,

I'm trying some data from a drone flight (Phantom 3 Advanced) with OpenDroneMap (installed on MacOs and Docker) but it keep crashing at the dense point cloud generation. Above an extract of the console:

`[...] DoG running...4253 dog done 177 Harris running ...2979 harris done DoG running...4251 dog done 178 Harris running ...2976 harris done DoG running...4255 dog done 180 Harris running ...2779 harris done DoG running...4255 dog done done Killed [ERROR] quitting cause: /code/SuperBuild/install/bin/pmvs2 /code/pmvs/recon0/ option-0000 returned with code 35072.

An error occurred. Check stdout above or the logs.`

I cannot remember of any errors at installing. I followed the instruction of the github site. Any idea?

[mojodna]

Killed makes me fairly certain that the system ran out of memory and the OOM Killer kicked in.

Try with a subset of the source imagery? How many photos are you trying to process and how much RAM is available?

[clercj]

The original data has 180 pictures. I've tried with a sample of it (~50 images) and this work fine. I guess this is also a run out of memory... But my computer has 16Gb RAM.

[mojodna]

Hmm, that seems reasonable. Try 10 or so?

You may also be hitting an edge case, so if you're willing to share the dataset, that could be helpful.

You can also try using --use-opensfm-pointcloud when running, which will use OpenSfm in place of pmvs during the phase where it crashes (I think).

[clercj]

Well, not better, I've tried with the whole set and opensfm...

`Traceback (most recent call last): File "/code/SuperBuild/src/opensfm/bin/opensfm", line 34, in command.run(args) File "/code/SuperBuild/src/opensfm/opensfm/commands/compute_depthmaps.py", line 25, in run dense.compute_depthmaps(data, graph, reconstructions[0]) File "/code/SuperBuild/src/opensfm/opensfm/dense.py", line 40, in compute_depthmaps merge_depthmaps(data, graph, reconstruction, neighbors) File "/code/SuperBuild/src/opensfm/opensfm/dense.py", line 162, in merge_depthmaps points, normals, colors = dm.merge() MemoryError [ERROR] quitting cause: PYTHONPATH=/code/SuperBuild/install/lib/python2.7/dist-packages /code/SuperBuild/src/opensfm/bin/opensfm compute_depthmaps /code/opensfm returned with code 256.

An error occurred. Check stdout above or the logs.`

Which parameter do I change to allocate more ram on docker/ODM?

The data here: https://transfer.pcloud.com/download.html?code=5Z5fwPZst1rS5LJJbbZ9zhJZXUqumLOpGohME6CTHUgHaJg8YovV

[dakotabenjamin]

@mojodna would camera calibration reduce memory costs here?

My experience with DJI cameras is that its pretty much necessary to do that.

[dakotabenjamin]

FWIW I had no problem running on my machine: 16 core 44GB RAM using docker. Looks great. No calibration either.

Try pulling the latest from master and building docker with this command:

docker build -t <tag> -f core2.Dockerfile .

It could be a macOS problem ( @pierotofy is this one of the instances you are referring to?)

[pierotofy]

Mm, no, using core2.Dockerfile would not solve the problem. The regular Dockerfile already compiles the code for the current CPU. core2.Dockerfile is to be used only for building an image that gets pushed to the docker hub (if I compile it on a Linux machine with a newer CPU, I want to make sure it runs on a Mac with an older CPU).

If I build the image myself via docker build, then there's no problem. Hope this makes sense.

[dakotabenjamin]

I see, thanks.

[dakotabenjamin]

Using default cmvs/pmvs, you can try adjusting --cmvs-maxImages n and set n to something smaller than the 500 default until it works. try maybe 100 first.

On another note, I ran the first time no problem, but now I can't seem to get it to work again, even though I'm deleting the file system using --rm. Anyone with more docker experience have advice?

docker run -it --rm -v ~/clercj/images:/code/images -v ~/clercj/orth:/code/odm_orthophoto odm_image

[pierotofy]

I had the same problem as you @dakotabenjamin (no such file or directory: ... reconstruction.json) a few days ago using Docker. I think somehow the images are getting changed/saved in some way. Does OpenDroneMap change the original images in any way? (The ones mounted on /code/images/)?

[dakotabenjamin]

There's a bug: https://github.com/OpenDroneMap/OpenDroneMap/pull/411#issuecomment-262599320

[clercj]

I've tried with --cmvs-MaxImages 100 but I get the same error as before...

I've tried to follow the thread, so I'm not the only one that get some memory errors

[pierotofy]

Try to follow these instructions to increase the memory of your Docker environment: http://stackoverflow.com/a/39720010

[clercj]

I could increase the memory of Docker and I could process with success the whole dataset of 180 images. But now, I'm not able anymore to reproduce these results. I get again the "PYTHONPATH"-error again.

[pierotofy]

This is the bug in #411

Make sure that the images in your images folder are new (haven't been processed before). This is going to be fixed soon.

As an alternative until #411 is fixed, use https://github.com/pierotofy/node-OpenDroneMap which uploads brand new copies of the images each time for processing.

[clercj]

Ok, that's what I thought.

Thanks in advance for all of your great work!

[pierotofy]

Thank you for reporting the issue!

[dakotabenjamin]

@clercj Can you pull from master and try with a clean dataset? (not rerun)

[clercj]

I've tried both with PMVS and OpenSfm, it works pretty well with each time a clean dataset.

I'm now struggling with using GCP's...

[dakotabenjamin]

Have you looked at the wiki?

[clercj]

Sure. More than once.

Here is the input:

'MacBook-Pro:opendronemap jonasclerc$ docker run -it --user root -v $(pwd)/project/images:/code/images -v $(pwd)/project/pmvs:/code/pmvs -v $(pwd)/project/odm_meshing:/code/odm_meshing -v $(pwd)/project/odm_texturing:/code/odm_texturing -v $(pwd)/project/odm_georeferencing:/code/odm_georeferencing
-v $(pwd)/project/odm_orthophoto:/code/odm_orthophoto --rm odm_image
--odm_georeferencing-useGcp '

and the "gcp_list.txt" is located in the Project-folder

Here the file: gcp_list.txt

the log after it crashes: odm_georeferencing_log.txt

and the console message

`[INFO] Running ODM Texturing Cell - Finished [INFO] Running ODM Georeferencing Cell [DEBUG] running /code/build/bin/odm_georef -bundleFile /code/opensfm/bundle_r000.out -inputCoordFile /code/odm_georeferencing/coords.txt -inputFile /code/odm_texturing/odm_textured_model.obj -outputFile /code/odm_texturing/odm_textured_model_geo.obj -inputPointCloudFile /code/pmvs/recon0/models/option-0000.ply -outputPointCloudFile /code/odm_georeferencing/odm_georeferenced_model.ply -logFile /code/odm_georeferencing/odm_georeferencing_log.txt -georefFileOutputPath /code/odm_georeferencing/odm_georeferencing_model_geo.txt Error in Georef: Failed opening coordinate file /code/odm_georeferencing/coords.txt for reading.

[ERROR] quitting cause: /code/build/bin/odm_georef -bundleFile /code/opensfm/bundle_r000.out -inputCoordFile /code/odm_georeferencing/coords.txt -inputFile /code/odm_texturing/odm_textured_model.obj -outputFile /code/odm_texturing/odm_textured_model_geo.obj -inputPointCloudFile /code/pmvs/recon0/models/option-0000.ply -outputPointCloudFile /code/odm_georeferencing/odm_georeferenced_model.ply -logFile /code/odm_georeferencing/odm_georeferencing_log.txt -georefFileOutputPath /code/odm_georeferencing/odm_georeferencing_model_geo.txt returned with code 256.

An error occurred. Check stdout above or the logs.`

[dakotabenjamin]

Yeah you found a bug!

[dakotabenjamin]

So when connecting volumes using -v, if the file is in the project root then it won't get copied over or whatever (I don't pretend to understand how docker works). So I recommend for now to put that file in /images or /odm_georeferencing then use --odm_georeferencing-gcpFile /code/images/gcp_list.txt . I'm testing this method now.

[dakotabenjamin]

@clercj I opened a new issue for this problem. Let's continue the conversation there. Please reopen this issue if you think we haven't solved the current issue.

[dakotabenjamin]

I wanted to comment on your particular dataset including the GCP file. I noticed that the georeferencing, while it was successful, was off by quite a bit from the google satellite image.

We are working on georeferencing improvements, but there are some things you can do. You can add more GCP points. You can use the QGIS plugin georeferencer to manually fix the data. You can try doing camera calibration (see here), which may or may not help.