search

OpenDroneMap / NodeMICMAC

A Lightweight REST API to Access MICMAC Photogrammetry and SFM Engine.
https://opendronemap.org/nodemicmac/
GNU General Public License v3.0
79 stars 21 forks source link

MicMac results in vignetting and holes (no data) in orthomosaic #25

Closed NicholasK7 closed 2 years ago

NicholasK7 commented 5 years ago

How did you install WebODM? (Docker, natively, ...)?

Installed WebODM 0.9.1 in DockerToolbox $ ./webodm.sh update

What's your browser and operating system?

Chrome 75 on Windows 10

What is the problem?

Output on MicMac node (default processing options) resulted in vignetting and minor yet noticeable holes orthmosaic aoi (see MicMac default with swipe over ODM custom here)

Note: images may take a bit longer than usual to load in Juxtapose

What should be the expected behavior?

Ran default and custom options with ODM 0.9.0 (see comparison here) without the vignetting, but ODM custom options did produce holes, although not apparently in aoi (ie uniform block of trees)...

How can we reproduce this? (What steps did you do to trigger the problem? What parameters are you using for processing? If possible please include a copy of your dataset uploaded on Google Drive or Dropbox. Be detailed)

Original flight parameters, ODM custom options, and system specs here

Unable to provide a copy of the image set...However, I am willing to run the images through Lightning Network and modify the options with any suggestions if there is a way to select MicMac processing.

dronemapper-io commented 5 years ago

Thanks for providing detailed information and screenshots. I think there are a few optimizations to the Ortho/Radiometric Balance process that may improve your situation. I'll do an update tomorrow and you can re-test/process.

dronemapper-io commented 5 years ago

MM and NodeMicMac updated, should improve results: https://github.com/dronemapper-io/NodeMICMAC/commit/fa7a53e32e7001a249b9050ebe1610e36013ddfa https://github.com/dronemapper-io/micmac/commit/c6923e6251e4f29599f500d997e2804d5764932e

dronemapper-io commented 5 years ago

closing for now, please attempt processing with latest docker and improved parameters. thanks

NicholasK7 commented 5 years ago

Updated to WebODM 1.0.0 via command: $ ./webodm.sh update

after update was successful ran command $ ./webodm.sh restart --with-micmac

Then reprocessed same 697 images but till get same results as before. Is this sufficient to capture your changes?

Thank you

pierotofy commented 5 years ago

Nop, you should update with:

./webodm.sh update --with-micmac

NicholasK7 commented 5 years ago

After update --with-micmac reprocessed same 697 images (with exif tags) now I get no Orthophoto or Point Cloud ??

MicMac_Orthofail

(btw, when I try to download the console file it gives me a Failed - Network error)

DSM that was generated appears to have some differences from the previous process. The overall range in raster values (meters) went from 44.0235 - 27.1404 to 44.6808 - 26.5689, (16.8831‬ to 18.1119).

Q1: is there an indication in the processing that exif tags are not being correctly processed? Q2: are dsm files generated from the same original photos going to vary from each processing?

Thanks

dronemapper-io commented 5 years ago

Sorry for the problems. If you could share the options/parameters you used for processing that might help. Also, if you would like to share the log file privately -- we can evaluate.

Q1 reply: The only indication is the failure of the processing after tie-point extraction and during the first bundle block stage Q2 reply: Yes the DSM could differ but it shouldn't significantly change

Something that might help would be to use the "multi-scale" tie-point options and set the initial image size to 800. It will take longer but produce more feature matches in homogenous terrain.

If you could share the dataset privately, I could evaluate that as well.

NicholasK7 commented 5 years ago

Options were default. Log file download from webodm fails. Do you have the path to the file in docker?

Before the last update I tried the 'multi-scale' option but it failed, seemed to overload already minimal memory resources.

Unfortunately I can not share the dataset.

dronemapper-io commented 5 years ago

No worries, understand. If the 'multi-scale' option fails due to server resources / memory that could be related to the issue. You can try the default settings but use the initial image size as 1000 or more pixels. I noticed it took 62 hrs to process which seems abnormally long.

NicholasK7 commented 5 years ago

...It has always taken 60+ hours. However, the micmac orthomosaic qualities (minus the vignetting and no data holes) seemed to be better than the default of any other cloud based processing option...is there any system benchmarking data for processing times?

Do you have an alternative path to the console (process log) file? Hoping that will provide insight into what is causing the problem.

dronemapper-io commented 5 years ago

I'm not sure if WebODM saves the console log somewhere. Maybe others can help identify that location if it exists?

NicholasK7 commented 5 years ago

I'm currently reprocessing original 697 images running default options with nodeMicMac on a Google Cloud Platform instance (Machine type: n1-standard-8 (8 vCPUs, 30 GB memory; 100GB storage) in hopes it will isolate any local resource issues.

image

The inital console output is indicating the following:

Error: Directory Iop: Next pointer is out of bounds; ignored. Error: Directory (Last IFD item) with 65205 entries considered invalid; not read...

(error message is repeated with different numbers before the 'entries', eventually followed by a subset of the images like shown below, then more 'error:'...followed by the next subset of images)

Not sure if this error was present when I first ran MicMac and was able to generate an orthophoto on the same dataset, but so far this is the only indication something is not processing correctly...

NicholasK7 commented 5 years ago

MicMac failed to generate an Orthophoto & LAS on the dataset it had previously been successful with. The failure was the same on a Google Cloud instance so I can't see it being a local resource issue...

image

...although it only took 13 hrs on Google Cloud vs 60+ on my laptop.

Complete console output file here

dronemapper-io commented 5 years ago

Thanks for the log. That helps! I believe the problem might be a recently introduced option to Tawny DEq=1 or other options ... I will reverse that and push a new release. It might help. 

[DEBUG]   running mm3d Tawny Ortho-MEC-Malt DEq=1 RadiomEgal=1 DegRapXY=4 SzV=25
------------------------------------------------------------
|   Sorry, the following FATAL ERROR happened
|
|    REadPt
|
------------------------------------------------------------
-------------------------------------------------------------
|       (Elise's)  LOCATION :
|
| Error was detected
|          at line : 1169
|          of file : /var/www/micmac/src/util/pt2di.cpp
-------------------------------------------------------------
Bye  (press enter)
[ERROR]   Child returned 1
[DEBUG]   running mm3d Nuage2Ply MEC-Malt/NuageImProf_STD-MALT_Etape_9.xml Attr=Ortho-MEC-Malt/Orthophotomosaic.tif 64B=1 Out=/var/www/data/73277a88-0d38-4a6a-8148-5553eeeb62a6/odm_georeferencing/odm_georeferenced_model.ply
ERROR: colour image [Ortho-MEC-Malt/Orthophotomosaic.tif] does not exist
[ERROR]   Child returned 1
dronemapper-io commented 5 years ago

I've made some updates to the ortho creation/color balance. https://github.com/dronemapper-io/NodeMICMAC/commit/4070a6eb28b13a9bd35588f6fb92934322b2370c

I would wait for docker hub to finish building the new image and give it a try. Thanks

kikislater commented 5 years ago

@dronemapper-io strange for DEq=1!

dronemapper-io commented 5 years ago

@kikislater no worries, probably other Tawny commands / options as well. I switched it over to Porto with a custom config xml that we've been using at DroneMapper. Should work better for many cases.... fingers crossed https://github.com/dronemapper-io/micmac/commit/1646049bbd0cb5abf12ff663fe93a65c56fa37cb

NicholasK7 commented 5 years ago

After running update --with-micmac attempt 2 on GCP with MicMac failed to generate an Orthophoto and render PLY...

60457123-1ea38500-9bf0-11e9-9ede-05c8fe4cee55

...I was able to download a .ply file individually that was in the MB range of previous MicMac processing, but no luck with the orthophoto

Console output here

dronemapper-io commented 5 years ago

I've reverted one other MicMac commit -- you can test it once the docker image is done building. https://github.com/dronemapper-io/micmac/commit/b9475765c20f8297630f4a4f3319451dfe08bba4

I would also recommend testing your entire process with the DroneMapper 4th Ave. example data set as well. Thanks

NicholasK7 commented 5 years ago

After starting a new instance on Google Cloud and running latest updates yesterday the issues with no ortho/PLY outputs on the 697 image set persisted, same output as noted July 1 (screenshots here)

However I did run a different 263 image set, over varying forest conditions, on a new instance Google Cloud and was able to get all the outputs...though there are some ortho and dsm discrepancies. The ortho/dsm comparison between nodeODM and nodeMicMac is here

Btw, is there a way to get a '.las' file instead of the '.ply' in the MicMac output?

Initial assessment of 4th Ave output using node MicMac in Google Cloud looks good (screenshot here)

NicholasK7 commented 5 years ago

Update Still no luck with the 697 image dataset...latest console output txt file and WebODM screenshots here

Any thoughts on why the ortho is not rendering?

Just fyi, after analyzing the 4th Ave ortho and dsm my processing varied considerably, particularly for the trees...GCP instance is 8vCPUs x 30GB memory x 150GB storage: monitoring graphs Results .zip Console output txt file

dronemapper-io commented 5 years ago

I'll take a look.. but unfortunately, I can't really diagnose and fix the issue or determine the issue without the data/imagery. Thanks

NicholasK7 commented 5 years ago

Let me know if you get this message and I can send a gdrive link to the 697 image set I have been having trouble with in nodeMicMac.

Also wondering if I could upload the data through DroneMapper and if you offer a pay as you go service similar to Lightning?

Thank you for any help on this.

-Nick

On Mon, Jul 15, 2019 at 5:37 PM DroneMapper.com notifications@github.com wrote:

I'll take a look.. but unfortunately, I can't really diagnose and fix the issue or determine the issue without the data/imagery. Thanks

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/dronemapper-io/NodeMICMAC/issues/25?email_source=notifications&email_token=AMIU65LXPSXGXZD7Z4LJ4O3P7UJ3HA5CNFSM4HVO75O2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZ7LA4I#issuecomment-511619185, or mute the thread https://github.com/notifications/unsubscribe-auth/AMIU65NTSD6AVWPGJTPLC3TP7UJ3HANCNFSM4HVO75OQ .

dronemapper-io commented 5 years ago

We can definitely take a look and try the processing locally with nodeMicMac.. please send a contact via https://dronemapper.com

dronemapper-io commented 5 years ago

I wasn't able to fix the bug yet. But for this data set a potential workaround is to remove every other image and run the processing. I got a well balanced Ortho, DEM and Point Cloud. I shared the results privately to you. Thanks

image

kikislater commented 2 years ago

Closed due to inactivity