Closed patrickelectric closed 7 years ago
kikislater
commented
7 years ago Here is a Parrot sequoia, you try to upload 3 band RGB 16mpix and 4 x 1 band 1.2mpix at the same time. Also did you apply vignetting correction and reflectance estimation of each band before processing ?
patrickelectric
commented
7 years ago @kikislater I am doing this in the same time to see if it's possible to process all bands with the RGB exif info. @kikislater I will add this configuration now, but do you know how to realize the orthomosaic with the others bands that don't have exif gps data ? Update: I can't find vignetting and reflectance configurations in ODM : (
kikislater
commented
7 years ago It's a bad idea to process RGB and 4 bands. What you should expect, no way to do that ! I don't understand and if you are a gis user it's a non sense to doing it.
If you don't have GPS in exif with 4 bands, there is a problem during acquisition. Low power of sequoia. I use other software at this time to process it, but I'm starting a project few weeks ago to process it with opensource software. Could be fastest ... time will say it !
pierotofy
commented
7 years ago I'm pretty sure the error is caused by the low default DATA_UPLOAD_MAX_NUMBER_FIELDS value (which is set to 1000).
I will increase this to 10000 for the next release, in the meanwhile, you might want to add the following to your webodm\settings.py file and rebuild webodm (if using docker) or restart it if running natively:
DATA_UPLOAD_MAX_NUMBER_FIELDS=10000Thanks for reporting the issue!
patrickelectric
commented
7 years ago @kikislater
It's a bad idea to process RGB and 4 bands. What you should expect, no way to do that ! I don't understand and if you are a gis user it's a non sense to doing it.
Pix4D does that.. I was just testing it with ODM. This project that ou talked about is in GitHub ? Can you share it here ?
Wow, @pierotofy tys !
pierotofy
commented
7 years ago Note that even after increasing DATA_UPLOAD_MAX_NUMBER_FIELDS I'm still having other issues with larger than usual datasets. I will post updates when I know more.
kikislater
commented
7 years ago What you have after with pix4d ? An RGB ortho and a multispectral ortho ? If it's that, there is a simply script to separate datas and process but afaik ODM does not support it ! Not at this time, project is in my computer ^_^ but I will push it on ODM or QGIS ...
I answer myself : Imagery taken with Sequoia will be automatically recognized since the camera is in the Pix4Dmapper camera database. RGB images should be processed in a different project. from here : https://support.pix4d.com/hc/en-us/articles/209362146-How-to-process-Sequoia-imagery#gsc.tab=0 So it's just a script to separate jpeg and tif and having 2 projects
pierotofy
commented
7 years ago Besides setting a higher value for DATA_UPLOAD_MAX_NUMBER_FIELDS, the problem goes all the way down to the operating system...
# ulimit -a
core file size (blocks, -c) unlimited
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 95659
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 95659
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimitedThe open files limit sets a cap on the number of images that can be currently uploaded with the current code. It's possible to increase this limit by running:
# ulimit -n 3000I'm currently evaluating possible workarounds to this issue.
patrickelectric
commented
7 years ago @pierotofy It's possible to upload 1k photos, and than when clicking in upload again, it will upload more 1k photos if the user select to append the actual project photos with the one that he want to upload.
pierotofy
commented
7 years ago We can do better than that :)
These new changes https://github.com/OpenDroneMap/WebODM/pull/230/commits/d376a1e779bc197aa01a95b435b91d9ebec72104 allow WebODM to handle a lot more files, now capped by the amount of RAM available on the computer and not by the "open files" OS limit.
In the future we will have to switch to a streamed approach to handle even larger file uploads.
Changes will be available in the next few days, as soon as #230 is merged in.
I am testing with different bands, GRE, NIR, RED, REG and RGB.
Computer: