Closed merkato closed 8 years ago
Very cool though, I compared the results of these SW products but not the time needed for processing. Please be aware of the fact that Photoscan might utilize your GPU, as well as other things like reconstruction density... (Standard outcome for the geotiff was in my case 150mb (ODM) to approx. 1.5gb in Photoscan.)
On Boruszyn dataset, with same hardware specs (Lenovo T500 laptop), output was 496 MB Tiff (ODM), 52 MB (LZW, Photoscan). Execution times were: 1h20' for ODM, 5h15' for Photoscan. And results are comparable - holes in texturing are visible. On the other side, for my Koczy Zamek site dataset (46 photos + GPS), with old quarry, Photoscan is many times better, OpenSfM created unconected 3 reconstructions, and ortho for that was textured with sky from photos. In Photoscan Orthophoto was generated correctly. How can I help with development for making better results with mixed datasets (oblique + nadir, only oblique, only horizontal), and for more difficult terrain (hills, rocks, etc.)
This is awesome, @merkato. When you run git rev-parse HEAD
inside the ODM repo, what do you get in response?
What parameters, if any are you running ODM with?
~/dev/OpenDroneMap$ git rev-parse HEAD
3fcc58da7a88f1d2f21807f97395d78b29692ae5
Boruszyn dataset was reconstructed with default settings !!! :dart:
Keep calm and do better SfM ;)
I added it to the roadmap for the codesprint
@merkato if you wanna help with development, we are doing a code sprint this week. Feel free to join on the gitter chat
I've been looking into profiling / benchmarking and found cProfile which profiles every single command, but this seems a little too verbose to be useful:
python -m cProfile -s cumtime run.py --project-path tests/test_data/ --rerun pmvs
...
250266 function calls (248001 primitive calls) in 997.591 seconds
Ordered by: cumulative time
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.009 0.009 997.594 997.594 run.py:3(<module>)
3 0.000 0.000 991.982 330.661 system.py:16(run)
3 991.981 330.660 991.981 330.660 {posix.system}
1 0.001 0.001 990.734 990.734 pmvs.py:35(process)
1 0.010 0.010 3.798 3.798 dataset.py:21(process)
57 0.035 0.001 3.787 0.066 types.py:16(__init__)
57 0.020 0.000 3.748 0.066 types.py:51(parse_pyexiv2_values)
57 3.280 0.058 3.280 0.058 {cv2.imread}
1 0.007 0.007 2.485 2.485 odm_georeferencing.py:25(process)
1 0.000 0.000 1.249 1.249 types.py:127(convert_to_las)
57 0.017 0.000 1.228 0.022 types.py:146(utm_to_latlon)
57 0.003 0.000 0.521 0.009 system.py:30(run_and_return)
57 0.512 0.009 0.513 0.009 metadata.py:110(write)
171 0.002 0.000 0.371 0.002 subprocess.py:473(_eintr_retry_call)
57 0.003 0.000 0.360 0.006 subprocess.py:650(__init__)
57 0.014 0.000 0.350 0.006 subprocess.py:1187(_execute_child)
2 0.078 0.039 0.270 0.135 __init__.py:29(<module>)
1 0.002 0.002 0.233 0.233 odm_app.py:1(<module>)
1 0.054 0.054 0.230 0.230 types.py:1(<module>)
57 0.220 0.004 0.220 0.004 {posix.read}
3876 0.004 0.000 0.199 0.000 exif.py:188(_get_value)
3876 0.012 0.000 0.195 0.000 exif.py:171(_compute_value)
3876 0.016 0.000 0.193 0.000 metadata.py:196(__getitem__)
3876 0.104 0.000 0.171 0.000 metadata.py:163(_get_exif_tag)
4104 0.026 0.000 0.164 0.000 exif.py:257(_convert_to_python)
57 0.002 0.000 0.156 0.003 subprocess.py:768(communicate)
114 0.151 0.001 0.151 0.001 {method 'read' of 'file' objects}
So I'll write something more specific to this project. Something like for each part in the chain append the total runtime for that process to a benchmark file. Should be simple enough.
@merkato do you think you could check out this PR? #258 has implemented a fairly simple benchmarking tool. Is there anything else you think I should add to it? EDIT: For your reference, this is the file it produces:
ODM Benchmarking file created Wed Feb 24 15:41:40 2016
Number of Cores: 32
Resizing runtime: 0.001394 seconds
OpenSfM runtime: 0.000214 seconds
CMVS runtime: 0.000133 seconds
PMVS runtime: 0.000111 seconds
Meshing runtime: 0.000129 seconds
Texturing runtime: 0.000129 seconds
Georeferencing runtime: 2.216099 seconds
+1
+1 I'll try on saturday morning. Right now I'm learning new things on railway. Till 10th of March There's no free time. @dakotabenjamin thank you for this PR.
I've cloned your cmake-benchmarks from your fork, and build it with no problems.
But... There's no effects :(
With this parameters run.py --time --project-path /home/mechanik/dev/odm_data/odm_boruszyn_kap/ --odm_georeferencing-gcpFile /home/mechanik/dev/odm_data/odm_boruszyn_kap/gcp_list.txt --odm_georeferencing-useGcp
project gets reconstructed, but georef is failed...
[INFO] Running OMD Georeferencing Cell
[DEBUG] running /home/mechanik/dev/OpenDroneMap/build/bin/odm_georef -bundleFile /home/mechanik/dev/odm_data/odm_boruszyn_kap/opensfm/bundle_r000.out -inputCoordFile /home/mechanik/dev/odm_data/odm_boruszyn_kap/odm_georeferencing/coords.txt -inputFile /home/mechanik/dev/odm_data/odm_boruszyn_kap/odm_texturing/odm_textured_model.obj -outputFile /home/mechanik/dev/odm_data/odm_boruszyn_kap/odm_texturing/odm_textured_model_geo.obj -inputPointCloudFile /home/mechanik/dev/odm_data/odm_boruszyn_kap/pmvs/recon0/models/option-0000.ply -outputPointCloudFile /home/mechanik/dev/odm_data/odm_boruszyn_kap/odm_georeferencing/odm_georeferenced_model.ply -logFile /home/mechanik/dev/odm_data/odm_boruszyn_kap/odm_georeferencing/odm_georeferencing_log.txt -georefFileOutputPath /home/mechanik/dev/odm_data/odm_boruszyn_kap/odm_georeferencing/odm_georeferencing_model_geo.txt
Error in Georef:
Failed opening coordinate file /home/mechanik/dev/odm_data/odm_boruszyn_kap/odm_georeferencing/coords.txt for reading.
quitting cause:
/home/mechanik/dev/OpenDroneMap/build/bin/odm_georef -bundleFile /home/mechanik/dev/odm_data/odm_boruszyn_kap/opensfm/bundle_r000.out -inputCoordFile /home/mechanik/dev/odm_data/odm_boruszyn_kap/odm_georeferencing/coords.txt -inputFile /home/mechanik/dev/odm_data/odm_boruszyn_kap/odm_texturing/odm_textured_model.obj -outputFile /home/mechanik/dev/odm_data/odm_boruszyn_kap/odm_texturing/odm_textured_model_geo.obj -inputPointCloudFile /home/mechanik/dev/odm_data/odm_boruszyn_kap/pmvs/recon0/models/option-0000.ply -outputPointCloudFile /home/mechanik/dev/odm_data/odm_boruszyn_kap/odm_georeferencing/odm_georeferenced_model.ply -logFile /home/mechanik/dev/odm_data/odm_boruszyn_kap/odm_georeferencing/odm_georeferencing_log.txt -georefFileOutputPath /home/mechanik/dev/odm_data/odm_boruszyn_kap/odm_georeferencing/odm_georeferencing_model_geo.txt
returned with code 256.
git rev-parse HEAD
2db0411b27dae745e29307dd8de1383caf152cb9
And there's no benchmark file. Maybe I can't find it in project folder.
Check the new PR #264
Merged #264
If we start ODM with --use-opensfm-pointcloud benchmark file looks like this:
Resizing runtime: 20.945608 seconds
Meshing runtime: 78.041546 seconds
Texturing runtime: 269.293121 seconds
Georeferencing runtime: 191.849991 seconds
Orthophoto runtime: 79.996433 seconds
I think we're missing OpenSfM sparse and dense reconstruction here.
git rev-parse HEAD
5b0cd53f2ecd7422ca5461cb14cc55eeb7c34fba
It will be somehow practical to create some benchmarking - for example log file with execution time for toolchain elements. Right now these things are echoed to terminal - we only need some command line switch (eg. --time) and results file.
Right now I'm trying to compare ODM with Agisoft PhotoScan on my machine, and ODM worked about 1h20 min, Agisoft is working 2h 10 minutes right now, and probably next 2h for Dense Reconstruction stage ;) Stay tuned for results ;)