search

OpenDroneMap / ODM

A command line toolkit to generate maps, point clouds, 3D models and DEMs from drone, balloon or kite images. 📷
https://opendronemap.org
GNU Affero General Public License v3.0
4.71k stars 1.08k forks source link

OpenSfM fails: TypeError: int expected instead of float #885

Closed vgorte closed 5 years ago

vgorte commented 5 years ago

How did you install OpenDroneMap? (Docker, natively, ...)?

Docker

What's your browser and operating system?

Ubuntu 16.04.5 LTS

What is the problem?

Hello everyone,

I was trying to run ODM via Docker on a machine in my university. It works at first and it calculates the focal length for all images. When it should start with the opensfm part it suddenly crashes and gives me the following output:

[DEBUG] running PYTHONPATH=/code/SuperBuild/install/lib/python2.7/dist-packages /code/SuperBuild/src/opensfm/bin/opensfm detect_features /code/opensfm Traceback (most recent call last): File "/code/SuperBuild/src/opensfm/bin/opensfm", line 34, in command.run(args) File "/code/SuperBuild/src/opensfm/opensfm/commands/detect_features.py", line 30, in run parallel_map(detect, arguments, processes) File "/code/SuperBuild/src/opensfm/opensfm/context.py", line 37, in parallel_map with get_reusable_executor(max_workers=num_proc, timeout=None) as e: File "/usr/local/lib/python2.7/dist-packages/loky/reusable_executor.py", line 108, in get_reusable_executor executor_id=executor_id, **kwargs) File "/usr/local/lib/python2.7/dist-packages/loky/reusable_executor.py", line 144, in init initializer=initializer, initargs=initargs) File "/usr/local/lib/python2.7/dist-packages/loky/process_executor.py", line 894, in init self._setup_queues(job_reducers, result_reducers) File "/usr/local/lib/python2.7/dist-packages/loky/reusable_executor.py", line 205, in _setup_queues job_reducers, result_reducers, queue_size=queue_size) File "/usr/local/lib/python2.7/dist-packages/loky/process_executor.py", line 908, in _setup_queues reducers=job_reducers, ctx=self._context) File "/usr/local/lib/python2.7/dist-packages/loky/process_executor.py", line 287, in init super(_SafeQueue, self).init(max_size, reducers=reducers, ctx=ctx) File "/usr/local/lib/python2.7/dist-packages/loky/backend/queues.py", line 52, in init self._sem = ctx.BoundedSemaphore(maxsize) File "/usr/local/lib/python2.7/dist-packages/loky/backend/context.py", line 214, in BoundedSemaphore return BoundedSemaphore(value) File "/usr/local/lib/python2.7/dist-packages/loky/backend/synchronize.py", line 156, in init SemLock.init(self, SEMAPHORE, value, value) File "/usr/local/lib/python2.7/dist-packages/loky/backend/synchronize.py", line 70, in init unlink_now) File "/usr/local/lib/python2.7/dist-packages/loky/backend/semlock.py", line 157, in init self.handle = _sem_open(self.name.encode('ascii'), value) File "/usr/local/lib/python2.7/dist-packages/loky/backend/semlock.py", line 85, in _sem_open ctypes.c_int(value)) TypeError: int expected instead of float Traceback (most recent call last): File "/code/run.py", line 47, in plasm.execute(niter=1) File "/code/scripts/run_opensfm.py", line 116, in process (context.pyopencv_path, context.opensfm_path, tree.opensfm)) File "/code/opendm/system.py", line 34, in run raise Exception("Child returned {}".format(retcode)) Exception: Child returned 1

Interestingly enough it says that there is a TypeError where an int is expected but a float is received and I don't really understand why that is. The command I used to start ODM via Docker is : 

docker run -it  --cpus=8 --memory=30000mb  --rm \
    -v $(pwd)/images:/code/images \
    -v $(pwd)/opensfm:/code/opensfm \
    -v $(pwd)/odm_meshing:/code/odm_meshing \
    -v $(pwd)/odm_texturing:/code/odm_texturing \
    opendronemap/opendronemap

I'd appreciate any kind of help and if there is more information needed to identify the cause I'd gladly provide this information. Thanks!

pierotofy commented 5 years ago

@vgorte would you be able to share your dataset images so that we can try to reproduce this?

timmycam commented 5 years ago

Hi, @vgorte were you able to fix this? What did you have to do? I'm getting the same error while running natively Linux 16.04 LTS. Thanks!

smathermather commented 5 years ago

Reopening:

How did you install OpenDroneMap? (Docker, natively, ...)?

Natively

What's your operating system?

Ubuntu 16.04

What is the problem?

Running ODM on a functioning dataset results in the following error:

2019-01-21 03:03:44,105 INFO: Extracting ROOT_HAHOG features for image DSC00302.JPG
2019-01-21 03:03:44,375 DEBUG: Found 10461 points in 5.05034089088s
2019-01-21 03:03:44,386 INFO: Extracting ROOT_HAHOG features for image DSC00264.JPG
2019-01-21 03:03:44,652 DEBUG: Found 10543 points in 3.22489500046s
2019-01-21 03:03:45,117 DEBUG: Found 11043 points in 5.2380361557s
2019-01-21 03:03:49,549 DEBUG: Found 11059 points in 3.3836979866s
2019-01-21 03:03:49,712 DEBUG: Found 11519 points in 3.54604816437s
Traceback (most recent call last):
  File "/home/useruser/ODM/SuperBuild/src/opensfm/bin/opensfm", line 34, in <module>
    command.run(args)
  File "/home/useruser/ODM/SuperBuild/src/opensfm/opensfm/commands/detect_features.py", line 30, in run
    parallel_map(detect, arguments, processes)
  File "/home/useruser/ODM/SuperBuild/src/opensfm/opensfm/context.py", line 38, in parallel_map
    return list(e.map(func, args))
  File "/usr/local/lib/python2.7/dist-packages/loky/process_executor.py", line 794, in _chain_from_iterable_of_lists
    for element in iterable:
  File "/usr/local/lib/python2.7/dist-packages/loky/_base.py", line 589, in result_iterator
    yield future.result()
  File "/usr/local/lib/python2.7/dist-packages/loky/_base.py", line 433, in result
    return self.__get_result()
  File "/usr/local/lib/python2.7/dist-packages/loky/_base.py", line 381, in __get_result
    raise self._exception
TypeError: expected str, bytes or os.PathLike object, not unicode

This was caused directly by
"""
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/loky/process_executor.py", line 418, in _process_worker
    r = call_item()
  File "/usr/local/lib/python2.7/dist-packages/loky/process_executor.py", line 272, in __call__
    return self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python2.7/dist-packages/loky/process_executor.py", line 337, in _process_chunk
    return [fn(*args) for args in chunk]
  File "/home/useruser/ODM/SuperBuild/src/opensfm/opensfm/commands/detect_features.py", line 78, in detect
    data.save_features(image, p_sorted, f_sorted, c_sorted)
  File "/home/useruser/ODM/SuperBuild/src/opensfm/opensfm/dataset.py", line 422, in save_features
    self._save_features(self._feature_file(image), image, points, descriptors, colors)
  File "/home/useruser/ODM/SuperBuild/src/opensfm/opensfm/dataset.py", line 407, in _save_features
    colors=colors)
  File "/usr/local/lib/python2.7/dist-packages/numpy/lib/npyio.py", line 691, in savez_compressed
    _savez(file, args, kwds, True)
  File "/usr/local/lib/python2.7/dist-packages/numpy/lib/npyio.py", line 700, in _savez
    file = os_fspath(file)
  File "/usr/local/lib/python2.7/dist-packages/numpy/compat/py3k.py", line 237, in os_fspath
    "not " + path_type.__name__)
TypeError: expected str, bytes or os.PathLike object, not unicode
"""
Traceback (most recent call last):
  File "run.py", line 47, in <module>
    plasm.execute(niter=1)
  File "/home/useruser/ODM/scripts/run_opensfm.py", line 115, in process
    (context.pyopencv_path, context.opensfm_path, tree.opensfm))
  File "/home/useruser/ODM/opendm/system.py", line 34, in run
    raise Exception("Child returned {}".format(retcode))
Exception: Child returned 1

What should be the expected behavior? If this is a feature request, please describe in detail the changes you think should be made to the code, citing files and lines where changes should be made, if possible.

Well, I'd like to get on to matching, SfM, MVS, mesh, ortho, etc... . :)

How can we reproduce this? (What steps did you do to trigger the problem? What parameters are you using for processing? If possible please include a copy of your dataset uploaded on Google Drive or Dropbox. Be detailed)

Replicable with https://github.com/OpenDroneMap/odm_data_aukerman run on a fresh install on Ubuntu 16.04. @dakotabenjamin has seen this too. I'm stumped, and last I heard, he was too.

python run.py odm_data_aukerman

pierotofy commented 5 years ago

Ah so I'm not crazy :) I noticed it too yesterday. I think this is due to some package that has been updated either on apt or pip. Although the error is different here than what was originally posted.

smathermather commented 5 years ago

Should we backport any fixes to 0.4 branch at least as well as backporting https://github.com/OpenDroneMap/ODM/pull/927?

FYI, running apt updates on a machine where it's already installed doesn't seem to cause this, just new installs.

vgorte commented 5 years ago

@timmycam No. I didn't find a solution to this bug. It worked for me when I removed the specifications for the cpu's and the memory and after that it worked when I started the process via

docker run --cpus=16 --memory=30000mb -ti --rm -v $(pwd):/datasets/code opendronemap/opendronemap --project-path /datasets --time --verbose --min-num-features 2000

I have no idea why tho... Also, I have not worked with ODM since because I only needed it at that time for my bachelors thesis where I compared ODM and Pix4D results for research. Therefore I am not sure if my initial problem hasn't already been fixed and the now occuring error is caused by something else. :)

pierotofy commented 5 years ago

Yeah it's a different bug. @smathermather it's probably a pip package.

timmycam commented 5 years ago

Thanks @vgorte, all good it's been picked up as bug, as the guys have discussed :)

pierotofy commented 5 years ago

I think it's numpy (I've opened a fix upstream), just not sure if we should forcibly patch it for the time being. https://github.com/numpy/numpy/pull/12819

pierotofy commented 5 years ago

Yeah it's numpy. https://github.com/numpy/numpy/pull/12754

timmycam commented 5 years ago

Ahh it is too, cheers for that!

effigies commented 5 years ago

On our end (nipy/nipype), we just add a !=1.16.0 constraint to numpy (banking on 0.16.1 including numpy/numpy#12754).

pierotofy commented 5 years ago

Fixed.