[Bug] commands hang when trying to scan/build files with the same filename

connormanning / entwine

Entwine - point cloud organization for massive datasets

https://entwine.io

Other

451 stars 128 forks source link

[Bug] commands hang when trying to scan/build files with the same filename #190

Closed pierotofy closed 5 years ago

pierotofy commented 5 years ago

When trying to use the scan command for two files on the filesystem:

entwine scan -i "/data/model0/input.laz" "/data/model1/input.laz" -o /data/result

Entwine hangs with:

Scanning:
    Input: 2 files
    Threads: 8
    Trust file headers? yes

1/2: /data/model0/input.laz
2/2: /data/model1/input.laz
Determined SRS from an input file

Writing details to /data/result...  Failed PUT attempt 1: /data/result/ept-sources/
    Failed PUT attempt 2: /data/result/ept-sources/
    Failed PUT attempt 3: /data/result/ept-sources/
.....

Using latest master branch.

pierotofy commented 5 years ago

This seems to also affect the build command:

entwine build -i "/data/model0/input.laz" "/data/model1/input.laz" -o /data/ept

pierotofy commented 5 years ago

I suspect this is due to the filename being the same, but in different directories. After renaming one of the two input files, both commands work fine.

pierotofy commented 5 years ago

I think it's triggered by the unique flag not being set at https://github.com/connormanning/entwine/blob/5d9a8786d17a416c29d5afde0253299c49a33a20/entwine/types/files.cpp#L77

connormanning commented 5 years ago

Interesting one, thanks for this investigation. Fix coming shortly.

pierotofy commented 5 years ago

I got it to run properly with the changes in #191, the EPT JSON looks OK from inspection, not sure if I missed something else on the larger picture.

connormanning commented 5 years ago

Looks good - thanks for the patch!

pierotofy commented 5 years ago

Happy to help! Entwine is awesome.