Combine las files during ply2las

[dlebauer]

Is there a reason to keep the east and west las files separate, or can these be merged into a single file?

Completion Criteria

• [x] decide if it is reasonable to combine each pair of point clouds into a single file
• if yes
  • [x] implement
  • [x] deploy
• if no
  • [ ] explain and close issue

[solmazhajmohammadi]

I dont see a reason to keep east and west files, as long as we have ply files we should be OK. The routine to combine the pair is listed in issue https://github.com/terraref/computing-pipeline/issues/98 , under suggestion section.

[ZongyangLi]

Totally agree! I would prefer a single file for each scan.

Is there anywhere i can find such an example of combined las file?

[ghost]

@max-zilla plz help

[dlebauer]

Or @ZongyangLi could you update the extractor? I know that there is a lasmerge function in lastools.

[solmazhajmohammadi]

Following command will do the merge docker run -v /Path/ToThe/File/data:/data -t pdal/pdal pdal merge /data/InputEast.las /data/InputWest.las /data/output/MergedOutput.las

[max-zilla]

I'll add this command and redeploy this afternoon.

[max-zilla]

This will take a slight tweak to extractor logic, but I don't think it's a big deal:

• right now, it triggers when .ply file is added to clowder and converts it to .las
• as a result, extractor is only aware of one single .ply file at a time (the one that triggered it)

changes:

• instead, we need this to operate on a DATASET level because it needs to see both east and west PLY - dataset extractors are aware of all files in a dataset - it'll get triggered whenever a file is added to a dataset
• in code, we check to see if we have both east/west .ply files in dataset
• if so, convert both to LAS and merge
• if not, we skip. If we only have one ply file, it would be too messy to convert it to LAS and then later convert the other to LAS and combine them and remove the first LAS from Clowder and upload the merged one etc etc.. we should generally always upload both east/west halves at the same time anyway.

this is all trivial boilerplate code but I just wanted to make the change known.

[max-zilla]

https://github.com/terraref/extractors-3dscanner/pull/2/files FYI.

[dlebauer]

alternatively, would it make sense to have a separate extractor that combines two las files and deletes the originals?

On Wed, Jan 25, 2017 at 12:08 PM Max Burnette notifications@github.com wrote:

https://github.com/terraref/extractors-3dscanner/pull/2/files FYI.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/terraref/computing-pipeline/issues/237#issuecomment-275185742, or mute the thread https://github.com/notifications/unsubscribe-auth/AAcX5yuI7WcLRrDeOuEq5LF0qvAeUF4wks5rV4-HgaJpZM4LplFu .

[max-zilla]

@dlebauer I considered that, but my argument would be that is unnecessary extra complexity to write a whole separate extractor. We don't currently have PLY->LAS conversions we want to do WITHOUT merge, and it didn't seem like having a _east LAS file while the _west LAS file (& merge) are still pending would be very valuable alone?

Either way we arrive at the same destination, but this way we handle both steps at once and we don't have to worry about deleting files from Roger/Clowder (which is not something that really fits into other extractor workflows anyway - we've been very wary of deletion)

FWIW I've already added the tweaks to this extractor to accomplish this and am going to test soon. if you'd rather have 2 separate stages we can still do so in the near future.

[dlebauer]

Up to you. I wasn't sure if that would make it easier to do post processing rather than re-generating all of the files. e.g. when we update the in-file metadata we won't want to re-compute (especially if/when we do so for hyperspectral data).

[ghost]

Max will keep the East and West files AND create a merged file. What outputs are needed?

[max-zilla]

@ZongyangLi @solmazhajmohammadi 3 options:

we start with 2 east/west PLY files

option 1 - 2 PLY, 1 LAS (this is current version) keep 2 east/west PLY files create 1 merged LAS file

option 2 - 2 PLY, 3 LAS keep 2 east/west PLY files create east/west LAS files create 1 merged LAS file

option 3 - 3 PLY, 1 LAS keep 2 east/west PLY files create 1 merged PLY file create 1 merged LAS file

@ZongyangLi is option 1 good for you, or do you need 2 or 3?

[ZongyangLi]

@max-zilla option 1 works for me. It's just @dlebauer comment that 'deletes the originals', and I think it will be better to keep 2 existed ply files. Sorry for the misunderstanding, @dlebauer was saying deletes 2 las files.

[solmazhajmohammadi]

Option 1 works for me as well, I merge the ply files in my extractor.

[ghost]

@ZongyangLi and @solmazhajmohammadi can this issue be closed?

[dlebauer]

Last I checked these are still generated separately On Thu, Feb 23, 2017 at 10:42 AM Rachel Shekar notifications@github.com wrote:

@ZongyangLi https://github.com/ZongyangLi and @solmazhajmohammadi https://github.com/solmazhajmohammadi can this issue be closed?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/terraref/computing-pipeline/issues/237#issuecomment-282047826, or mute the thread https://github.com/notifications/unsubscribe-auth/AAcX5wabkjFj2FaoDy1O-2M73d0xXyRkks5rfbb-gaJpZM4LplFu .

[max-zilla]

I have added code to accomplish this, but haven't yet back-processed old data to generate merged files. It should occur on new files, however - will close this and include a note to reprocess in #221.