search

brycefrank / pyfor

Tools for analyzing aerial point clouds of forest data.
MIT License
93 stars 19 forks source link

Single Photon Lidar data #47

Closed jfprieur closed 5 years ago

jfprieur commented 5 years ago

Hello,

First of all as a fellow PhD student working in forestry, thank you so much for your efforts on this package. I work on tree species identification at the individual tree level and have developped a bunch of python scripts to calculate 3D and intensity features from ALS data. Your package is exactly what we need to make our process as Python-centric as possible.

I am working with Leica SPL100 data (in LAS 1.4 format which I know causes problems in some open source implementations, since you are using laspy I believe this is not the problem), the concept of a scan angle does not exist for this type of sensor. When I try to load a LAS file with this data, I get:

LaspyException: Dimension: scan_angle_rank not found.

Instead of an exception would it be possible to just ignore it?

Once again many thanks for your work on this, much appreciated!

bw4sz commented 5 years ago

I'm greedily jumping into this thread (even though I've got nothing to offer on the issue) and saying @jfprieur if you want to talk individual tree detection/identification, give me a ring. This is me: https://www.biorxiv.org/content/10.1101/532952v1, putting bryce's great work into use. Always looking for ways to add more lidar into the deep learning.

brycefrank commented 5 years ago

@jfprieur

Thank you for the kind words!

Have you tried loading the file with only laspy? i.e.:

import laspy

laspy.file.File('your_spl100_file.las')

If the error still occurs, I suggest taking it to the laspy people.

jfprieur commented 5 years ago

Yes, importing with laspy works.

I have been using geopandas and laspy to work with lidar for the last 2 years, learning from scratch as I had little programming experience before starting my graduate studies. Here is how I go from LAS to geopandas dataframe (a few minor differences now, I remove the scan_angle_rank obviously for SPL data). Your package did not exist back then unfortunately/fortunately for my learning experience!

https://gis.stackexchange.com/questions/230574/inserting-lidar-points-from-laspy-in-geodataframe-without-using-a-numpy-array

I then use geopandas sjoin with a shapefile of tree crowns to isolate specific trees to calculate 3D and intensity features on. I am interested to compare your clipping process to this in terms of speed eventually.

brycefrank commented 5 years ago

@jfprieur

Well if it works in laspy then it is definitely my problem to fix!

If possible, it would help if I had a small sample file to try, you can send it here and I will take a look tomorrow:

bfrank70@gmail.com

Also, did you attend Silvilaser in 2017? I think I remember the name.

Best, Bryce

jfprieur commented 5 years ago

Yes, I did. Your name is familiar as well, small world ours!

I will send you the sample data ASAP

brycefrank commented 5 years ago

Great.

As for the clipping, it should be reasonably performant. I take advantage of an interesting package that compiles simple functions to machine code called numba. You can see the implementation here:

https://github.com/brycefrank/pyfor/blob/01624f7a982c736ecbb91cd02a77ec3234f788d8/pyfor/clip.py#L27-L57

It was a life saver for the package early on, and one of the major hurdles before I was convinced I could write a Python only processing package (without having to dip into C++, for example). I found the source code somewhere on the GIS stack exchange, but I can no longer remember where. If sjoin can beat it, then by all means I can replace.

brycefrank commented 5 years ago

Also, good to see Howard Butler on that thread, always a good resource 👍

brycefrank commented 5 years ago

@jfprieur

I generalized the point reading a bit in the following:

https://github.com/brycefrank/pyfor/blob/144491872349ee3f49cd62083628a647bd905c30/pyfor/cloud.py#L93-L108

I do not have samples of all las specification formats in the testing suite yet, so I expect someone to run into an error (for example, I had to out-manuever laspy for the classifcation renaming they do, it is clunky but working on your tile and my test tile).

This is a bit more generalizable than my hard-coded solution from before.

brycefrank commented 5 years ago

Unfortunately I ran into a rather deceptive bug with this yesterday, I will revert back to something more stable (a bit messy) but should still fix this particular issue.

brycefrank commented 5 years ago

I have implemented a stop-gap change that should: a) fix this issue and b) avoid the bug from last time.

It is certainly messy, but until I have the time to dig into laspy internals a bit more, it will have to do.