Area-based metrics. New deep learning tools.

[bw4sz]

Using this issue to do a bit of introductions. @brycefrank , meet @Eayrey. Elias just published an awesome example of deep learning for area-based metrics. Bryce is working on a lidar forest python module. I've been pushing my friends at the Forest service to explore very similar ideas. Paper attached. remotesensing-10-00649.pdf

If there is a trained model available (?) from

https://github.com/Eayrey/3D-Convolutional-Neural-Networks-with-LiDAR

it might be fairly straightforward to pass a PointCloud method to the trained model and get biomass/treecount predictions. Just an idea.

[brycefrank]

Thanks for the tip!

There are a few lower level issues I need to address before the ABA stuff can be dealt with properly, most of which has to do with creating a framework to access information across multiple tiles with plot shape files. This will likely take some time as I have a few other projects on the rise.

Once that is in place it would should be straightforward like you say (passing the Cloud object to another script). The place I see pyfor sitting nicely is reducing the larger las/z acquisition to information the analyst needs (be it segments, grid metrics, smaller point clouds, etc) that could then be passed to something like Elias has written.

I also see he implements voxelization, which is also on my to-do list (should just be another call to np.digitize in the Grid.__init__ function.

That said, it could make a nice sample for the wiki :+1:

[eayrey]

Hi Bryce,

Nice to meet you! Looks like you're doing some interesting work making LiDAR more accessible. I'll have to poke around your repositories on Monday.

Voxelization is actually something I'm struggling with at the moment. I'm using np.histogramdd but that's not so efficient at a landscape scale. I'm also looking for an efficient normalization method in python, I'll have to give yours a try.

I do have trained neural networks in that repository, and I have a bunch more which are doing better than the models in my paper. I wouldn't trust them to make accurate predictions outside of the Northeast though. But their weights could be used to retrain models elsewhere with a fraction of the effort. Let me know if you're looking for help with any of it.

-Elias

On Sat, Apr 28, 2018 at 6:17 PM, Bryce Frank notifications@github.com wrote:

Thanks for the tip!

There are a few lower level issues I need to address before the ABA stuff can be dealt with properly, most of which has to do with creating a framework to access information across multiple tiles with plot shape files. This will likely take some time as I have a few other projects on the rise.

Once that is in place it would should be straightforward like you say (passing the Cloud object to another script). The place I see pyfor sitting nicely is reducing the larger las/z acquisition to information the analyst needs (be it segments, grid metrics, smaller point clouds, etc) that could then be passed to something like Elias has written.

I also see he implements voxelization, which is also on my to-do list (should just be another call to np.digitize in the Grid.init function.

That said, it could make a nice sample for the wiki 👍

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/brycefrank/pyfor/issues/8#issuecomment-385209184, or mute the thread https://github.com/notifications/unsubscribe-auth/Aa8_rKjNdDOaQgYAsq8bAJceTVVPDTtYks5ttOqHgaJpZM4TrkKy .

[brycefrank]

Feel free to try the one in the package, I have a sample written up at the following:

https://github.com/brycefrank/pyfor/blob/master/samples/Normalization.ipynb

I have had a few issues using it at small scales, but large scales seem to perform as expected. I have been hoping someone would give it a go because it does need some tweaking still! On top of this, I have wanted to add other normalization options other than the Zhang filter (i.e. Hudak's MCC). I can't remember the specifics, but I was able to normalize about 8 tiles in a reasonable amount of time on a 12 core machine (I used joblib to parallelize, it was straightforward). I have yet to compare this to other products, but its another thing to work on.

If we are speaking the same thing, voxelization as a regular 3 dimensional grid, then I think the process I use for binning the X and Y dimensions is the fastest I have found:

https://github.com/brycefrank/pyfor/blob/33ef2fb487c6df2b78af1607921570fa2bf2f0b7/pyfor/rasterizer.py#L41-L42

I think that was inspired from a stack overflow post, but I can't remember where it is! My guess is that adding a bins_z call with the correct arguments would get me to voxels. I will try this out this week and see if anything comes out of it. As for speed, this part has been marginal compared to other tasks, like normalization.

Best, B

[brycefrank]

@Eayrey

I added an implementation of a regular voxel grid on the 0.2.1 branch:

https://github.com/brycefrank/pyfor/blob/0.2.1/pyfor/voxelizer.py

I was able to play around with it today and it seemed to work as expected (I even got a few cheesy voxel plots to look at in matplotlib!). The main feature here is probably the VoxelGrid.data dataframe, this assigns each point to its respective voxel cell. In terms of efficiency my small 100m x 100m las file took about 8 milliseconds to initialize. I threw a voxel_raster method in there for kicks as well.

I would be curious if this outperforms np.histogramdd, feel free to take what tidbits you need!

[bw4sz]

I got an email from the developers here

https://github.com/Open-3D/Open3D/releases/

new python wrapper for the c++ code. I am going to look into this today, maybe suggest a fast voxelization tool.

On Sun, Apr 29, 2018 at 8:05 PM, Bryce Frank notifications@github.com wrote:

@Eayrey https://github.com/Eayrey

I added an implementation of a regular voxel grid on the 0.2.1 branch:

https://github.com/brycefrank/pyfor/blob/0.2.1/pyfor/voxelizer.py

I was able to play around with it today and it seemed to work as expected (I even got a few cheesey voxel plots to look at in matplotlib!). The main feature here is probably the VoxelGrid.data dataframe, this assigns each point to its respective voxel cell. In terms of efficiency my small 100m x 100m las file took about 8 milliseconds to initialize. I threw a voxel_raster method in there for kicks as well.

I would be curious if this outperforms np.histogramdd, feel free to take what tidbits you need!

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/brycefrank/pyfor/issues/8#issuecomment-385305519, or mute the thread https://github.com/notifications/unsubscribe-auth/ABJwrMUOPIuZ84Vs86JxeZNcd3C1E1Ttks5ttn9tgaJpZM4TrkKy .

-- Ben Weinstein, Ph.D. Postdoctoral Fellow University of Florida http://benweinstein.weebly.com/