ReadMe

This is the repository for the Remote Sensing of Environment article: Deep Point Cloud Regression for Above-Ground Forest Biomass Estimation from Airborne LiDAR.

When cloning the repository, make sure to also get submodules:

git clone --recurse-submodules https://github.com/StefOe/DPCR-AGB.git

We include code, evaluation scripts, model weights (soon), and the dataset.

Regarding the code: We forked the torch-points3d framework and added support for regression tasks including datasets, tracking, and models on our own. In the process, we also simplified the usage of package.

In addition, we also included our code to load the trained linear regression and random forest in the pointcloud_stats_method folder. Just run the notebook learn_with_stats.ipynb.

Finally, the results/plots for each method can be seen in the eval_scripts folder within the eval_deep_learning_v2.ipynb. The results for the network size experiment are in eval_deep_learning_v2_size.ipynb.

results on the test set:

Install torch-points3d

We setup our environment in the following way (conda is already installed):

1. go to pointcloud-biomass-estimator/torch-points3d
2. Make sure to install cuda 11.8 (don't forget to deselect the driver install if your drivers are current)

wget https://developer.download.nvidia.com/compute/cuda/11.8.0/local_installers/cuda_11.8.0_520.61.05_linux.run
sudo sh cuda_11.8.0_520.61.05_linux.run

3. after installing close and reopen the terminal to check if the PATH is set correctly with echo $PATH. It should not have /usr/local/cuda-10.2 but should have something like /usr/local/cuda-11.8 in there

4. install mamba (optional but highly recommended)

conda install mamba -c conda-forge

3. create conda environment:

mamba env create -f env.yml

or for cpu-version:

mamba env create -f env_cpu.yml

4. activate environment:

mamba activate pts

5. install missing pip packages for Minkowski networks

pip install -U git+https://github.com/NVIDIA/MinkowskiEngine -v --no-deps --config-settings blas_include_dirs=${CONDA_PREFIX}/include blas=openblas

or for cpu-version:

pip install -U git+https://github.com/NVIDIA/MinkowskiEngine -v --no-deps --config-settings blas=openblas

5. compile KPConv scripts

sh compile_wrappers.sh

Get the Data

to get the preprocessed lidar and nfi data, go to the torch-points3d folder (cd torch-points3d) and download:

wget https://sid.erda.dk/share_redirect/bB1TBPTsEk
mv bB1TBPTsEk nfi_preprocessed_data.zip
unzip nfi_preprocessed_data.zip

the data should now be in data/biomass/processed_nfi_reg/ given the root folder is torch-points3d.

Training for Regression

run from within the torch-points3d folder.

MSENet50:

python -u train.py task=instance models=instance/minkowski_baseline model_name=SENet50 data=instance/NFI/reg data.transform_type=sparse_xy training=nfi/minkowski lr_scheduler=cosineawr update_lr_scheduler_on=on_num_batch

MSENet14:

python -u train.py task=instance models=instance/minkowski_baseline model_name=SENet14 data=instance/NFI/reg data.transform_type=sparse_xy training=nfi/minkowski lr_scheduler=cosineawr update_lr_scheduler_on=on_num_batch

KPConv:

python -u train.py task=instance models=instance/kpconv model_name=KPConv data=instance/NFI/reg training=nfi/kpconv data.transform_type=xy lr_scheduler=cosineawr update_lr_scheduler_on=on_num_batch

PointNet:

python -u train.py task=instance models=instance/minkowski_baseline model_name=MPointNet data=instance/NFI/reg training=nfi/pointnet data.transform_type=sparse_xy lr_scheduler=cosineawr update_lr_scheduler_on=on_num_batch

Calibration batch normalization

to calibrate the trained models batch norm statistics. Note that the checkpoint directory has to be an absolute path, e.g.: checkpoint_dir=/home/user/torch-points3d/weights/SENet50/0

for Minkowski or Pointnet (model_name=SENet50, model_name=SENet14, or model_name=MPointNet):

python calibrate_bn.py model_name=${model_name} checkpoint_dir=${checkpoint_dir} data=instance/NFI/reg num_workers=4 task=instance weight_name="total_BMag_ha_rmse" batch_size=64 num_workers=4 data.transform_type=sparse_xy epochs=20

for KPConv:

python calibrate_bn.py model_name=KPConv checkpoint_dir=${checkpoint_dir} data=instance/NFI/reg num_workers=4 task=instance weight_name="total_BMag_ha_rmse" batch_size=64 num_workers=4 data.transform_type=xy epochs=20

Evaluating our models

run from within the torch-points3d folder. Note that the checkpoint directory has to be an absolute path, e.g.: PATHTOFRAMEWORK=/home/user/torch-points3d Also, there are 5 weights for each model (from different trials): TRIAL=1

MSENet50:

python eval.py model_name=SENet50 checkpoint_dir=${PATHTOFRAMEWORK}/weights/SENet50/${TRIAL}/ weight_name="latest" batch_size=32 num_workers=4 eval_stages=["train","val","test"] data.transform_type=sparse_xy_eval data=instance/NFI/reg task=instance

the save folder location is weights/msenet50/eval.

MSENet14:

python eval.py model_name=SENet14 checkpoint_dir=${PATHTOFRAMEWORK}/weights/SENet14/${TRIAL}/ weight_name="latest" batch_size=32 num_workers=4 eval_stages=["train","val","test"] data.transform_type=sparse_xy_eval data=instance/NFI/reg task=instance

the save folder location is weights/msenet14/eval.

KPConv:

python eval.py model_name=KPConv checkpoint_dir=${PATHTOFRAMEWORK}/weights/KPConv/${TRIAL}/ weight_name="latest" batch_size=32 num_workers=4 eval_stages=["train","val","test"] data.transform_type=xy_eval data=instance/NFI/reg task=instance

the save folder location is weights/kpconv/eval.

PointNet:

python eval.py model_name=MPointNet checkpoint_dir=${PATHTOFRAMEWORK}/weights/PointNet/${TRIAL}/ weight_name="latest" batch_size=32 num_workers=4 eval_stages=["train","val","test"] data.transform_type=sparse_xy_eval data=instance/NFI/reg task=instance

the save folder location is weights/pointnet/eval.

Using tree-adding augmentations during test

same as before, but the transform type changes to use tree augmentations, e.g.:

python eval.py model_name=MPointNet checkpoint_dir=${PATHTOFRAMEWORK}/weights/pointnet/ weight_name="total_rmse" batch_size=32 num_workers=4 eval_stages=["train","val","test"] data.transform_type=sparse_xy_eval_treeadd