Some questions

[Deanoh90]

Thank you for your impressive work.

I would like to ask some questions about your paper and code.

1.

Using "data_generator.py", we can generate reflectivity data and i guess that code is based on Eq. (4) from your paper. To utilize your code, i think the "eta_fit_grass.npy" file should be provided. How do we generate "eta_fit_grass.npy"? Also, is it effective to use grass-related data for a whole LiDAR point cloud which is mixed with different classes?

2.

Furthermore, in "data_generator.py", ins[index] = ins[index]/p(rang[index])*1.8 is used and i'm wondering why "1.8" is used here.

3.

To train alpha, we need to use "train_alpha.py" and 4 missing .npy files ("train_data_dotv2_test, train_GT_dotv2_test, val_data_dotv2_test, val_GT_dotv2_test") are required. Would you like to share those files? And is it possible to let me know how we generate those files just in case of a custom dataset?

Eager to hear from you soon.

Regards, Inyoung Oh.

[kasiv008]

Thank you for the great questions.

1

As pointed out the fitting coefficients were missing, thank you and the same have been added in the utility directory. As stated in the paper, the eta function is estimated by removing the reflectivity (class: grass), range and angle of incidence dependence in the near range effect region. As the grass dependent reflectivity factor is removed, the estimated eta function is now generalized across the point cloud. In addition we did the same across all classes and the obtained very close estimates.

2.

The 1.8 factor was multiplied to process the Velodyne dataset as there was an offset in the near region between Velodyne and Ouster dataset. The same is not required for Rellis-3D ouster dataset.

3.

The angle of incidence predictor is from our previous work (https://github.com/MOONLABIISERB/lidar-intensity-predictor). The repository consists of all the required utilities for training. We will soon update this repo for custom dataset training as well.

Thanks!

[Deanoh90]

Thank you for your answer and i'd like to ask few more questions.

1

As you said that removing the reflectivity of class:grass in the near range effect region is done by using "eta_fit_grass.npy" and you you did the same across all classes and the obtained very close estimates, i'm a bit confused. So I wonder if this means that class:grass objects are mainly in the near-range region and it is valid to correct the near-range effect only for these objects, or if objects other than class:grass are also in the near-range region, but their near-range parameters are very similar to class:grass, so it is sufficient to use only the Eta function of class:grass. lastly, thank you for sharing eta.npy files.

2

i'm looking forward to seeing the updated repository for training and inferencing the angle of incidence. Meanwhile, i tried to estimate the angle of incidence (alpha) by using "train_alpha.py" and it showed the results as follows. [fig.1. Raw intensity] [fig.2. Raw intensity after removing near-range region] [fig.3.The estimated alpha of the remaining points after removal] In Fig.3., most of the values are clustered around 1.23, and the rest seem to show the same trend as the outlier. What i understood is to generate ground truth alpha is to calculate alpha like arccos(Intensity(R)/MaxIntensity(R)) for "class-wise" (Off-Road LiDAR Intensity Based Semantic Segmentation, Eq.5.) and train it using FCN. Thus, i expected the results to be class-dependent, i.e., to follow a certain distribution with specific values for each class, but the results were unexpected (Most classes tend to be near a certain value, like 1.23), as shown in Fig. 3. I'm sure the upcoming repo will be helpful, but I was wondering if you had any intuition about the above thoughts and results.

3

Last question, there is the equation of reflectivity coefficients in Eq. (3). and (4). Especially, Eq. (4) can be derived by using cos(Alpha), I_c(Rho), R^2, and I_c(R, Alpha, Rho). Here, R and cos(Alpha) are easily calculated, and I_c(Rho) can be calculated by a centroid I_c(Rho) for each class. But, I'm not sure what I_c(R, Alpha, Rho) means and how it can be calculated.

Sorry to reply late.

Best regards, Inyoung Oh.

[kasiv008]

In intensity, the parameter that defines an object(classes like grass, tree etc.) dependence is the reflectivity factor. To estimate the eta function we remove this dependence by dividing the intensity values in the near range by the reflectivity parameter outside the near-range effect (The reflectivity parameter doesn't change with range. By doing this for class 'grass', we ideally generalize it across all classes. Moreover we have taken the class "Grass" because of its major share in the near range points as you pointed!!

2

What is your input into the alpha predictor? It should be the estimated normals (open3d) of the point cloud. l can also look into the data if you can send it across.

3 I_c(R, alpha, rho) is the raw intensity within near range effect, whereas I_c(rho) is the calibrated intensity(reflectivity) outside the near-range effect. The elements in the brackets are the dependencies/function variables.

[Deanoh90]

Regarding the question No.2, 00000/os1_cloud_node_color_ply/frame002269-1581624879_663.ply is used. The input into the alpha predictor is estimated normals and the output of this prediction is angles. (C1. https://github.com/MOONLABIISERB/lidar-intensity-predictor/blob/main/intensity_analyser.py, line 66) (C2. https://github.com/unmannedlab/LiDAR-reflectivity-segmentation/blob/master/utlis/data_generator.py, line 82)

The output graph was generated by using the C1. For now, I will also visualize the estimated normal vector used as input, but my first thought was that it was strange that the estimated normal vectors on the Gaussian sphere and the distribution of the output alpha seemed unrelated. Obviously, the normal values estimated from bush, tree, etc. would be output as diverging due to the sparse distribution of the nearest points rather than converging.

[kasiv008]

Theoretically Alpha is estimated from: alpha = acos(normal . (x,y,z)/||x,y,x||). Due to the error in normal estimates, this computation gives erroneous results. Hence we came up with the FCN to learn the relation. One could use physics informed neural networks(PINNs) to get beter fitting but currently out of our scope.

[Deanoh90]

Always thank you for your quick response. I'll take a look at the PINN and this might be my last question regarding the generation of GT alpha. Let's assume that we have three classes in a point cloud. The intensity of each class is as follows. class 1: 0.34, 0.67, 0.89, 0.95 class 2: 0.05, 0.02, 0.08, 0.8 class 3: 0.53, 0.43, 0.99, 0.01

Accoding to the "Off-Road LiDAR Intensity Based Semantic Segmentation", The ground truth alpha for the each class can be calculated by Eq. (5) of the paper and that is alpha=arccos(Intensity(R)/MaxIntensity(R)).

Therefore, the calculated GT alpha of each class is class 1: 0.34/0.95, 0.67/0.95, 0.89/0.95, 0.95/0.95 class 2: 0.05/0.8, 0.02/0.8, 0.08/0.8, 0.8/0.8 class 3: 0.53/0.99, 0.43/0.99, 0.99/0.99, 0.01/0.99. Is it correct? Then, the generated GT alpha is used to train the FCN using estimated surface normals. right?

Sincerely, Inyoung Oh.

[kasiv008]

Equation 5 represents: Let for class A, MaxIntensity(R) represents the maximum intensity at range(R) of class A (figure 2b in the offroad lidar intensity paper). Intensity(R) represents the any intensity at given range(R) of class A. The idea is that maximum intensity at a particular range is obtained when cos(alpha) = 1 given the equation: maxIntensity(R,rho,alpha) = I(rho)xcos(alpha)/R^2 = I(rho)/R^2. For any other alpha at that particular range R for class A: cos(alpha) = Intensity(R,rho,alpha)/maxIntensity(r,rho,alpha) at every range. I think you are missing "for corresponding range" as this computation can be used only when the range is same

[Deanoh90]

If i have a further question, i'll reopen issue.

Thank you for everything!