search

OpenDroneMap / ODMSemantic3D

An open photogrammetry dataset of classified 3D point clouds for automated semantic segmentation. CC BY-SA 4.0
Other
15 stars 7 forks source link

Added pierotofy walter point cloud #30

Closed HeDo88TH closed 1 year ago

HeDo88TH commented 1 year ago

As title says!

u4gbot commented 1 year ago
Comparison for brighton_beach_small (Differences) Accuracy: 95.76% vs 3.10% | Label | Accuracy | Recall | Precision | F1 | | -------------------- | ---------- | ---------- | ---------- | ---------- | | | 36.89% | 37.05% | 98.85% | 0.54 | | human_made_object | 0.00% | 0.00% | N/A | N/A | | | -36.89% | -37.05% | N/A | N/A | | | | | | | | | 96.07% | 98.65% | 97.36% | 0.98 | | ground | 0.00% | 0.00% | N/A | N/A | | | -96.07% | -98.65% | N/A | N/A | | | | | | | | | 22.14% | 53.03% | 27.53% | 0.36 | | building | 0.00% | 0.00% | 0.00% | N/A | | | -22.14% | -53.03% | -27.53% | N/A | | | | | | | | | 23.78% | 29.40% | 55.45% | 0.38 | | low_vegetation | 3.10% | 100.00% | 3.10% | 0.06 | | | -20.68% | 70.60% | -52.34% | -0.32 | | | | | | |
Comparison for sheffield_park_small (Differences) Accuracy: 23.63% vs 64.99% | Label | Accuracy | Recall | Precision | F1 | | -------------------- | ---------- | ---------- | ---------- | ---------- | | | 1.43% | 1.50% | 21.78% | 0.03 | | human_made_object | 2.49% | 96.40% | 2.49% | 0.05 | | | 1.07% | 94.90% | -19.29% | 0.02 | | | | | | | | | 0.44% | 0.44% | 86.19% | 0.01 | | ground | 72.72% | 74.57% | 96.69% | 0.84 | | | 72.28% | 74.13% | 10.50% | 0.83 | | | | | | | | | 51.70% | 70.70% | 65.80% | 0.68 | | building | 1.43% | 1.44% | 86.82% | 0.03 | | | -50.27% | -69.26% | 21.02% | -0.65 | | | | | | | | | 18.60% | 99.98% | 18.60% | 0.31 | | low_vegetation | 49.65% | 57.28% | 78.84% | 0.66 | | | 31.05% | -42.70% | 60.24% | 0.35 | | | | | | |

Average (2 datasets)

Accuracy: 59.70% vs 34.04%

Label Accuracy Recall Precision F1
19.16% 19.28% 60.32% 0.28
human_made_object 1.25% 48.20% 1.25% 0.02
-17.91% 28.92% -59.07% -0.26
48.26% 49.54% 91.77% 0.49
ground 36.36% 37.29% 48.35% 0.42
-11.90% -12.26% -43.43% -0.07
36.92% 61.87% 46.67% 0.52
building 0.72% 0.72% 43.41% 0.01
-36.20% -61.15% -3.26% -0.51
21.19% 64.69% 37.02% 0.35
low_vegetation 26.38% 78.64% 40.97% 0.36
5.19% 13.95% 3.95% 0.01
pierotofy commented 1 year ago

So did it improve? Why are these numbers N/A?

image

Can we improve the table to be:

human_made_object Accuracy Recall Precision F1
Before 36.89% 37.05% 98.85% 0.54
After 0.00% 0.00% N/A N/A
Diff -36.89% -37.05% N/A N/A
u4gbot commented 1 year ago
brighton_beach_small Overall accuracy from 95.76% to 3.10% | ground | Accuracy | Recall | Precision | F1 | | -------------------- | -------- | -------- | --------- | -------- | | Before | 96.07% | 98.65% | 97.36% | 0.98 | | After | 0.00% | 0.00% | N/A | N/A | | Diff | -96.07% | -98.65% | N/A | N/A | | low_vegetation | Accuracy | Recall | Precision | F1 | | -------------------- | -------- | -------- | --------- | -------- | | Before | 23.78% | 29.40% | 55.45% | 0.38 | | After | 3.10% | 100.00% | 3.10% | 0.06 | | Diff | -20.68% | 70.60% | -52.34% | -0.32 | | human_made_object | Accuracy | Recall | Precision | F1 | | -------------------- | -------- | -------- | --------- | -------- | | Before | 36.89% | 37.05% | 98.85% | 0.54 | | After | 0.00% | 0.00% | N/A | N/A | | Diff | -36.89% | -37.05% | N/A | N/A | | building | Accuracy | Recall | Precision | F1 | | -------------------- | -------- | -------- | --------- | -------- | | Before | 22.14% | 53.03% | 27.53% | 0.36 | | After | 0.00% | 0.00% | N/A | N/A | | Diff | -22.14% | -53.03% | N/A | N/A |
sheffield_park_small Overall accuracy from 23.63% to 80.43% | ground | Accuracy | Recall | Precision | F1 | | -------------------- | -------- | -------- | --------- | -------- | | Before | 0.44% | 0.44% | 86.19% | 0.01 | | After | 86.51% | 97.93% | 88.12% | 0.93 | | Diff | 86.07% | 97.49% | 1.93% | 0.92 | | low_vegetation | Accuracy | Recall | Precision | F1 | | -------------------- | -------- | -------- | --------- | -------- | | Before | 18.60% | 99.98% | 18.60% | 0.31 | | After | 46.36% | 49.00% | 89.58% | 0.63 | | Diff | 27.76% | -50.97% | 70.98% | 0.32 | | human_made_object | Accuracy | Recall | Precision | F1 | | -------------------- | -------- | -------- | --------- | -------- | | Before | 1.43% | 1.50% | 21.78% | 0.03 | | After | 6.00% | 71.95% | 6.14% | 0.11 | | Diff | 4.57% | 70.45% | -15.64% | 0.09 | | building | Accuracy | Recall | Precision | F1 | | -------------------- | -------- | -------- | --------- | -------- | | Before | 51.70% | 70.70% | 65.80% | 0.68 | | After | 0.79% | 0.79% | 63.08% | 0.02 | | Diff | -50.92% | -69.91% | -2.73% | -0.67 |

Average (2 datasets)

Overall accuracy from 59.70% to 41.77%

ground Accuracy Recall Precision F1
Before 48.26% 49.54% 91.77% 0.49
After 43.25% 48.96% 44.06% 0.46
Diff -5.00% -0.58% -47.71% -0.03
low_vegetation Accuracy Recall Precision F1
Before 21.19% 64.69% 37.02% 0.35
After 24.73% 74.50% 46.34% 0.35
Diff 3.54% 9.82% 9.32% -0.00
human_made_object Accuracy Recall Precision F1
Before 19.16% 19.28% 60.32% 0.28
After 3.00% 35.98% 3.07% 0.06
Diff -16.16% 16.70% -57.25% -0.23
building Accuracy Recall Precision F1
Before 36.92% 61.87% 46.67% 0.52
After 0.39% 0.40% 31.54% 0.01
Diff -36.53% -61.47% -15.13% -0.51
pierotofy commented 1 year ago

@HeDo88TH Can you explain why this is N/A? Is it a bug?

image

u4gbot commented 1 year ago
brighton_beach_small Overall accuracy from 95.76% to 3.53% | human_made_object | Accuracy | Recall | Precision | F1 | | -------------------- | -------- | -------- | --------- | -------- | | Before | 36.89% | 37.05% | 98.85% | 0.54 | | After | 0.00% | 0.00% | N/A | N/A | | Diff | -36.89% | -37.05% | N/A | N/A | | low_vegetation | Accuracy | Recall | Precision | F1 | | -------------------- | -------- | -------- | --------- | -------- | | Before | 23.78% | 29.40% | 55.45% | 0.38 | | After | 6.00% | 95.97% | 6.01% | 0.11 | | Diff | -17.78% | 66.57% | -49.43% | -0.27 | | ground | Accuracy | Recall | Precision | F1 | | -------------------- | -------- | -------- | --------- | -------- | | Before | 96.07% | 98.65% | 97.36% | 0.98 | | After | 0.00% | 0.00% | N/A | N/A | | Diff | -96.07% | -98.65% | N/A | N/A | | building | Accuracy | Recall | Precision | F1 | | -------------------- | -------- | -------- | --------- | -------- | | Before | 22.14% | 53.03% | 27.53% | 0.36 | | After | 1.08% | 80.88% | 1.09% | 0.02 | | Diff | -21.05% | 27.84% | -26.45% | -0.34 |
sheffield_park_small Overall accuracy from 23.63% to 79.37% | human_made_object | Accuracy | Recall | Precision | F1 | | -------------------- | -------- | -------- | --------- | -------- | | Before | 1.43% | 1.50% | 21.78% | 0.03 | | After | 6.43% | 85.99% | 6.50% | 0.12 | | Diff | 5.01% | 84.48% | -15.28% | 0.09 | | low_vegetation | Accuracy | Recall | Precision | F1 | | -------------------- | -------- | -------- | --------- | -------- | | Before | 18.60% | 99.98% | 18.60% | 0.31 | | After | 44.04% | 45.02% | 95.28% | 0.61 | | Diff | 25.43% | -54.96% | 76.68% | 0.30 | | ground | Accuracy | Recall | Precision | F1 | | -------------------- | -------- | -------- | --------- | -------- | | Before | 0.44% | 0.44% | 86.19% | 0.01 | | After | 85.33% | 97.18% | 87.49% | 0.92 | | Diff | 84.89% | 96.74% | 1.30% | 0.91 | | building | Accuracy | Recall | Precision | F1 | | -------------------- | -------- | -------- | --------- | -------- | | Before | 51.70% | 70.70% | 65.80% | 0.68 | | After | 1.10% | 1.11% | 60.30% | 0.02 | | Diff | -50.60% | -69.59% | -5.50% | -0.66 |

Average (2 datasets)

Overall accuracy from 59.70% to 41.45%

human_made_object Accuracy Recall Precision F1
Before 19.16% 19.28% 60.32% 0.28
After 3.22% 42.99% 3.25% 0.06
Diff -15.94% 23.72% -57.07% -0.22
low_vegetation Accuracy Recall Precision F1
Before 21.19% 64.69% 37.02% 0.35
After 25.02% 70.49% 50.65% 0.36
Diff 3.83% 5.81% 13.63% 0.01
ground Accuracy Recall Precision F1
Before 48.26% 49.54% 91.77% 0.49
After 42.66% 48.59% 43.74% 0.46
Diff -5.59% -0.95% -48.03% -0.03
building Accuracy Recall Precision F1
Before 36.92% 61.87% 46.67% 0.52
After 1.09% 40.99% 30.69% 0.02
Diff -35.83% -20.88% -15.98% -0.50