OpenDroneMap / ODMSemantic3D

An open photogrammetry dataset of classified 3D point clouds for automated semantic segmentation. CC BY-SA 4.0
Other
15 stars 7 forks source link

Add toledo dataset #7

Closed HeDo88TH closed 1 year ago

HeDo88TH commented 1 year ago

First contribution!

HeDo88TH commented 1 year ago

Test comment

u4gbot commented 1 year ago
Comparison for brighton_beach_small (Differences) Accuracy: 94.11% vs 96.17% | Label | Accuracy | Recall | Precision | F1 | | -------------------- | ---------- | ---------- | ---------- | ---------- | | | 14.64% | 99.56% | 14.65% | 0.26 | | building | 19.54% | 41.52% | 26.96% | 0.33 | | | 33.48% | -58.29% | 84.04% | 28.01% | | | | | | | | | 66.95% | 95.07% | 69.36% | 0.80 | | low_vegetation | 33.01% | 41.60% | 61.54% | 0.50 | | | -50.69% | -56.25% | -11.28% | -38.11% | | | | | | | | | 45.35% | 83.98% | 49.64% | 0.62 | | human_made_object | 40.96% | 41.06% | 99.39% | 0.58 | | | -9.68% | -51.11% | 100.23% | -6.87% | | | | | | | | | 94.06% | 94.12% | 99.93% | 0.97 | | ground | 96.46% | 98.74% | 97.66% | 0.98 | | | 2.55% | 4.91% | -2.27% | 1.30% | | | | | | |
Comparison for sheffield_park_small (Differences) Accuracy: 97.85% vs 30.29% | Label | Accuracy | Recall | Precision | F1 | | -------------------- | ---------- | ---------- | ---------- | ---------- | | | 92.46% | 94.00% | 98.27% | 0.96 | | building | 51.58% | 56.93% | 84.58% | 0.68 | | | -44.22% | -39.43% | -13.93% | -29.17% | | | | | | | | | 95.84% | 98.10% | 97.66% | 0.98 | | low_vegetation | 19.81% | 99.95% | 19.81% | 0.33 | | | -79.34% | 1.89% | -79.72% | -66.22% | | | | | | | | | 34.76% | 94.35% | 35.50% | 0.52 | | human_made_object | 3.44% | 3.76% | 28.78% | 0.07 | | | -90.10% | -96.01% | -18.94% | -87.10% | | | | | | | | | 98.04% | 98.33% | 99.70% | 0.99 | | ground | 11.10% | 11.30% | 86.39% | 0.20 | | | -88.67% | -88.50% | -13.36% | -79.81% | | | | | | |

Average (2 datasets)

Accuracy: 95.98% vs 63.23%

Label Accuracy Recall Precision F1
53.55% 96.78% 56.46% 0.61
building 35.56% 49.23% 55.77% 0.50
-5.37% -48.86% 35.05% -0.58%
81.40% 96.58% 83.51% 0.89
low_vegetation 26.41% 70.77% 40.67% 0.41
-65.01% -27.18% -45.50% -52.17%
40.06% 89.17% 42.57% 0.57
human_made_object 22.20% 22.41% 64.09% 0.32
-49.89% -73.56% 40.64% -46.99%
96.05% 96.22% 99.82% 0.98
ground 53.78% 55.02% 92.03% 0.59
-43.06% -41.80% -7.81% -39.26%
pierotofy commented 1 year ago

The human_made_object class should be 64, but during export to LAZ 1.2 it was truncated to 0 ? That might partially explain the low performance numbers.

Never mind, I just opened the file with a broken version of cloudcompare. Duh.

u4gbot commented 1 year ago
Comparison for brighton_beach_small (Differences) Accuracy: 94.11% vs 95.35% | Label | Accuracy | Recall | Precision | F1 | | -------------------- | ---------- | ---------- | ---------- | ---------- | | | 14.64% | 99.56% | 14.65% | 0.26 | | building | 21.24% | 67.19% | 23.70% | 0.35 | | | 6.60% | -32.36% | 9.05% | 0.09 | | | | | | | | | 94.06% | 94.12% | 99.93% | 0.97 | | ground | 95.58% | 97.49% | 98.00% | 0.98 | | | 1.52% | 3.37% | -1.94% | 0.01 | | | | | | | | | 66.95% | 95.07% | 69.36% | 0.80 | | low_vegetation | 34.40% | 48.58% | 54.09% | 0.51 | | | -32.55% | -46.49% | -15.27% | -0.29 | | | | | | | | | 45.35% | 83.98% | 49.64% | 0.62 | | human_made_object | 37.91% | 37.91% | 100.00% | 0.55 | | | -7.44% | -46.08% | 50.36% | -0.07 | | | | | | |
Comparison for sheffield_park_small (Differences) Accuracy: 97.85% vs 28.69% | Label | Accuracy | Recall | Precision | F1 | | -------------------- | ---------- | ---------- | ---------- | ---------- | | | 92.46% | 94.00% | 98.27% | 0.96 | | building | 52.07% | 61.41% | 77.38% | 0.68 | | | -40.39% | -32.58% | -20.88% | -0.28 | | | | | | | | | 98.04% | 98.33% | 99.70% | 0.99 | | ground | 8.48% | 8.57% | 89.54% | 0.16 | | | -89.55% | -89.76% | -10.17% | -0.83 | | | | | | | | | 95.84% | 98.10% | 97.66% | 0.98 | | low_vegetation | 19.46% | 99.98% | 19.46% | 0.33 | | | -76.38% | 1.89% | -78.20% | -0.65 | | | | | | | | | 34.76% | 94.35% | 35.50% | 0.52 | | human_made_object | 0.33% | 0.33% | 23.97% | 0.01 | | | -34.44% | -94.02% | -11.53% | -0.51 | | | | | | |

Average (2 datasets)

Accuracy: 95.98% vs 62.02%

Label Accuracy Recall Precision F1
53.55% 96.78% 56.46% 0.61
building 36.65% 64.30% 50.54% 0.52
-16.90% -32.47% -5.92% -0.09
96.05% 96.22% 99.82% 0.98
ground 52.03% 53.03% 93.77% 0.57
-44.02% -43.19% -6.05% -0.41
81.40% 96.58% 83.51% 0.89
low_vegetation 26.93% 74.28% 36.77% 0.42
-54.47% -22.30% -46.74% -0.47
40.06% 89.17% 42.57% 0.57
human_made_object 19.12% 19.12% 61.99% 0.28
-20.94% -70.05% 19.42% -0.29
HeDo88TH commented 1 year ago

!skip