search

OSGeo / grass-addons

GRASS GIS Addons Repository
https://grass.osgeo.org/grass-stable/manuals/addons/
GNU General Public License v2.0
99 stars 150 forks source link

[Bug] r.confusionmatrix testsuite error #831

Open neteler opened 1 year ago

neteler commented 1 year ago

Name of the addon r.confusionmatrix

Describe the bug After the recent updates to r.kappa (https://github.com/OSGeo/grass/pull/2573) the r.confusionmatrix testsuite fails.

To Reproduce

Install the addon and run the testsuite script of it:

GRASS nc_spm_08_grass7/user1:testsuite >

g.extension r.confusionmatrix
cd grass-addons/src/raster/r.confusionmatrix/testsuite/

python test_r_confusionmatrix.py 
['', '', 'Reference Map', 'landclass96']
['', '', 'developed', 'agriculture', 'herbaceous', 'shrubland', 'forest', 'water', 'sediment', '8', '9', '10', '11', '15', '18', '20', '21', 'User Accuracy', 'Commission Error']
['Classified Map', 'High Intensity Developed', 34931, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 100.0, 0.0]
['landuse96_28m', 'Low Intensity Developed', 38146, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0, 100.0]
['', 'Cultivated', 0, 2137, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0, 100.0]
['', 'Managed Herbaceous Cover', 0, 0, 25488, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0, 100.0]
['', 'Upland Herbaceous', 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0, 100.0]
['', 'Riverine/Estuarine Herbaceous', 0, 0, 30, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0, 100.0]
['', 'Evergreen Shrubland', 0, 0, 0, 16464, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0, 100.0]
['', 'Deciduous Shrubland', 0, 0, 0, 320, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0, 100.0]
['', 'Mixed Shrubland', 0, 0, 0, 45, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0, 100.0]
['', 'Mixed Hardwoods', 0, 0, 0, 0, 7893, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0, 100.0]
['', 'Bottomland Hardwoods/Hardwood Swamps', 0, 0, 0, 0, 19733, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0, 100.0]
['', 'Southern Yellow Pine', 0, 0, 0, 0, 64774, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0, 100.0]
['', 'Mixed Hardwoods/Conifers', 0, 0, 0, 0, 33865, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0, 100.0]
['', 'Water Bodies', 0, 0, 0, 0, 0, 5303, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0, 100.0]
['', 'Unconsolidated Sediment', 0, 0, 0, 0, 0, 0, 194, 0, 0, 0, 0, 0, 0, 0, 0, 0.0, 100.0]
['', 'Producer Accuracy', 47.8, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 'Overall Accuracy', 14.01]
['', 'Omission Error', 52.2, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 'Kappa coefficient', 0.09]

Cleaning up...

F['', '', 'Reference Map', 'landclass96_class1only']
['', '', '1', 'User Accuracy', 'Commission Error']
['Classified Map', '1', 73077, 100.0, 0.0]
['landuse96_28m_class1only', '', '', '', '']
['', 'Producer Accuracy', 100.0, 'Overall Accuracy', 100.0]
['', 'Omission Error', 0.0, 'Kappa coefficient', nan]

/home/mneteler/.grass8/addons/scripts/r.confusionmatrix:386: RuntimeWarning: invalid value encountered in double_scalars
  kappa = (overall_accuracy / 100 - pc) / (1 - pc)
WARNING: Only one class in reference dataset.
Cleaning up...

F
======================================================================
FAIL: test_confusionmatrix_with_raster_reference (__main__.Testconfusionmatrix)
Test confusionmatrix with raster reference
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/mneteler/software/grass-addons/src/raster/r.confusionmatrix/testsuite/test_r_confusionmatrix.py", line 71, in test_confusionmatrix_with_raster_reference
    self.assertFilesEqualMd5(
  File "/home/mneteler/software/grass_main/dist.x86_64-pc-linux-gnu/etc/python/grass/gunittest/case.py", line 733, in assertFilesEqualMd5
    self.fail(self._formatMessage(msg, stdmsg))
AssertionError: Output file is not equal to reference file 
Files output_confusionmatrix.csv and data/confusionmatrix_raster_matrix.csv don't have the same MD5 sums

======================================================================
FAIL: test_confusionmatrix_with_singleclass (__main__.Testconfusionmatrix)
Test confusionmatrix with a single class reference and classification
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/mneteler/software/grass-addons/src/raster/r.confusionmatrix/testsuite/test_r_confusionmatrix.py", line 90, in test_confusionmatrix_with_singleclass
    self.assertFilesEqualMd5(
  File "/home/mneteler/software/grass_main/dist.x86_64-pc-linux-gnu/etc/python/grass/gunittest/case.py", line 733, in assertFilesEqualMd5
    self.fail(self._formatMessage(msg, stdmsg))
AssertionError: Output file is not equal to reference file 
Files output_confusionmatrix.csv and data/confusionmatrix_raster_matrix_oneclass.csv don't have the same MD5 sums

----------------------------------------------------------------------
Ran 2 tests in 1.537s
FAILED (failures=2)

Expected behavior Successful run of testsuite

System description (please complete the following information):

g.version -rge
version=8.3.dev
date=2022
revision=8dd3ce7d0c
build_date=2022-11-04
build_platform=x86_64-pc-linux-gnu
build_off_t_size=8
libgis_revision=b35aca1c58
libgis_date=2022-08-28T12:21:16+00:00
proj=8.2.1
gdal=3.4.3
geos=3.10.2
sqlite=3.36.0
neteler commented 1 year ago

Perhaps also interesting:

Add json output option to r.kappa https://github.com/OSGeo/grass/pull/2666