SAGA Reclassify values python error

[skampus]

hi, I noticed that the SAGA module in Processing called "Reclassify values" doesn't work. It can be selected one of the following reclassification methods: [0] single; [1] range; [2] simple table

the method chosen excludes the others. Instead, if I select the method [0] and I insert the related parameters, once the module is launched, it stops with an error asking for the compilation of the parameters of the method [1] and then subsequently of the method [2]. obviously at the end the module doesn't work. it seems that once the reclassifyi method has been chosen, this does not exclude the other two.

In version 2.18 everything works fine.

I attach the description file of the module contained in [1]

Error occurs in QGIS LTR dev, QGIS 3.16, QGIS dev in Windows 10

[1] C: \ OSGeo4W64 \ apps \ qgis-ltr-dev \ python \ plugins \ processing \ algs \ saga \ description \ ReclassifyGridValues.txt

ReclassifyGridValues.txt

[gioman]

@skampus I used this tools recently, and it worked. I'll have a look.

[gioman]

@skampus I used this tools recently, and it worked. I'll have a look.

it seems that "recently" for me meant 3.4 where it "worked".

it seems that once the reclassifyi method has been chosen, this does not exclude the other two.

As far as I know we don't have a way to do it in Processing. In the past we used to split the tools (in this case there would have been 3 separate, one for each method) not sure/don't remember why we don't do it anymore.

In the specific case of this tool it is then necessary to fill all the parameters even the unused ones (including the lookup table) also because it seems that parameters of type "ParameterSelection" cannot be made optional (I may be wrong): the parameters used are the ones that are related to the chosen "method". (I'm saying something silly @nyalldawson @alexbruy ?)

But here it comes the regression: on QGIS > 3.4 when creating a lookup table (even if unused) a python error is returned

Traceback (most recent call last):
File "C:/OSGEO4~1/apps/qgis-ltr/./python/plugins\processing\algs\saga\SagaAlgorithm.py", line 278, in processAlgorithm
with open(tempTableFile, 'w') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'C:/Users/qgis/AppData/Local/Temp/processing_e6524b471e754906bf44ce8152392269\\5fbd8794474ce3.txt'

This affects also the "reclassify values (simple)" tool.

[nyalldawson]

The native reclassify algorithms should do everything this saga one does, but with the benefit that they actually work and are supported. I don't see any reason why the SAGA ones should ever be picked over the native tools here.

[gioman]

I don't see any reason why the SAGA ones should ever be picked over the native tools here.

@nyalldawson well this may be the case for this specific tool, but there are counter examples, such tools not available as native QGIS ones, or tools that are still more robust on SAGA (i.e. some vector geoprocessing operation). The bottom line is that having some redundancy/diversity is good and sometimes very handy (i.e. when the bug is in QGIS tools). Any idea why QGIS now does not finds this temp text files where the look-up table is written (or were the regression may come from)?

[gioman]

But here it comes the regression: on QGIS > 3.4 when creating a lookup table (even if unused) a python error is returned

This seems to affect all the SAGA tools that require a look up table, which is very bad of course.

[nyalldawson]

The bottom line is that having some redundancy/diversity is good and sometimes very handy

Right, but in this case I'm pointing out that there's an ideal solution for @skampus's issue which works in all recent QGIS releases, and can be done right now without having to wait for a bug fix.

[gioman]

Right, but in this case I'm pointing out that there's an ideal solution for @skampus's issue which works in all recent QGIS releases, and can be done right now without having to wait for a bug fix.

yes of course.

[skampus]

I have always used this SAGA module for raster reclassification instead of the raster calculator which, for me, is complex for this operation. anyway you are right: QGIS core reclassification module does its job and works fine (it also acts on no data and this is great). However, in my opinion, the problem remains: a non-functioning third-party module should be excluded from Processing.

Thanks for the attention s.

[gioman]

a non-functioning third-party module should be excluded from Processin

better fix it, as it was working as expected until recently (and as said the root problem affects many other SAGA tools).

[alexbruy]

I'd say even better to drop OTB, SAGA, and maybe GRASS, providers from core, letting interested people to maintain them as 3rd party plugins.

[gioman]

I'd say even better to drop OTB, SAGA, and maybe GRASS, providers from core, letting interested people to maintain them as 3rd party plugins.

@alexbruy yes I (we?) understand this rationale, and I agree. But as far as I remember we can't do it in QGIS 3, correct?

[alexbruy]

Just tested on Linux with master (30a99ab472) and SAGA 7.3.0 from ubuntugis and can't reproduce error from this comment. For me it generates correct result.

[gioman]

Just tested on Linux with master (30a99ab) and SAGA 7.3.0 from ubuntugis and can't reproduce error from this comment. For me it generates correct result.

@alexbruy yes, I forgot a "little" detail in my comment... seems to be a Windows only issue. Anyway before wasting your time Let me check again on a clean Windows machine. I'll get back later.

[gioman]

@alexbruy and as usual... on a clean testing env (Windows 10, 3.10 and 3.16 standalone installers)... all good... shame on me for not having did it yesterday.

[skampus]

So, what do I have to tell to all users having this bug?

[gioman]

So, what do I have to tell to all users having this bug?

if you refer to your original description you have to tell them that all parameters must be filled, even the ones for the unused methods. As said in the past we used to split this tools, but at some point they were all merged together again, don't remember why.

[skampus]

Ok, thank you for your quick reply.

s.

Il mer 25 nov 2020, 21:12 Giovanni Manghi notifications@github.com ha scritto:

So, what do I have to tell to all users having this bug?

if you refer to your original description you have to tell them that all parameters must be filled, even the ones for the unused methods. As said in the past we used to split this tools, but at some point they were all merged together again, don't remember why.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/qgis/QGIS/issues/40260#issuecomment-733924994, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABCY5OTVQOIWB2N2ENSQ4MTSRVQLZANCNFSM4UBCIHGQ .

[nyalldawson]

@skampus my recommendation would be to direct your users to the native qgis reclassify tools.

Whenever I'm teaching qgis I always recommend that users pick algorithms in this order:

1. Qgis native provider: it's the most actively maintained and will work with ALL layer sources supported by qgis, also has nice functionality like saving results directly into existing geopackages, postgres database, etc, and the ability to "data define" parameters values so that they vary feature by feature. Also very fast, and no loss of attributes/quality caused by conversion to intermediate formats.

2. Gdal: a fantastic choice for any raster operations. Algorithms are super robust and extremely well tested, and heavily optimised for speed.

3. Grass: algorithms are extremely robust and mature, but very difficult to learn and suffer from a confusing mash of options. You definitely need to read the manual before using a tool. Also can be quite slow due to forced conversion the grass data types, which can also cause loss/modification of attribute information. But on the plus side, an absolute ton of functionality which isn't available elsewhere.

4. Saga. Use as an absolute last resort. Don't touch the vector algorithms as all have better alternatives in the other providers. Badly tested, frequently breaks, and unfortunately also often outputs incorrect results. Use only when there's no other alternative, and make sure you perform your own qa on the results to verify that the tool is indeed working as advertised.

That's my 2c at least, but it's based on my years of experience in using and maintaining (and developing) these tools.

[alexbruy]

As said in the past we used to split this tools, but at some point they were all merged together again, don't remember why.

Actually there are still separated simplified tools for each reclassification method, don't why issue author ignored them.

[agiudiceandrea]

@gioman, see https://github.com/qgis/QGIS/pull/32097: the tool is still splitted, but the unsplitted one was left in place for backguard compatibility.

[skampus]

As said in the past we used to split this tools, but at some point they were all merged together again, don't remember why.

Actually there are still separated simplified tools for each reclassification method, don't why issue author ignored them.

Sorry I don't understand your note. The point is another: if a module doesn't work, it has to be solved or removed, saying "I don't know why the issue author didn't use other tools" is not useful for anyone.. I solved the problem in another way, but if a module does not work the first thing an user thinks is that QGIS causes the error (which is actually the case). So I expect as answer not "Use anothere tool" (as I did) but 'Thanks for the report'.

[gioman]

@gioman, see #32097: the tool is still splitted, but the unsplitted one was left in place for backguard compatibility.

@agiudiceandrea that sounded familiar :) And now I understand the confusion, the split tools only show in 3.16 not 3.10 (which is my daily driver for work).

[gioman]

but if a module does not work the first thing an user thinks is that QGIS causes the error (which is actually the case).

@skampus the module does work. The one with all the methods together is "just" cumbersome to use, this is why it was split.

[skampus]

ok, it's clear for me now. thank you all and sorry for the noise.

s.

[gioman]

Whenever I'm teaching qgis I always recommend that users pick algorithms in this order:

1. Qgis native provider: it's the most actively maintained and will work with ALL layer sources supported by qgis, also has nice functionality like saving results directly into existing geopackages, postgres database, etc, and the ability to "data define" parameters values so that they vary feature by feature. Also very fast, and no loss of attributes/quality caused by conversion to intermediate formats.

2. Gdal: a fantastic choice for any raster operations. Algorithms are super robust and extremely well tested, and heavily optimised for speed.

3. Grass: algorithms are extremely robust and mature, but very difficult to learn and suffer from a confusing mash of options. You definitely need to read the manual before using a tool. Also can be quite slow due to forced conversion the grass data types, which can also cause loss/modification of attribute information. But on the plus side, an absolute ton of functionality which isn't available elsewhere.

4. Saga. Use as an absolute last resort. Don't touch the vector algorithms as all have better alternatives in the other providers. Badly tested, frequently breaks, and unfortunately also often outputs incorrect results. Use only when there's no other alternative, and make sure you perform your own qa on the results to verify that the tool is indeed working as advertised.

That's my 2c at least, but it's based on my years of experience in using and maintaining (and developing) these tools.

I don't program/develop but I do my share of geoprocessing work and I do teach a lot, and while I see your general point I think it is also a bit unfair. Also for me SAGA tools are kind of last resort when they have counterparts, but the main reason in the lack of documentation, not because they are not robust or because they return wrong results. Sure there is some tool that we know return the wrong result (in the obsolete version we are using in QGIS), but should we go and check how many times this also happened in QGIS? The truth is that for how much we would like to have a fully featured QGIS/GDAL only Processing toolbox we are very far from it, and there is quite a few people out there that still need use tools that we likely will never have as native ones (one example: SAGA is the only provider we have that offers kriging interpolation). Personally I can do all I need using GRASS and SAGA and OTB etc. on the command line, but is not something we can ask to the vast majority of our users. This is why I spent quite a lot a of time in trying keeping this tools working in QGIS. I understand that in QGIS 4 this providers must go and move to plugins, let's focus on that so that we won't leave suddenly our users without important tools.