Error/Crash when "clip multiple raster" with a shapefile

[Fylgier]

Hey, you requested I send you the crashes I had when attempting to clip satellite images from Copernicus / ESA An important note is that I could not replicate this error on my stationary PC. The laptop is the doing it, but a workaround is to just cut with marking the area I want and simply not using a shapefile as a template. So I am not sure this error is because of the plugin.

CRS for the raster I want to clip with:

CRS for the images:

The chosen bands:

The cutting settings:

Location of the new clipped images: (no capital letters or spaces)

Pressing RUN results in an immediate crash of QGIS:

Crash report:

h2. User Feedback

h2. Report Details

Crash ID: fca156ba5922aa97c2b216f5144684720f7019a9

Stack Trace

PyBytes_AsStringAndSize :
PyInit__ogr :
PyCFunction_Call :
PyEval_EvalFrameDefault :
PyErr_Occurred :
PyObject_GenericGetAttr :
PyEval_EvalFrameDefault :
PyErr_Occurred :
PyObject_GenericGetAttr :
PyEval_EvalFrameDefault :
PyErr_Occurred :
PyObject_GenericGetAttr :
PyEval_EvalFrameDefault :
PyErr_Occurred :
PyObject_GenericGetAttr :
PyEval_EvalFrameDefault :
PyFunction_FastCallDict :
PyObject_CallFunctionObjArgs :
PyObject_Call :
PyInit_QtCore :
PyInit_QtCore :
PyInit_QtCore :
PyInit_QtCore :
PyInit_QtCore :
QMetaObject::activate :
QAbstractButton::clicked :
QAbstractButton::click :
QAbstractButton::mouseReleaseEvent :
QToolButton::mouseReleaseEvent :
QWidget::event :
QApplicationPrivate::notify_helper :
QApplication::notify :
QgsApplication::notify :
QCoreApplication::notifyInternal2 :
QApplicationPrivate::sendMouseEvent :
QSizePolicy::QSizePolicy :
QSizePolicy::QSizePolicy :
QApplicationPrivate::notify_helper :
QApplication::notify :
QgsApplication::notify :
QCoreApplication::notifyInternal2 :
QGuiApplicationPrivate::processMouseEvent :
QWindowSystemInterface::sendWindowSystemEvents :
QEventDispatcherWin32::processEvents :
CallWindowProcW :
DispatchMessageW :
QEventDispatcherWin32::processEvents :
qt_plugin_query_metadata :
QEventLoop::exec :
QCoreApplication::exec :
main :
BaseThreadInitThunk :
RtlUserThreadStart :

QGIS Info QGIS Version: 3.0.3-Girona QGIS code revision: commit:8a899c8758 Compiled against Qt: 5.9.2 Running against Qt: 5.9.2 Compiled against GDAL: 2.2.4 Running against GDAL: 2.2.4

System Info CPU Type: x86_64 Kernel Type: winnt Kernel Version: 10.0.17134

[jgrocha]

Thanks @Fylgier for reporting this problem. I was able to confirm it on another Windows install, using the same SCP 6.2.1 version. I'm not able to reproduce in Linux.

[jgrocha]

It does not seem to be a data related issue, because I was able to clip the raster using the QGIS raster tools, without any problem. Doing the same clip with SCP, QGIS crashes.

[semiautomaticgit]

Hello and thank you for reporting this. I tested the clip on Windows with the sample data and I couldn't reproduce the issue. Possibly it is related to the combination of QGIS version and system, as you mentioned that it did work on your stationary PC. I would suggest to open also an issue on the QGIS project, because this could be related to other QGIS issues.

[gis2015]

Hi. I'm following tutorial 1. The 'multiple raster clip' operation worked fine initially. I cut using coordinates (UL, LR) and polygons. Now the operation no longer works in any image. QGIS 3.2.2 does not close but stops running. The error happens on several machines (laptop). System: Windows10 Log message: Traceback (most recent call last): File "C: / Usuários / Sergio / AppData / Roaming / QGIS / QGIS3 \ profiles \ default / python / plugins \ SemiAutomaticClassificationPlugin \ maininterface \ clipmultiplerasters.py", line 82, in clipRastersAction self.clipRasters () File "C: / Usuários / Sergio / AppData / Roaming / QGIS / QGIS3 \ profiles \ default / python / plugins \ SemiAutomaticClassificationPlugin \ maininterface \ clipmultiplerasters.py", line 363, in clipRasters cfg.osSCP.remove (tPMD2) PermissionError: [WinError 32] The file is already being used by another process: 'C: /Usuários/Sergio/AppData/Local/Temp/semiautomaticclassification/20180908_073642185542/20180908_074028324800calc_raster.tif'

[semiautomaticgit]

Hi, a permission error is usually related to system settings. You can try to disable any antivirus. Please follow these steps to copy the plugin log file https://semiautomaticclassificationmanual.readthedocs.io/en/latest/faq.html#how-can-i-report-an-error

[gis2015]

Hi. Thank you for your attention. Follow log file and other documents.

log file.txt

Python error.docx

Python warning.docx

Work sequence.docx

[semiautomaticgit]

Thank you for the documents. I'll try to look into this

[gis2015]

Hi. The QGIS 3.2.3 release eliminated these errors (just in my case). Thank you.

[semiautomaticgit]

Thank you very much for your feedback

[gis2015]

Hi, Luca. There is an error similar to that reported on September 8, 2018 (QGIS 3.2.2) when I try to cut a set of bands. The QGIS 3.2.3 release eliminated those errors.

Now I'm using QGIS 3.4.7. The QGIS continues to work. The plugin does not close, but stops responding to commands. I used several LandSat images. The last attempt was with Frascati dataset. The error report is attached. report_clip_frascati.txt

Hi, Luca. I updated QGIS to version 3.4.8. Unfortunately the error continues to occur. The most recent error report and the SCP print screen are attached.

report_clip_frascati_3_4_8.txt

More information: It seems that the problem is in the Clip Raster by Extent (or by Mask Layer) GDAL tool when using the Run as Batch Process option.

[semiautomaticgit]

Thank you very much. I'll try to investigate this issue

[semiautomaticgit]

I'm closing this because of the new version 7. If you still get the same issue, please reopen it

[ssinghngri]

MacOS: Big Sur version 11.1 QGIS version 3.16.2-Hannover SCP Plugin: version 7.4.0

The clipped image is not viewed in the "MAP VIEW" even though the projection of my shapefile and the Landsat image band projections are same. What is the problem here? Any help will be deeply appreciated. Images are attached here.

Thanks.

Some more images attach here after my research on error.

It looks like some issue with GDAL but i am not able to resolve it. Any detailed help will be most welcome. Please keep in mind the MacOS version (Mac OS is actually messing up everything after update to Big Sur) and QGIS and SCP versions while responding.

[semiautomaticgit]

Hello @ssinghngri , please update the plugin to version 7.4.1

[ssinghngri]

Hi Luca,

Thank you so much for your prompt response. I did update the plugin to version 7.4.1. However, still having the same issue. Images are getting clipped but not viewed in the "MAP VIEW". See the below errors getting generated now.

Thank you for taking a dig to resolve the issue.

[semiautomaticgit]

@ssinghngri please update again to v. 7.4.2

[ssinghngri]

Thanks again. I updated to 7.4.2 and still having the same issue. Please refer below.

[semiautomaticgit]

Thank you @ssinghngri please attach the log file

[ssinghngri]

errors_logfile.txt here is the log file

[semiautomaticgit]

Thank you. Please open the tab Processing setting and try to enter in Python executable path: /usr/bin/python3

Also, in GDAL directory you should enter the path to the directory containing tools such as gdal_translate and gdalwarp

[ssinghngri]

Hello,

I did made those changes as suggested yet no success. Please see these attached here.

Changes made here:

GDAL location is here:

I performed clip file after changing the path, quitting and re-starting QGIS but no success.

Help!

Is it something to do with coordinate systems? I have both layers in same coordinate system but still thinking if I'm doing anything wrong here.

[semiautomaticgit]

Thank you. It is an issue related to MacOS and plugin installation. Please update to version 7.4.3 and copy the log file again

[ssinghngri]

Did as you suggested. Updated to 7.4.3, still no success. 👎

error_logfile_2.txt

A New problem of crashing came up now. See below, when I click on "TEST DEPENDENCIES", QGIS stops to responding.

[semiautomaticgit]

Thank you, I suspect it is something related to Python dependencies. Please update to version 7.4.4, and test the dependencies to provide information about the cause of the issue.

[ssinghngri]

Thanks. I updated again to 7.4.4 and it did not crash due to "dependencies", however still some issues. Please see below the screenshot and logfile.

error_logfile3.txt

[semiautomaticgit]

Thank you, I think that you should add "Programs" to the GDAL path

[ssinghngri]

Thank you. Okay one error gone with modification to GDAL path. Still one for GDAL multiprocessing remains. Any suggestions for this one.

error_logfile4.txt

I believe if these errors will not be gone, i won't be able to get clipped image successfully.

[semiautomaticgit]

Yes, this error prevents clipping unfortunately. Please update again to version 7.4.5, hopefully this will provide information about Python multiprocessing.

[ssinghngri]

Thanks. version 7.4.5 is still not showing available

[ssinghngri]

Thanks. Some new python problems after updating to 7.4.5.

error_logfile5.txt error_logfile5.txt

[semiautomaticgit]

Yes, sorry that was my fault. Please update to version 7.4.6

[macromicrobe]

Clicking the "Test dependencies" button now causes Q to not respond, and will not generate a log file or errors. I have attempted 10 times now, all with the same result.

[ssinghngri]

Thanks LUCA but no success.

Yes @macromicrobe same here. application not responding with "TEST DEPENDENCIES" after updating to 7.4.6

[semiautomaticgit]

Clicking the "Test dependencies" button now causes Q to not respond, and will not generate a log file or errors. I have attempted 10 times now, all with the same result.

That's quite strange, I think I'll have to look into it more

[ssinghngri]

Yes please and many thanks for your continuous efforts to help me. 👍

[macromicrobe]

Thank you very much for your support, @semiautomaticgit !

[semiautomaticgit]

Thank you very much @ssinghngri and @macromicrobe for your cooperation. Please try again with version 7.4.7

[ssinghngri]

Still application QGIS NOT RESPONDING with updated SCP 7.4.7 "TEST DEPENDENCIES"

[ssinghngri]

Thank you @semiautomaticgit for keeping on.

[semiautomaticgit]

You are very welcome @ssinghngri Please try version 7.4.8

[ssinghngri]

Thank you. Updated to version 7.4.8. However, stil application (QGIS) not responding. Crashing.

[semiautomaticgit]

You are welcome. Did you set also the Python path in Processing setting?

[semiautomaticgit]

Other users reported that SCP works with KyngChaos package QGIS 3.16.2-2, maybe you can try this

[ssinghngri]

MacOS: Big Sur version 11.1 QGIS version 3.16.2-Hannover SCP Plugin: version 7.4.8

QGIS version	3.16.2-Hannover	QGIS code revision	c6ac5a6052
Compiled against Qt	5.14.2	Running against Qt	5.14.2
Compiled against GDAL/OGR	3.1.2	Running against GDAL/OGR	3.1.2
Compiled against GEOS	3.8.1-CAPI-1.13.3	Running against GEOS	3.8.1-CAPI-1.13.3
Compiled against SQLite	3.31.1	Running against SQLite	3.31.1
PostgreSQL Client Version	12.3	SpatiaLite Version	4.3.0a
QWT Version	6.1.4	QScintilla2 Version	2.11.4
Compiled against PROJ	6.3.2	Running against PROJ	Rel. 6.3.2, May 1st, 2020
OS Version	macOS 10.16
Active python plugins	SemiAutomaticClassificationPlugin; quick_map_services; Qgis2threejs; qgis-maptiler-plugin; processing; db_manager; MetaSearch

Dear Luca,

Thank you for your response. I did try everything possible I could. Uninstalled QGIS and installed again as suggested from KyngChaos package. Apparently it looks like that QGIS is using python version 3.7 although it says it should be using python 3.9. Anyway installing everything afresh leads to NO crashing of QGIS now. Providing paths in the Processing Settings also leads to remove few minor errors with GDAL. However, unfortunately, the GDAL Multiprocess still FAILS while checking "TEST DEPENDENCIES" and hence clipping as per polygon shapefile of LANDSAT images is not happening.

Log File here:

error_logfile.txt

WHY sys.path not showing python 3.9.1 installed in my system, rather QGIS 3.16.2 is using python3.7?

sys.path ['/Applications/QGIS.app/Contents/MacOS/../Resources/python', '/Users/shatrughansingh/Library/Application Support/QGIS/QGIS3/profiles/default/python', '/Users/shatrughansingh/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins', '/Applications/QGIS.app/Contents/MacOS/../Resources/python/plugins', '/Applications/QGIS.app/Contents/MacOS/lib/python3.7/site-packages/opencv_contrib_python-4.3.0.36-py3.7-macosx-10.13.0-x86_64.egg', '/Applications/QGIS.app/Contents/MacOS/lib/python3.7/site-packages/scipy-1.5.1-py3.7-macosx-10.13.0-x86_64.egg', '/Applications/QGIS.app/Contents/MacOS/lib/python3.7/site-packages/Rtree-0.9.4-py3.7.egg', '/Applications/QGIS.app/Contents/MacOS/lib/python3.7/site-packages/GDAL-3.1.2-py3.7-macosx-10.13.0-x86_64.egg', '/Applications/QGIS.app/Contents/MacOS/lib/python3.7/site-packages/geopandas-0.8.1-py3.7.egg', '/Applications/QGIS.app/Contents/MacOS/lib/python37.zip', '/Applications/QGIS.app/Contents/MacOS/lib/python3.7/site-packages', '/Applications/QGIS.app/Contents/MacOS/lib/python3.7/site-packages/numba-0.50.1-py3.7-macosx-10.13.0-x86_64.egg', '/Applications/QGIS.app/Contents/MacOS/lib/python3.7/site-packages/Pillow-7.2.0-py3.7-macosx-10.13.0-x86_64.egg', '/Applications/QGIS.app/Contents/MacOS/lib/python3.7/site-packages/statsmodels-0.11.1-py3.7-macosx-10.13.0-x86_64.egg', '/Applications/QGIS.app/Contents/MacOS/lib/python3.7/site-packages/snuggs-1.4.7-py3.7.egg', '/Applications/QGIS.app/Contents/MacOS/lib/python3.7/site-packages/rasterio-1.1.5-py3.7-macosx-10.13.0-x86_64.egg', '/Applications/QGIS.app/Contents/MacOS/lib/python3.7/site-packages/affine-2.3.0-py3.7.egg', '/Applications/QGIS.app/Contents/MacOS/lib/python3.7/site-packages/click_plugins-1.1.1-py3.7.egg', '/Applications/QGIS.app/Contents/MacOS/lib/python3.7/site-packages/pandas-1.1.0-py3.7-macosx-10.13.0-x86_64.egg', '/Applications/QGIS.app/Contents/MacOS/lib/python3.7/site-packages/netCDF4-1.5.3-py3.7-macosx-10.13.0-x86_64.egg', '/Applications/QGIS.app/Contents/MacOS/lib/python3.7/site-packages/patsy-0.5.1-py3.7.egg', '/Applications/QGIS.app/Contents/MacOS/lib/python3.7/site-packages/matplotlib-3.3.0-py3.7-macosx-10.13.0-x86_64.egg', '/Applications/QGIS.app/Contents/MacOS/lib/python3.7/site-packages/numpy-1.19.1-py3.7-macosx-10.13.0-x86_64.egg', '/Applications/QGIS.app/Contents/MacOS/lib/python3.7', '/Applications/QGIS.app/Contents/MacOS/lib/python3.7/site-packages/pyproj-2.6.0-py3.7-macosx-10.13.0-x86_64.egg', '/Applications/QGIS.app/Contents/MacOS/lib/python3.7/site-packages/cftime-1.2.1-py3.7-macosx-10.13.0-x86_64.egg', '/Applications/QGIS.app/Contents/MacOS/lib/python3.7/site-packages/Fiona-1.8.13.post1-py3.7-macosx-10.13.0-x86_64.egg', '/Applications/QGIS.app/Contents/MacOS/lib/python3.7/lib-dynload', '/Users/shatrughansingh/Library/Application Support/QGIS/QGIS3/profiles/default/python']

SIMILARLY GDAL installed is 3.2 while QGIS using 3.1. Don't know how to address these settings/issues.

Thanks for your help.

[semiautomaticgit]

Dear @ssinghngri , I think that QGIS is compiled with specific Python and GDAL versions (but I'm not sure) . You could try to clear the SCP Settings related to GDAL and Python paths and restart QGIS. However I think the issue is related to the installation method.

Other user reported that SCP worked using KyngChaos package QGIS 3.16.2-2 https://www.kyngchaos.com/software/qgis/ that includes GDAL 3.2 and Python 3.9. Python must be installed before installing GDAL and QGIS. Also, the readme instructions should describe how to clean the old path settings and unset GRASS docs path. Finally add custom environment variable Proj-Network.