Need threadsafe updates crash resolved

ConservationInternational / trends.earth

trends.earth - measure land change

http://trends.earth

GNU General Public License v2.0

109 stars 45 forks source link

Need threadsafe updates crash resolved #579

Closed timlinux closed 2 years ago

timlinux commented 2 years ago

@azvoleff please share some diagnostics.

gdaldegan commented 2 years ago

Was running Custom SOC tool 'Execute Locally option' and QGIS crashed See below for report details:

User Feedback

QGIS 22 crash

Report Details

Crash ID: 7b6e447485c2e9c50abcdcd590b4ee21cb2da07f

Stack Trace

QMetaObject::cast :
UiaNodeTraverser::NavigateProvider :
UiaNodeFactory::ConstructUiaNode :
UiaNodeFactory::FromPartialNodeInfo :
UiaNodeFactory::FromLocalProvider :
UiaNodeFactory::FromLocalProvider :
EventMap::AddEntry :
UiaRaiseAutomationEvent :
QAccessible::updateAccessibility :
QLCDNumber::value :
QObject::event :
QWidget::event :
QLineEdit::event :
QgsPresetSchemeColorRamp::clone :
QApplicationPrivate::notify_helper :
QApplication::notify :
QgsApplication::notify :
QCoreApplication::notifyInternal2 :
QCoreApplicationPrivate::sendPostedEvents :
qt_plugin_query_metadata :
QEventDispatcherWin32::processEvents :
qt_plugin_query_metadata :
QEventLoop::exec :
QCoreApplication::exec :
main :
BaseThreadInitThunk :
RtlUserThreadStart :

QGIS Info QGIS Version: 3.22.5-Bia?owie?a QGIS code revision: c2723178 Compiled against Qt: 5.15.2 Running against Qt: 5.15.2 Compiled against GDAL: 3.4.1 Running against GDAL: 3.4.1

System Info CPU Type: x86_64 Kernel Type: winnt Kernel Version: 10.0.19044

azvoleff commented 2 years ago

And one more trace, which came up during a background update of the trends.earth datasets window. Unforrtunately not too much in this one:

User Feedback

QGIS open to Trends.Earth plugin page. Update of dataset status proceeding in background.

Report Details

Python Stack Trace

Windows fatal exception: access violation

Current thread 0x00004484 (most recent call first):
<no Python frame>

Stack Trace No stack trace is available.

QGIS Info QGIS Version: 3.24.0-Tisler QGIS code revision: 6b44a42058 Compiled against Qt: 5.15.2 Running against Qt: 5.15.2 Compiled against GDAL: 3.4.1 Running against GDAL: 3.4.1

System Info CPU Type: x86_64 Kernel Type: winnt Kernel Version: 10.0.19044

gdaldegan commented 2 years ago

got this one when moving QGIS from laptop screen to external screen

Stack Trace

QPainter::drawPixmap : PyInit_QtGui : PyArg_ParseTuple_SizeT : PyEval_EvalFrameDefault : PyObject_GC_Del : PyFunction_Vectorcall : PyFloat_FromDouble : PyVectorcall_Call : PyObject_Call : PyInit_QtWidgets : QTreeView::drawRow : PyInit_QtWidgets : QTreeView::drawTree : QTreeView::paintEvent : PyInit_QtWidgets : QWidget::event : QFrame::event : QAbstractItemView::viewportEvent : PyInit_QtWidgets : QCoreApplicationPrivate::sendThroughObjectEventFilters : QApplicationPrivate::notify_helper : QApplication::notify : QgsApplication::notify : QCoreApplication::notifyInternal2 : QWidgetPrivate::drawWidget : QWidgetPrivate::paintSiblingsRecursive : QWidgetPrivate::drawWidget : QWidgetPrivate::paintSiblingsRecursive : QWidgetPrivate::drawWidget : QWidgetPrivate::paintSiblingsRecursive : QWidgetPrivate::drawWidget : QWidgetPrivate::paintSiblingsRecursive : QWidgetPrivate::paintSiblingsRecursive : QWidgetPrivate::drawWidget : QWidgetPrivate::paintSiblingsRecursive : QWidgetPrivate::drawWidget : QWidgetPrivate::paintSiblingsRecursive : QWidgetPrivate::drawWidget : QWidgetPrivate::paintSiblingsRecursive : QWidgetPrivate::drawWidget : QWidgetPrivate::moveRect : QWidget::event : QApplicationPrivate::notify_helper : QApplication::notify : QgsApplication::notify : QCoreApplication::notifyInternal2 : QWidgetPrivate::scrollRect : QApplication::windowIcon : QSizePolicy::QSizePolicy : QObject::qt_static_metacall : QWindowPrivate::emitScreenChangedRecursion : QWindowPrivate::setTopLevelScreen : QGuiApplicationPrivate::processWindowScreenChangedEvent : QWindowSystemInterface::handleWindowStateChanged : UserCallWinProcCheckWow : DispatchClientMessage : _fnINOUTLPRECT : KiUserCallbackDispatcherContinue : NtUserMessageCall : RealDefWindowProcWorker : RealDefWindowProcW : DoMsgDefault : OnDwpSysCommand : _ThemeDefWindowProc : ThemeDefWindowProcW : DefWindowProcW : UserCallWinProcCheckWow : DispatchClientMessage : _fnDWORD : KiUserCallbackDispatcherContinue : NtUserMessageCall : RealDefWindowProcWorker : RealDefWindowProcW : DoMsgDefault : OnDwpNcLButtonDown : _ThemeDefWindowProc : ThemeDefWindowProcW : DefWindowProcW : UserCallWinProcCheckWow : DispatchMessageWorker : QEventDispatcherWin32::processEvents : qt_plugin_query_metadata : QEventLoop::exec : QCoreApplication::exec : main : BaseThreadInitThunk : RtlUserThreadStart :

QGIS Info QGIS Version: 3.22.5-Bia?owie?a QGIS code revision: c2723178 Compiled against Qt: 5.15.2 Running against Qt: 5.15.2 Compiled against GDAL: 3.4.1 Running against GDAL: 3.4.1

System Info CPU Type: x86_64 Kernel Type: winnt Kernel Version: 10.0.19044

gkahiu commented 2 years ago

tasks_threading

Since QMessageLog is based on QObject and the static method logMessage call initializes it in the main thread, then there is a high likelihood calling logger.log in background threads - through the workers or QgsTask subclasses - is causing the crashes. I will remove the references in other workers and hopefully this will help address the issue.

azvoleff commented 2 years ago

But the calls to logger.debug here are calling python's logging module. Those I believe are threadsafe, no?

gkahiu commented 2 years ago

Aah sorry I was too quick there, I was thinking the LocalJobTask uses QMessageLog. If its Python logging then its thread-safe.

For the workers, any that uses logger.log inside the run/work method will potentially cause crashes.

azvoleff commented 2 years ago

Here is a more useful trace - I wasn't doing anything other than having a dialog box (for the SDG Indicator) open.

Report Details

Python Stack Trace

Windows fatal exception: access violation

Thread 0x00004adc (most recent call first):
  File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\jobs\manager.py", line 86 in _get_extent_tuple_vector
    rect = QgsVectorLayer(str(path), "vector file", "ogr").extent()
  File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\jobs\manager.py", line 129 in _set_results_extents_vector
    job.results.extent = _get_extent_tuple_vector(job.results.vector.uri.uri)
  File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\jobs\manager.py", line 144 in set_results_extents
    _set_results_extents_vector(job)
  File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\jobs\manager.py", line 952 in _get_local_jobs
    set_results_extents(job)
  File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\jobs\manager.py", line 888 in _refresh_local_downloaded_jobs
    for j in self._get_local_jobs(jobs.JobStatus.DOWNLOADED)
  File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\jobs\manager.py", line 308 in refresh_local_state
    self._refresh_local_downloaded_jobs()
  File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\main_widget.py", line 418 in update_local_state
    job_manager.refresh_local_state()
  File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\main_widget.py", line 108 in work
    self.widget.update_local_state()
  File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\main_widget.py", line 78 in run
    self.work()

Current thread 0x00009710 (most recent call first):
  File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\jobs\manager.py", line 86 in _get_extent_tuple_vector
    rect = QgsVectorLayer(str(path), "vector file", "ogr").extent()
  File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\jobs\manager.py", line 129 in _set_results_extents_vector
    job.results.extent = _get_extent_tuple_vector(job.results.vector.uri.uri)
  File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\jobs\manager.py", line 144 in set_results_extents
    _set_results_extents_vector(job)
  File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\jobs\manager.py", line 952 in _get_local_jobs
    set_results_extents(job)
  File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\jobs\manager.py", line 888 in _refresh_local_downloaded_jobs
    for j in self._get_local_jobs(jobs.JobStatus.DOWNLOADED)
  File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\jobs\manager.py", line 308 in refresh_local_state
    self._refresh_local_downloaded_jobs()
  File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\main_widget.py", line 418 in update_local_state
    job_manager.refresh_local_state()
  File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\main_widget.py", line 108 in work
    self.widget.update_local_state()
  File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\main_widget.py", line 78 in run
    self.work()

Thread 0x00008410 (most recent call first):
  File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\main_widget.py", line 512 in launch_algorithm_execution_dialogue
    result=dialog.exec_()

Stack Trace No stack trace is available.

QGIS Info QGIS Version: 3.24.0-Tisler QGIS code revision: 6b44a42058 Compiled against Qt: 5.15.2 Running against Qt: 5.15.2 Compiled against GDAL: 3.4.1 Running against GDAL: 3.4.1

System Info CPU Type: x86_64 Kernel Type: winnt Kernel Version: 10.0.19044

gkahiu commented 2 years ago

Thanks @azvoleff this was very helpful, there were calls to the main widget inside the thread workers. I have updated the PR, kindly review and let me know if the issue still persists.

Perhaps for future implementation, the JobManager might need some refactoring to make it thread safe especially locking *_jobs variables, access to file resources and QObject-based objects.

azvoleff commented 2 years ago

Almost there. One more it looks like. Below happened when I was running a job while also the reports were apparently being generated in the background.

Report Details

Python Stack Trace

Windows fatal exception: access violation

Thread 0x00001354 (most recent call first):
  File "C:\OSGeo4W\apps\Python39\lib\subprocess.py", line 1470 in _wait
    result = _winapi.WaitForSingleObject(self._handle,
  File "C:\OSGeo4W\apps\Python39\lib\subprocess.py", line 1189 in wait
    return self._wait(timeout=timeout)
  File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\reports\generator.py", line 989 in run
    self._process.wait()

Thread 0x0000942c (most recent call first):
  File "C:\OSGeo4W\apps\Python39\lib\site-packages\GDAL-3.4.1-py3.9-win-amd64.egg\osgeo\gdal.py", line 4189 in TranslateInternal
    return _gdal.TranslateInternal(*args)
  File "C:\OSGeo4W\apps\Python39\lib\site-packages\GDAL-3.4.1-py3.9-win-amd64.egg\osgeo\gdal.py", line 457 in Translate
    return TranslateInternal(destName, srcDS, opts, callback, callback_data)
  File "D:\Code\LandDegradation\trends.earth\LDMP\ext-libs\te_algorithms\gdal\workers.py", line 190 in cut_tile
    res = gdal.Translate(
  File "D:\Code\LandDegradation\trends.earth\LDMP\ext-libs\te_algorithms\gdal\workers.py", line 181 in _cut_tiles_sequential
    results.append(cut_tile(param))
  File "D:\Code\LandDegradation\trends.earth\LDMP\ext-libs\te_algorithms\gdal\workers.py", line 289 in work
    _cut_tiles_sequential(params)
  File "D:\Code\LandDegradation\trends.earth\LDMP\ext-libs\te_algorithms\gdal\land_deg\land_deg.py", line 1103 in _process_region
    tiles = translate_worker.work()
  File "D:\Code\LandDegradation\trends.earth\LDMP\ext-libs\te_algorithms\gdal\land_deg\land_deg.py", line 1172 in _compute_ld_summary_table
    this_summary_table, these_reproj_paths, these_output_paths, error_message = _process_region(
  File "D:\Code\LandDegradation\trends.earth\LDMP\ext-libs\te_algorithms\gdal\land_deg\land_deg.py", line 310 in summarise_land_degradation
    summary_table, output_path, reproj_path = _compute_ld_summary_table(
  File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\localexecution\ldn.py", line 367 in compute_ldn
    return summarise_land_degradation(
  File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\jobs\manager.py", line 162 in run
    self.completed_job = execution_handler(

Current thread 0x00009034 (most recent call first):
  File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\jobs\mvc.py", line 182 in paint
    painter.drawPixmap(option.rect.x(), option.rect.y(), pixmap)

Stack Trace No stack trace is available.

QGIS Info QGIS Version: 3.24.0-Tisler QGIS code revision: 6b44a42058 Compiled against Qt: 5.15.2 Running against Qt: 5.15.2 Compiled against GDAL: 3.4.1 Running against GDAL: 3.4.1

System Info CPU Type: x86_64 Kernel Type: winnt Kernel Version: 10.0.19044

gdaldegan commented 2 years ago

QGIS keeps crashing when running 'Indicator for SDG15.3.1' or 'Drought vulnerability summary table'.

I have this version installed: te_version

This is the report I get:

Report Details

Crash ID: 09f58f9881cb6ddc389f5a9fc67100846079b985

Stack Trace

GDALRasterBand::GetBand :
VRTSimpleSource::SetSrcBand :
VRTSourcedRasterBand::ConfigureSource :
GDALBuildVRT :
GDALBuildVRT :
GDALBuildVRT :
PyInit__gdal :
PyInit__gdal :
PyArg_ParseTuple_SizeT :
PyObject_Call :
PyObject_Call :
PyEval_EvalFrameDefault :
PyObject_GC_Del :
PyEval_EvalFrameDefault :
PyObject_GC_Del :
PyFunction_Vectorcall :
PyEval_EvalFrameDefault :
PyFunction_Vectorcall :
PyEval_EvalFrameDefault :
PyEval_EvalFrameDefault :
PyObject_GC_Del :
PyFunction_Vectorcall :
PyEval_EvalFrameDefault :
PyEval_EvalFrameDefault :
PyFunction_Vectorcall :
PyFloat_FromDouble :
PyVectorcall_Call :
PyObject_Call :
QgsPresetSchemeColorRamp::clone :
QgsTask::start :
QThreadPoolPrivate::reset :
QThread::start :
BaseThreadInitThunk :
RtlUserThreadStart :

QGIS Info QGIS Version: 3.22.6-Bia?owie?a QGIS code revision: b6f33b1a Compiled against Qt: 5.15.2 Running against Qt: 5.15.2 Compiled against GDAL: 3.4.2 Running against GDAL: 3.4.2

System Info CPU Type: x86_64 Kernel Type: winnt Kernel Version: 10.0.22000 0

azvoleff commented 2 years ago

And another, similar to what I saw before:

Report Details

Python Stack Trace

Windows fatal exception: access violation

Thread 0x0000b9b4 (most recent call first):
<no Python frame>

Thread 0x000068c4 (most recent call first):
  File "D:\Code\LandDegradation\trends.earth\LDMP\ext-libs\te_algorithms\gdal\util_numba.py", line 89 in zonal_total
    if z[i] not in totals:
  File "D:\Code\LandDegradation\trends.earth\LDMP\ext-libs\te_algorithms\gdal\land_deg\land_deg.py", line 726 in _process_block_summary
    lc_summary = zonal_total(deg_lc, cell_areas, mask)
  File "D:\Code\LandDegradation\trends.earth\LDMP\ext-libs\te_algorithms\gdal\land_deg\worker.py", line 121 in work
    result = self.processing_function(
  File "D:\Code\LandDegradation\trends.earth\LDMP\ext-libs\te_algorithms\gdal\land_deg\land_deg.py", line 993 in _summarize_tile
    result = summarizer.work()
  File "D:\Code\LandDegradation\trends.earth\LDMP\ext-libs\te_algorithms\gdal\land_deg\land_deg.py", line 1040 in _aoi_process_sequential
    output = _summarize_tile(item)
  File "D:\Code\LandDegradation\trends.earth\LDMP\ext-libs\te_algorithms\gdal\land_deg\land_deg.py", line 1126 in _process_region
    summary_table, output_paths = _aoi_process_sequential(inputs)
  File "D:\Code\LandDegradation\trends.earth\LDMP\ext-libs\te_algorithms\gdal\land_deg\land_deg.py", line 1172 in _compute_ld_summary_table
    this_summary_table, these_reproj_paths, these_output_paths, error_message = _process_region(
  File "D:\Code\LandDegradation\trends.earth\LDMP\ext-libs\te_algorithms\gdal\land_deg\land_deg.py", line 310 in summarise_land_degradation
    summary_table, output_path, reproj_path = _compute_ld_summary_table(
  File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\localexecution\ldn.py", line 367 in compute_ldn
    return summarise_land_degradation(
  File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\jobs\manager.py", line 162 in run
    self.completed_job = execution_handler(

Current thread 0x00002514 (most recent call first):
  File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\jobs\mvc.py", line 182 in paint
    painter.drawPixmap(option.rect.x(), option.rect.y(), pixmap)

Stack Trace No stack trace is available.

QGIS Info QGIS Version: 3.24.0-Tisler QGIS code revision: 6b44a42058 Compiled against Qt: 5.15.2 Running against Qt: 5.15.2 Compiled against GDAL: 3.4.1 Running against GDAL: 3.4.1

System Info CPU Type: x86_64 Kernel Type: winnt Kernel Version: 10.0.19044

gkahiu commented 2 years ago

Thanks @azvoleff! Not sure exactly what might be the cause but it might be related to different threads attempting to access the same Job object i.e. the main UI thread (perhaps when refreshing the dataset widget) and that for the LocalJobTask object. Seems its specific to the summarise_land_degradation algorithm. Am unable to replicate it but could you check to see if other local algorithms are causing a crash?

azvoleff commented 2 years ago

Thanks looks correct. I think I've gotten if fixed (made sure to pass a copy of the job, and fixed so the few lines that are modifying it are in the main thread). I'll push this shortly @gabrieldaldegan so you can test (this should fix #645).

azvoleff commented 2 years ago

@gkahiu - I'm actually still getting this, even after confirming that the job object is never accessed/modified in the thread. Is it possible there is any link to recent changes in the reports function / fixes to address the crashes that were arising due to the way the job manager was refreshing? First I started seeing these crashes was last Fri.

Python Stack Trace Windows fatal exception: access violation

Thread 0x00008a84 (most recent call first): File "C:\OSGeo4W\apps\Python39\lib\subprocess.py", line 1420 in _execute_child hp, ht, pid, tid = _winapi.CreateProcess(executable, args, File "C:\OSGeo4W\apps\Python39\lib\subprocess.py", line 951 in init self._execute_child(args, executable, preexec_fn, close_fds, File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\reports\generator.py", line 992 in run self._process = subprocess.Popen(

Thread 0x00009354 (most recent call first): File "C:\OSGeo4W\apps\Python39\lib\site-packages\GDAL-3.4.1-py3.9-win-amd64.egg\osgeo\gdal.py", line 4189 in TranslateInternal return _gdal.TranslateInternal(*args) File "C:\OSGeo4W\apps\Python39\lib\site-packages\GDAL-3.4.1-py3.9-win-amd64.egg\osgeo\gdal.py", line 457 in Translate return TranslateInternal(destName, srcDS, opts, callback, callback_data) File "D:\Code\LandDegradation\trends.earth\LDMP\ext-libs\te_algorithms\gdal\workers.py", line 190 in cut_tile res = gdal.Translate( File "D:\Code\LandDegradation\trends.earth\LDMP\ext-libs\te_algorithms\gdal\workers.py", line 181 in _cut_tiles_sequential results.append(cut_tile(param)) File "D:\Code\LandDegradation\trends.earth\LDMP\ext-libs\te_algorithms\gdal\workers.py", line 289 in work _cut_tiles_sequential(params) File "D:\Code\LandDegradation\trends.earth\LDMP\ext-libs\te_algorithms\gdal\drought.py", line 843 in _summarize_over_aoi tiles = translate_worker.work() File "D:\Code\LandDegradation\trends.earth\LDMP\ext-libs\te_algorithms\gdal\drought.py", line 992 in _compute_drought_summary_table result, out_files, error_message = _summarize_over_aoi( File "D:\Code\LandDegradation\trends.earth\LDMP\ext-libs\te_algorithms\gdal\drought.py", line 641 in summarise_drought_vulnerability summary_table, out_path = _compute_drought_summary_table( File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\localexecution\drought.py", line 196 in compute_drought_vulnerability return summarise_drought_vulnerability( File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\jobs\manager.py", line 165 in run self.results = execution_handler(

Current thread 0x0000b46c (most recent call first): File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\jobs\mvc.py", line 182 in paint painter.drawPixmap(option.rect.x(), option.rect.y(), pixmap)

Stack Trace No stack trace is available.

QGIS Info QGIS Version: 3.24.0-Tisler QGIS code revision: 6b44a42058 Compiled against Qt: 5.15.2 Running against Qt: 5.15.2 Compiled against GDAL: 3.4.1 Running against GDAL: 3.4.1

System Info CPU Type: x86_64 Kernel Type: winnt Kernel Version: 10.0.19044

gkahiu commented 2 years ago

Ok, let me see if I can replicate and correlate with the changes to the report fixes.

gdaldegan commented 2 years ago

QGIS keeps crashing when tryin to run Indicator for SDG 15.3.1 or Drought vulnerability summary table

using the version below: te_version

gdaldegan commented 2 years ago

The issue above is still happening when running Indicator SDG 15.3.1 on the latest TE commitment: te_version

Unfortunately the report does not tell us much

azvoleff commented 2 years ago

One more - looks like the local state is getting updated from two separate threads at the same time:

Python Stack Trace Windows fatal exception: access violation

Thread 0x0000655c (most recent call first): File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\jobs\manager.py", line 87 in _get_extent_tuple_vector rect = QgsVectorLayer(str(path), "vector file", "ogr").extent() File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\jobs\manager.py", line 130 in _set_results_extents_vector job.results.extent = _get_extent_tuple_vector(job.results.vector.uri.uri) File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\jobs\manager.py", line 145 in set_results_extents _set_results_extents_vector(job) File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\jobs\manager.py", line 964 in _get_local_jobs set_results_extents(job) File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\jobs\manager.py", line 900 in _refresh_local_downloaded_jobs for j in self._get_local_jobs(jobs.JobStatus.DOWNLOADED) File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\jobs\manager.py", line 294 in refresh_local_state self._refresh_local_downloaded_jobs() File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\main_widget.py", line 78 in work job_manager.refresh_local_state() File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\main_widget.py", line 69 in run self.work()

Current thread 0x00004e6c (most recent call first): File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\jobs\manager.py", line 87 in _get_extent_tuple_vector rect = QgsVectorLayer(str(path), "vector file", "ogr").extent() File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\jobs\manager.py", line 130 in _set_results_extents_vector job.results.extent = _get_extent_tuple_vector(job.results.vector.uri.uri) File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\jobs\manager.py", line 145 in set_results_extents _set_results_extents_vector(job) File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\jobs\manager.py", line 964 in _get_local_jobs set_results_extents(job) File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\jobs\manager.py", line 900 in _refresh_local_downloaded_jobs for j in self._get_local_jobs(jobs.JobStatus.DOWNLOADED) File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\jobs\manager.py", line 314 in refresh_local_state self._refresh_local_downloaded_jobs() File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\main_widget.py", line 78 in work job_manager.refresh_local_state() File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\main_widget.py", line 69 in run self.work()

Thread 0x00000610 (most recent call first): File "C:\OSGeo4W/apps/qgis/./python\qgis\utils.py", line 182 in open_stack_dialog dlg.showMessage() File "C:\OSGeo4W/apps/qgis/./python\qgis\utils.py", line 97 in showException open_stack_dialog(type, value, tb, msg) File "C:\OSGeo4W/apps/qgis/./python\qgis\utils.py", line 190 in qgis_excepthook showException(type, value, tb, None, messagebar=in_main_thread) File "D:\Documents and Settings/azvoleff/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LDMP\main_widget.py", line 556 in launch_algorithm_executiondialogue result=dialog.exec()

Stack Trace No stack trace is available.

QGIS Info QGIS Version: 3.24.0-Tisler QGIS code revision: 6b44a42058 Compiled against Qt: 5.15.2 Running against Qt: 5.15.2 Compiled against GDAL: 3.4.1 Running against GDAL: 3.4.1

System Info CPU Type: x86_64 Kernel Type: winnt Kernel Version: 10.0.19044

gkahiu commented 2 years ago

@azvoleff I've updated the PR to manage access to changes in local and remote states. Let me know if this fixes the issue.

gdaldegan commented 2 years ago

QGIS is still crashing when running SDG 1531 or SO3 (drought) summary tools -- sub-indicator function are running fine. Unfortunately not much details on the report: qgis_crash_sdg1531

version:

te_version

I'm starting to wonder if it is related to the QGIS version I currently have installed on my pc, which is the QGIS 3.22.6. I saw that John uses 3.24(?)

gdaldegan commented 2 years ago

@gkahiu please let me know how I can use First Aid to debug this. Thanks!