search

GIScience / orstools-qgis-plugin

Plugin for QGIS with a set of tools to use openrouteservice API´s, based on openstreetmap
https://plugins.qgis.org/plugins/ORStools/
MIT License
91 stars 31 forks source link

Python TypeError: can only concatenate str (not "bytes") to str #263

Open Erudition opened 1 month ago

Erudition commented 1 month ago

Here's what I did

Calculating an isochrones layer, to form a heatmap of driving distance from/to each of the public Compressed Natural Gas (CNG) Stations in North America. This can determine the areas that I should not drive to with my CNG vehicle, as I would run out of fuel before I can get back to a station. I used increments of 10 miles, up to 150 (the theoretical max of my tank), so I needed to run the ORS tool locally.

QGIS version: 3.30.1-'s-Hertogenbosch
Qt version: 5.15.10
Python version: 3.10.7
GDAL version: 3.6.1
GEOS version: 3.12.1-CAPI-1.18.1
PROJ version: Rel. 9.3.1, December 1st, 2023
Algorithm started at: 2024-05-30T01:45:32
Algorithm 'Isochrones from Point-Layer' starting…
Input parameters:
{ 'INPUT_AVOID_BORDERS' : None, 'INPUT_AVOID_COUNTRIES' : '', 'INPUT_AVOID_FEATURES' : [], 'INPUT_AVOID_POLYGONS' : None, 'INPUT_FIELD' : 'ID', 'INPUT_METRIC' : 1, 'INPUT_POINT_LAYER' : 'delimitedtext://file:///home/adroit/Downloads/alt_fuel_stations%20(May%2030%202024).csv?type=csv&maxFields=10000&detectTypes=yes&xField=Longitude&yField=Latitude&crs=EPSG:4326&spatialIndex=no&subsetIndex=no&watchFile=no', 'INPUT_PROFILE' : 0, 'INPUT_PROVIDER' : 1, 'INPUT_RANGES' : '16093,32187,48280,64374,80467,96561,112654,128748,144841,160934,177028,193121,209215,225308,241402\n', 'INPUT_SMOOTHING' : None, 'LOCATION_TYPE' : 0, 'OUTPUT' : '/media/adroit/Extreme SSD/Mapping/station isochrones.gpkg' }

Traceback (most recent call last):
File "/home/adroit/.local/share/QGIS/QGIS3/profiles/default/python/plugins/ORStools/proc/isochrones_layer_proc.py", line 218, in postProcessAlgorithm
self.isochrones.stylePoly(processed_layer)
File "/home/adroit/.local/share/QGIS/QGIS3/profiles/default/python/plugins/ORStools/common/isochrones_core.py", line 184, in stylePoly
field = layer.fields().indexOf(self.field_dimension_name)
AttributeError: 'NoneType' object has no attribute 'fields'

Execution completed in 13221.80 seconds (3 hours 40 minutes 22 seconds)
Results:
{'OUTPUT': '/media/adroit/Extreme SSD/Mapping/station isochrones.gpkg'}

Loading resulting layers
The following layers were not correctly generated.
• /media/adroit/Extreme SSD/Mapping/station isochrones.gpkg
You can check the 'Log Messages Panel' in QGIS main window to find more information about the execution of the algorithm.

The resulting database was not openable, all I could determine was this Python warning which repeated hundreds of times:

2024-05-30T01:46:29     WARNING    An error has occurred while executing Python code:

             TypeError: can only concatenate str (not "bytes") to str

             Traceback (most recent call last):
              File "/home/adroit/.local/share/QGIS/QGIS3/profiles/default/python/plugins/ORStools/common/networkaccessmanager.py", line 318, in replyFinished
              errString = self.reply.errorString() + self.http_call_result.content
             TypeError: can only concatenate str (not "bytes") to str

Here's what I got

This Python error was repeated in the Log Messages Panel:

2024-05-30T01:42:47     WARNING    An error has occurred while executing Python code:

             RuntimeError: wrapped C/C++ object of type QNetworkReply has been deleted

             Traceback (most recent call last):
              File "/home/adroit/.local/share/QGIS/QGIS3/profiles/default/python/plugins/ORStools/common/networkaccessmanager.py", line 297, in replyFinished
              err = self.reply.error()
             RuntimeError: wrapped C/C++ object of type QNetworkReply has been deleted

             Python version: 3.10.7 (main, Jan 1 1970, 00:00:01) [GCC 11.3.0]

             QGIS version: 3.30.1-'s-Hertogenbosch 's-Hertogenbosch, exported

             Python Path:

             /gnu/store/b7c544lbz5xpm0dalvijgn9angarvkpy-qgis-3.30.1/share/qgis/python/home/adroit/.local/share/QGIS/QGIS3/profiles/default/python/home/adroit/.local/share/QGIS/QGIS3/profiles/default/python/plugins/gnu/store/b7c544lbz5xpm0dalvijgn9angarvkpy-qgis-3.30.1/share/qgis/python/plugins/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python310.zip/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/lib-dynload/home/adroit/.local/lib/python3.10/site-packages/gnu/store/5bi15zfjs6a6ysvnwa1hrd9brdcxbc8d-python-mock-3.0.5/lib/python3.10/site-packages/gnu/store/5y9dr12rmhl9ga4w7gipavnf2j4nblk1-python-nose2-0.14.0/lib/python3.10/site-packages/gnu/store/pjvzl76bvc32rrxn570pj73k91qb263c-python-pyqt-builder-1.14.1/lib/python3.10/site-packages/gnu/store/mlrspvcdbs16c35g4cbl9iq8j1wfgf53-gdal-3.6.1/lib/python3.10/site-packages/gnu/store/3lxr2xg3yscdb3979blgjg0h7xd1n9la-python-3.10.7/lib/python3.10/site-packages/gnu/store/43nkqc392hpng6davs0xh4n1pkyppmq3-python-chardet-5.1.0/lib/python3.10/site-packages/gnu/store/i98syd9c13qfh1qc00s25mw7xiqsz9x6-python-dateutil-2.8.2/lib/python3.10/site-packages/gnu/store/ag8fpifhzmgz835ki4rknwnnwp1mq21r-python-future-0.18.2/lib/python3.10/site-packages/gnu/store/m64a0db46h4fbz79g6hjcq5w0vivr9mq-python-jinja2-3.1.1/lib/python3.10/site-packages/gnu/store/v3kh2k7b5w8x1yn22ri1g3dww44nc4y7-python-numpy-1.23.2/lib/python3.10/site-packages/gnu/store/hmiqdc14axychpcj338m0ks0kdgd0bir-python-owslib-0.19.2/lib/python3.10/site-packages/gnu/store/8amhnpfwjdqgr86k1wpl0c4c8nfs2h41-python-psycopg2-2.9.6/lib/python3.10/site-packages/gnu/store/ybg64ixq12wp4bpkifjaknflflcizaid-python-pygments-2.12.0/lib/python3.10/site-packages/gnu/store/psd72fb2lzllfnlm82qiplwqgma0cs6y-python-pyqt+qscintilla-5.15.9/lib/python3.10/site-packages/gnu/store/6lws6dmqncq04pw8faikrgs8wrbvvi9w-python-pytz-2022.1/lib/python3.10/site-packages/gnu/store/1jgpkbn7kd0gzsflw5dc5fic4d12rw02-python-pyyaml-6.0/lib/python3.10/site-packages/gnu/store/mz580v5si2bv3lqz38gbvk0afkfzddih-python-requests-2.28.1/lib/python3.10/site-packages/gnu/store/60khhb9j8gv8vwry3dkhv41x0nz598az-python-sip-6.7.7/lib/python3.10/site-packages/gnu/store/9knaypzgdd331mldnhi8n8nhijhj2dvp-python-six-1.16.0/lib/python3.10/site-packages/gnu/store/lrrb707g7magil7q35pbxv1h87kawfnr-python-urllib3-1.26.15/lib/python3.10/site-packages/gnu/store/0cn3b0z1a08rk0gl57xy4ipzl5y92s34-python-markupsafe-2.1.1/lib/python3.10/site-packages/gnu/store/ihnvc3cika8f6km0fwlnzjvc46av2ids-python-pyqt5-sip-12.11.1/lib/python3.10/site-packages/gnu/store/jv0k2gc0igz05midwapghg0h0jbm2vlj-python-idna-3.3/lib/python3.10/site-packages/gnu/store/k8a7zh8k9y4gpryfsk62467phh310qx9-python-charset-normalizer-2.1.0/lib/python3.10/site-packages/gnu/store/09v5d8wmp97876h9xsj4jl2fwbmlxb3i-python-certifi-2022.6.15/lib/python3.10/site-packages/gnu/store/0lk4szvb8rx6hjnncai4hy692msg1qxj-python-ply-3.11/lib/python3.10/site-packages/gnu/store/4ifgdvhqad0nv4gq1vbx36af9bxkk9fx-python-packaging-21.3/lib/python3.10/site-packages/gnu/store/w39id5lliy29l3kja3yf3hdczq7nb8fp-python-toml-0.10.2/lib/python3.10/site-packages/gnu/store/cxiy0418wz03lp6s3fxvm0dn67xqw02w-python-pysocks-1.7.1/lib/python3.10/site-packages/gnu/store/k96j15c36z8vsdh8c0q6hckyk998d1sz-python-pyopenssl-23.2.0/lib/python3.10/site-packages/gnu/store/qlvy7vx7vb4ms7x1g1v8yyclnqhjnddk-python-cryptography-41.0.4/lib/python3.10/site-packages/gnu/store/rii148j8pd3k1jv0znlhpihnakqcarw3-python-pyparsing-3.0.6/lib/python3.10/site-packages/gnu/store/vxzlha6s0168wswz7casagail30890bm-python-cffi-1.15.1/lib/python3.10/site-packages/gnu/store/21khkz9nyp0r7smz15rhfadhjijrav0x-python-pycparser-2.21/lib/python3.10/site-packages/home/adroit/.guix-profile/lib/python3.10/site-packages/run/current-system/profile/lib/python3.10/site-packages/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/home/adroit/.local/share/QGIS/QGIS3/profiles/default/python/home/adroit/Documents

2024-05-30T01:46:29     WARNING    An error has occurred while executing Python code:

             TypeError: can only concatenate str (not "bytes") to str

             Traceback (most recent call last):
              File "/home/adroit/.local/share/QGIS/QGIS3/profiles/default/python/plugins/ORStools/common/networkaccessmanager.py", line 318, in replyFinished
              errString = self.reply.errorString() + self.http_call_result.content
             TypeError: can only concatenate str (not "bytes") to str

             Python version: 3.10.7 (main, Jan 1 1970, 00:00:01) [GCC 11.3.0]

             QGIS version: 3.30.1-'s-Hertogenbosch 's-Hertogenbosch, exported

             Python Path:

             /gnu/store/b7c544lbz5xpm0dalvijgn9angarvkpy-qgis-3.30.1/share/qgis/python/home/adroit/.local/share/QGIS/QGIS3/profiles/default/python/home/adroit/.local/share/QGIS/QGIS3/profiles/default/python/plugins/gnu/store/b7c544lbz5xpm0dalvijgn9angarvkpy-qgis-3.30.1/share/qgis/python/plugins/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python310.zip/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/lib-dynload/home/adroit/.local/lib/python3.10/site-packages/gnu/store/5bi15zfjs6a6ysvnwa1hrd9brdcxbc8d-python-mock-3.0.5/lib/python3.10/site-packages/gnu/store/5y9dr12rmhl9ga4w7gipavnf2j4nblk1-python-nose2-0.14.0/lib/python3.10/site-packages/gnu/store/pjvzl76bvc32rrxn570pj73k91qb263c-python-pyqt-builder-1.14.1/lib/python3.10/site-packages/gnu/store/mlrspvcdbs16c35g4cbl9iq8j1wfgf53-gdal-3.6.1/lib/python3.10/site-packages/gnu/store/3lxr2xg3yscdb3979blgjg0h7xd1n9la-python-3.10.7/lib/python3.10/site-packages/gnu/store/43nkqc392hpng6davs0xh4n1pkyppmq3-python-chardet-5.1.0/lib/python3.10/site-packages/gnu/store/i98syd9c13qfh1qc00s25mw7xiqsz9x6-python-dateutil-2.8.2/lib/python3.10/site-packages/gnu/store/ag8fpifhzmgz835ki4rknwnnwp1mq21r-python-future-0.18.2/lib/python3.10/site-packages/gnu/store/m64a0db46h4fbz79g6hjcq5w0vivr9mq-python-jinja2-3.1.1/lib/python3.10/site-packages/gnu/store/v3kh2k7b5w8x1yn22ri1g3dww44nc4y7-python-numpy-1.23.2/lib/python3.10/site-packages/gnu/store/hmiqdc14axychpcj338m0ks0kdgd0bir-python-owslib-0.19.2/lib/python3.10/site-packages/gnu/store/8amhnpfwjdqgr86k1wpl0c4c8nfs2h41-python-psycopg2-2.9.6/lib/python3.10/site-packages/gnu/store/ybg64ixq12wp4bpkifjaknflflcizaid-python-pygments-2.12.0/lib/python3.10/site-packages/gnu/store/psd72fb2lzllfnlm82qiplwqgma0cs6y-python-pyqt+qscintilla-5.15.9/lib/python3.10/site-packages/gnu/store/6lws6dmqncq04pw8faikrgs8wrbvvi9w-python-pytz-2022.1/lib/python3.10/site-packages/gnu/store/1jgpkbn7kd0gzsflw5dc5fic4d12rw02-python-pyyaml-6.0/lib/python3.10/site-packages/gnu/store/mz580v5si2bv3lqz38gbvk0afkfzddih-python-requests-2.28.1/lib/python3.10/site-packages/gnu/store/60khhb9j8gv8vwry3dkhv41x0nz598az-python-sip-6.7.7/lib/python3.10/site-packages/gnu/store/9knaypzgdd331mldnhi8n8nhijhj2dvp-python-six-1.16.0/lib/python3.10/site-packages/gnu/store/lrrb707g7magil7q35pbxv1h87kawfnr-python-urllib3-1.26.15/lib/python3.10/site-packages/gnu/store/0cn3b0z1a08rk0gl57xy4ipzl5y92s34-python-markupsafe-2.1.1/lib/python3.10/site-packages/gnu/store/ihnvc3cika8f6km0fwlnzjvc46av2ids-python-pyqt5-sip-12.11.1/lib/python3.10/site-packages/gnu/store/jv0k2gc0igz05midwapghg0h0jbm2vlj-python-idna-3.3/lib/python3.10/site-packages/gnu/store/k8a7zh8k9y4gpryfsk62467phh310qx9-python-charset-normalizer-2.1.0/lib/python3.10/site-packages/gnu/store/09v5d8wmp97876h9xsj4jl2fwbmlxb3i-python-certifi-2022.6.15/lib/python3.10/site-packages/gnu/store/0lk4szvb8rx6hjnncai4hy692msg1qxj-python-ply-3.11/lib/python3.10/site-packages/gnu/store/4ifgdvhqad0nv4gq1vbx36af9bxkk9fx-python-packaging-21.3/lib/python3.10/site-packages/gnu/store/w39id5lliy29l3kja3yf3hdczq7nb8fp-python-toml-0.10.2/lib/python3.10/site-packages/gnu/store/cxiy0418wz03lp6s3fxvm0dn67xqw02w-python-pysocks-1.7.1/lib/python3.10/site-packages/gnu/store/k96j15c36z8vsdh8c0q6hckyk998d1sz-python-pyopenssl-23.2.0/lib/python3.10/site-packages/gnu/store/qlvy7vx7vb4ms7x1g1v8yyclnqhjnddk-python-cryptography-41.0.4/lib/python3.10/site-packages/gnu/store/rii148j8pd3k1jv0znlhpihnakqcarw3-python-pyparsing-3.0.6/lib/python3.10/site-packages/gnu/store/vxzlha6s0168wswz7casagail30890bm-python-cffi-1.15.1/lib/python3.10/site-packages/gnu/store/21khkz9nyp0r7smz15rhfadhjijrav0x-python-pycparser-2.21/lib/python3.10/site-packages/home/adroit/.guix-profile/lib/python3.10/site-packages/run/current-system/profile/lib/python3.10/site-packages/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/home/adroit/.local/share/QGIS/QGIS3/profiles/default/python/home/adroit/Documents

2024-05-30T01:46:29     WARNING    An error has occurred while executing Python code:

             TypeError: can only concatenate str (not "bytes") to str

             Traceback (most recent call last):
              File "/home/adroit/.local/share/QGIS/QGIS3/profiles/default/python/plugins/ORStools/common/networkaccessmanager.py", line 318, in replyFinished
              errString = self.reply.errorString() + self.http_call_result.content
             TypeError: can only concatenate str (not "bytes") to str

             Python version: 3.10.7 (main, Jan 1 1970, 00:00:01) [GCC 11.3.0]

             QGIS version: 3.30.1-'s-Hertogenbosch 's-Hertogenbosch, exported

             Python Path: ...

Here's what I was expecting

A valid gpkg file produced and a rendered heatmap layer.

2024-05-30T11:21:09     INFO    There was an error querying the database /media/adroit/Extreme SSD/Mapping/station-isochrones.gpkg: malformed database schema (station isochrones) - invalid rootpage

Here's what I think could be improved

Erudition commented 1 month ago

Here's the offending line: https://github.com/GIScience/orstools-qgis-plugin/blob/2b981c177fc67ba9f0ea7b05d53a13d1298610fc/ORStools/common/networkaccessmanager.py#L318 It seems to be concatenating the error message (string) with the reply data which is apparently bytes.

I tried changing it to just:

                errString = self.reply.errorString()

Which moved the error:

RuntimeError: wrapped C/C++ object of type QNetworkReply has been deleted

             Traceback (most recent call last):
              File "/home/adroit/.local/share/QGIS/QGIS3/profiles/default/python/plugins/ORStools/common/networkaccessmanager.py", line 297, in replyFinished
              err = self.reply.error()
             RuntimeError: wrapped C/C++ object of type QNetworkReply has been deleted

to an earlier line: https://github.com/GIScience/orstools-qgis-plugin/blob/2b981c177fc67ba9f0ea7b05d53a13d1298610fc/ORStools/common/networkaccessmanager.py#L297

merydian commented 3 weeks ago

Hi, interesting use case. You're running your own local ORS instance? Have you set maximum_intervals to a custom value? This is hard to reproduce without more information on your setup. Please provide us with some more.

Best regards

Erudition commented 3 weeks ago

Yep, running it locally as stated. Can't process isochrones this big with the public instance.

Darn, I was hoping we could improve the durability of the code based on these details alone, given that asking concatenating bytes to a string is apparently possible currently and shouldn't be. I've changed my setup significantly in the time before your reply, and I managed to get it to work by changing the OSM pbr file (evidently some stations were in Canada whereas the OSM data I had was only the US). The error has appeared in several other cases with varying setup, I'll see if I can remember the way my config file was set when it triggered so you can reproduce it.