QGIS crashes with Google Earth XYZ tile

[geokimbo]

As of a couple of days ago an update has broken QGis when a Google earth XYZ tile is visible. The issue appears to be scale related as I can work happily at 1:30000 without crashing but at 1:12000 it will reliably crash. I think the issue is related to a permissions failure on /usr/share/qgis/resources/data/world_map.shp as this is reported to the syslog - see attached grep "qgis" report. The references to double commander will be because I am invoking QGis by association with a qgs file from the double commander file manager. /usr/share/qgis/resources/data/world_map.shp is not a file I have generated nor is it listed in the qgs file so it is presumably an internal QGis file. It is owned by root and has 644 permissions. The directories from the root down to that file are all owned by root and have 755 permissions. I have 3 similarly configured Ubuntu 18:04 machines running QGis. We all use QGis daily. One user noticed this issue late last week, I noticed it today and the third user is yet to report it. We are working on projects that worked fine a few weeks ago but as a test have also created a project from scratch and still see the problem. Turning off the visibility of the google earth layer stops the crashing.

System details: Dell poweredge T610 - dual xeon hexacores, 48 Gb RAM, Ubuntu 18:04 QGIS 3.8.1 code revision dcd95cc648 Extract from syslog and the qgs file - renamed *.txt to allow it to be uploaded There are a bunch of QGIS3 files in /tmp but all are empty as are the corresponding directories.

overview.txt

syslog_grep_qgis.txt Cheers Kim

[gioman]

Just tested this

http://www.google.cn/maps/vt?lyrs%3Ds@189%26gl%3Dcn%26x%3D%7Bx%7D%26y%3D%7By%7D%26z%3D%7Bz%7D&zmax=19&zmin=0

on 3.8.1 and no crashes.

[geokimbo]

Nor did ours last week. This is related to either a QGis update we received last week or an update to one of the ubuntu or system libraries.

[gioman]

This is related to either a QGis update

I tested the same version, but not on the same platform. Will test on Ubuntu 18.04 later.

[geokimbo]

The reference above to /usr/share/qgis/resources/data/world_map.shp is a red herring. That is the world map which comes up when you select the CRS and get the pink coverage band in the picture in the lower right of the CRS dialogue. We no longer see it. I seem to recall a similar issue happening several versions ago and it was related to a changed folder permission in /usr - can't recall which one but one of the updates (not necessarily QGis) had changed the ownership and this was the result. I'll check if it has happened again.

[geokimbo]

A dead end. everything appears to be owned by root as expected so it looks like it wasn't a repeat of that issue. It looks like the GLib-ERROR **: 14:53:08.718: Creating pipes for GWakeup: Too many open files error may be more useful. The best reference I can find to that suggests it is caused by resource leakage. Something for someone more familiar with the code than me to look at.

[geokimbo]

Just forced a crash by having several rasters turned on and moving the screen (with the fist) quickly several times forcing redraws. That would be consistent with leakage and perhaps the reason it happens all the time at low scales with Google earth is because the image is regularly being redrawn as it is moved, rescaled and exposed???

[gioman]

Just forced a crash by having several rasters turned on and moving the screen (with the fist) quickly several times forcing redraws. That would be consistent with leakage and perhaps the reason it happens all the time at low scales with Google earth is because the image is regularly being redrawn as it is moved, rescaled and exposed???

can you attach a screencast that shows how to get tge crash? I'm not able to replicate it?

[geokimbo]

Thanks Geovanni

Using the QGis project I posted in my initial post here is a screen capture of the process to crash as well as the terminal output. The terminal output is pretty much the same as that I found when searching for the GLib Error. Although that referred to a different program (Albert??) QGis_crash.zip the cause is likely the same - resource leakage. I had to rename the video from .mkv to .txt to allow it to upload obviously you'll have to reverse the process. I tried zipping it as zip files were supposed to be OK but although it appeared to be uploading nothing is showing in this entry dialogue. terminal_output.txt QGis_crash.txt

Cheers Kim

[geokimbo]

Ahh There is is half way up!

[geokimbo]

In case it adds value here is an strace output strace.zip

[gioman]

Have you tried with a new QGIS profile? no 3rd party plugins installed.

[geokimbo]

I have not. I did try with a new QGis project with only a boundary shape file and Google Earth loaded (plus my existing plugins which there are only 9 active of 13 installed) Using that on my home machine, which has identical hardware to the 3 work machines, I could not get it to crash, perhaps I did not try for long enough?. However I note that my home machine seems to be a bit more forgiving with the crashing than the office machine so I'll give it another go on Monday from the office. I'll also try using another plugin free profile. The inability to crash it using just a GE tile and a single shape file on the same machine that crashes constantly with several layers visible and many layers in the project still points me to a resource leakage issue. I just tried opening the project I have been working on above and turned everything off (non visible) except for a single boundary shape file and google earth. It took me a lot more rapid moving in order to crash it but I finally did. This time the repeated messages "Warning: QNetworkDiskCache::prepare() unable to open temporary file" were missing from the terminal output but otherwise it was identical to the previous terminal output.

[gioman]

I have not.

do it please.

[geokimbo]

I created a new user and started QGis afresh from that profile on my office machine . No plugins other than those which are loaded by default. I opened the project above and turned on the google earth layer. Zoomed in to ~1:1500 and started draging the screen around. It took a bit longer than I recall but it locked up and then crashed. Terminal output attached. qgis_crash.txt

[gioman]

Please try:

only with the GE xyz layer

with your project but removing (group of) layers

we must understand if the issue is in the xyz layer itself or another layer in your specific project.

[geokimbo]

Not exactly what I think you are asking but I turned off all layers other than GE so only one layer visible. It took more work to crash it but I got there. As noted above I don't thing the issue is restricted to the GE XYZ tool. It happens when i have rasters turned on and force a lot of rapid redraws. The more rasters on the quicker it happens. here's the terminal output from only a GE layer visible. qgis_terminal.txt

[gioman]

ut I turned off all layers other than GE

turning them off may not be enough, you should remove them.

[geokimbo]

OK Geovanni. As the logical conclusion to this is to iterate to the hopefully, one bad layer which in concert with GE or other rasters is causing the crash and as there are a hundred or so objects in the project that will take a few hours of trial and error, I'm not going to be in a position to deliver this until later in the week as I am already doing 16 hour days trying unsuccessfully to keep up and have bugger all spare time. I'll try and slot it in progressively over coffee breaks.

I'd note here that the problem is not specific to this project but that all our projects have many layers - We use QGis as a digital light table to overlay many datasets in order to create an interpretation that is sustainable. It makes sense to just continue with this project to get to the bottom of the problem.

I would have thought that with the code, the screen video and the strace I supplied someone familiar with the code could go to the right line fairly easily and see what was happening?? I know I can with my code which has many more lines than QGis..

Cheers Kim

[roya0045]

I rapidly glanced over the terminal, there seems to be a lot of error in relation to permissions and network. Is it possibly that you could have some network constraint at work that you prevent either sending or receiving packets too rapidly?

Maybe something like DoS protection?

[geokimbo]

At the office I have a ClearOS gateway so is possible that it has DoS features although I don't recall seeing an option for it in the setup. At home I just have the ISP supplied router which does not have it. We have plenty of bandwidth (100Mbit connection at the office) so the latency of the LAN cabeling (CAT5 and 5E) would be approaching that of the WAN

I still have not has a long enough break to follow through on Geovanni's request but at worst should have time this weekend.

[geokimbo]

Also I was able to repeat the crash with GE turned off and a bunch of rasters living locally on the machine turned on so I doubt that the network is the issue.

[gioman]

Also I was able to repeat the crash with GE turned off

so maybe it isn't a XYZ layer problem after all.

[geokimbo]

No I believe it is most likely a raster drawing problem with an overflow occurring if the re-draw rate is faster than the clean up rate but that is just a guess without having seen the code. I'm now thinking that GE crashing is likely a symptom rather than a cause.

[roya0045]

Possibly, but you said you could not replicate it on your other machine. Otherwise could you make a small/video gif, just to see and to ensure similar replication. (something like the number and size of raster would also help since smaller rasters are drawn faster it would be good to have an idea of the necessary workload to reproduce or rescale.)

Another option would be for you to compile qgis with debugging and provide the crash but the leak would be hard to detect that way.

[geokimbo]

I have tried without any success to replicate the crash without GE turned on on both the office and home machines. Not sure what happened last week but its probably one of those pesky intermittent bugs which are very hard to track. The GE crash is at least consistent. I hope to have time this weekend to pull apart the project and delete layers until it no longer crashes, this weekend.

[geokimbo]

I haven't forgotten you just been time poor. I will still get to test this but can't promise when that will be. Sorry.

[geokimbo]

Not a lot of useful info to report I'm afraid. The behavior appears to be totally inconsistent.

I copied the original project to test.qgs

I opened up test.qgs and deleted the bottom 11 layers (but 1) in the Layer manager. The lowest layer was Google Earth so I deleted 11 layers above it. I saved the project with a new name (Test2) and then tried to crash it without success.

I reopened test.qgs and deleted the top 40 layers, some of which were inside groups. The lower 11 that I had deleted in step 1 were still there. I saved the project with a new name. (Test3). I could not crash it.

Thinking it might just be a case of resaving the project with a new name I tried that (test4) - still crashes.

Starting at the top I deleted a the first layer, saved to a new name and crashed it. I continued that cumulatively deleting one more layer or group each time until I got to the 9th layer/group and found I could not crash it. Test12.qgs = test.qgs less the top 9 layers

I then reopened test.qgs and just deleted the group that was at position 9 on the list from the top which was a group of points loaded as delimited text and plotted with category symbology but turned off. Saved to test13 and was able to crash it.

So the short summary is; test minus the bottom 11 layers - no crash test minus the top 40 layers - no crash test resaved as test 3 - crashed test minus the top 1 layers - crash test minus the top 2 layers - crash . . . test minus the top 8 layers - crash test minus the top 9 layers - no crash test minus the layer 9 - crash

The project has 739 layers, many in groups and some groups having sub-groups . I updated to 3.8.2 today and was hopeful that might fix things but alas.

Attached is the terminal output. The warnings at the top were common to all runs. crash or not. The crash looks to be to do with the GLib error. I googled it and there is a world of hits on it which were nicely summarised in one thread as

"Search for "Creating pipes for GWakeup: Too many open files" in Google it returns many results. Most of them have concluded that this is due to the main application having too many files open. So it may be actually true for your situation if you're operating these many torrents/files. You may be hitting an OS limitation" That would be a function of Qgis trying to create a temporary bitmap file of the raster (GE in this case) but because I was moving the view field too fast it was flooding the write buffer before each file could be created. That all makes sense but the reason I started on this error wasn't because I was moving the field of view about like a madman I just moved it a few times. Moving it like a madman is just a semi-guaranteed way to reproduce it without risking losing half an hours work. [qgis-crash-terminal.txt](https://github.com/qgis/QGIS/files/3512095/qgis-crash-terminal.txt)

[roya0045]

Is GE the only file that is not local? Or are the other files on servers too?

[geokimbo]

GE is the only remote file. All others local.

Note however that on one occasion I was able to repeat the crash with GE turned off and half a dozen local ones turned on. It may be easier to force with a remote raster because of the network latency adding to the write buffer delay??

[roya0045]

I was simply curious due to the QNetworkDiskCache::prepare() unable to open temporary file in your log.

[geokimbo]

Possibly Alex. I just tried repeating the exercise again with several local rasters on and GE off and can't get the warning so it looks like QNetworkDiskCache only applies to www files. According to the documentation for QNetworkDiskCache its default max cache size is 50MB, although that can be changed by :setMaximumCacheSize. 50Mb wouldn't give you many views of a detailed raster so presumably the code does resize it prior to use. They recommend that it be checked and flushed with :expire

[roya0045]

Ok, so having a high number of files and always creating and fetching a temporary from a web source could be the cause as established previously. During your tests did you notice if 'heavier' files were more linked to crashes? You said you had 700+ layers and that a somewhat 'random' removal would disable the crashes. I was curious if the fetching could be limited by total size rather than sheer number in this case.

[geokimbo]

Always possible but I doubt that it is our source as most of the meat was in the middle of the layer manager. The bottom 11 were all rasters - by and large registered scans from jpgs and pdfs. Registered using the georeferencer plugin. The top 40 were either vectors (all pretty sparse) or delimited text, many of which were classified with a category or gradational symbology. Also note that although there at >700 layers only 3-5 are turned on when I crash things. For the tests the only objects turned on were GE and a couple of vector layers and a delimited text point file with labels.

[gioman]

@geokimbo please test again with the latest 3.10 or 3.12.

[geokimbo]

I've been using 3.12 for some time and have not noted this problem for a while. However I haven't been as busy with QGis recently so it may just be lurking in the shadows waiting to jump. I'll try on the work machine on Monday as it seems more prone to crashing than home does.

[geokimbo]

Yes it still fails on 3.12.3

Here's the terminal output

ERROR 6: EPSG PCS/GCS code 37004 not found in EPSG support files. Is this a valid EPSG coordinate system? Warning 4: Failed to open /usr/share/qgis/resources/data/world_map.gpkg, Permission denied. Warning 4: Failed to open /usr/share/qgis/resources/data/world_map.gpkg, Permission denied. Warning 4: Failed to open /usr/share/qgis/resources/data/world_map.gpkg, Permission denied. Warning 4: Failed to open /usr/share/qgis/resources/data/world_map.gpkg, Permission denied. Warning 4: Failed to open /usr/share/qgis/resources/data/world_map.gpkg, Permission denied. Warning 4: Failed to open /usr/share/qgis/resources/data/world_map.gpkg, Permission denied. Warning: QNetworkDiskCache::prepare() unable to open temporary file

(qgis.bin:6968): GLib-ERROR **: 10:35:34.596: Creating pipes for GWakeup: Too many open files

QGIS died on signal 5Could not attach to process. If your uid matches the uid of the target process, check the setting of /proc/sys/kernel/yama/ptrace_scope, or try again as the root user. For more details, see /etc/sysctl.d/10-ptrace.conf ptrace: Operation not permitted. No thread selected No stack. gdb returned 0 Aborted (core dumped)

/etc/sysctl.d/10-ptrace.conf just had kernel.yama.ptrace_scope = 1

I couldn't get it to crash using rasters that were stored locally but as soon as I tuned on the GE layer and did a quick move it crashed. I wound the scale up to about 1:4000 as that tends to help make the problem repeatable.

A fix for the missing EPSG CRS would be welcome as it comes up every time I start QGis. However I haven't spent any time trying to track it down.

[gioman]

(qgis.bin:6968): GLib-ERROR **: 10:35:34.596: Creating pipes for GWakeup: Too many open files

@geokimbo can you try raise the limit of number of files that your system can open?

[geokimbo]

Giovani - I used ulimit -n to double the value from 1024 to 2048 and QGis was stable on test. I then reset ulimit -n back to 1024 and QGis failed as previously so it looks like you are on track

[gioman]

Giovani - I used ulimit -n to double the value from 1024 to 2048 and QGis was stable on test. I then reset ulimit -n back to 1024 and QGis failed as previously so it looks like you are on track

@geokimbo if this is the case I'm not sure there is anything to be fixed in QGIS.

[geokimbo]

Perhaps, but the links I found to it all point to it being a leakage problem. Either a straight memory leak or a process spawning child processes and not killing them or tidying up when done with them. raising ulimit may fix the symptom but doesn't sound like it is solving the problem.

[Pedro-Murteira]

@geokimbo Hello, i've been trying to reproduce this issue on the recent versions: 3.22.4 and 3.24.0 but I'm not being able to. Did you come across this issue more recently?

[geokimbo]

No I haven't but nor have I revisited that particular project (assumes it is project dependant?) It has been so long I've forgotten what the issue was:-0