Open jazz-it opened 5 years ago
I don't think this is an "incompatibility issue" with GTK - the warnings being thrown are caused by your GTK theme being slightly out of date.
Can you reproduce this crash consistently? What if you run variety in gdb?
Yes, the crash is consistent. Reg. gdb, how do you want me to test it? I tried with the following:
$ gdb GNU gdb (GDB) 8.2.1 Copyright (C) 2018 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-pc-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: http://www.gnu.org/software/gdb/bugs/. Find the GDB manual and other documentation resources online at: http://www.gnu.org/software/gdb/documentation/.
For help, type "help". Type "apropos word" to search for commands related to "word". (gdb) source /usr/bin/variety /usr/bin/variety:18: Error in sourced command file: Undefined command: "import". Try "help". (gdb)
BTW, this theme is default for Manjaro.
I have the same problem. Ubuntu 18.04. It works fine until this evening.
I will remove this app. It seems unstable and the community is not very cooperative.
@madjoe Please don't be rude.
Neither of us (the maintainers) have been able to reproduce these crashes. The problem with pure Python code is that it shouldn't ever segfault, meaning the cause of these crashes are in underlying libraries that are out of our control.
Hi @jlu5
I can reproduce this crash consistently and is ready to help by providing any required information and can help to debug this issue.
System specs (tell me if you need more info):
Linux Mint 19 Tara on laptop.
uname -a reports:
4.15.0-47-generic # 50-Ubuntu SMP Wed Mar 13 10:44:52 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
It is running MATE desktop environment.
variety was installed through this PPA:
Origin: Peter Levi's PPA:18.04/bionic [all
http://ppa.launchpad.net/peterlevi/ppa/ubuntu/pool/main/v/variety/variety_0.7.1~git201810231556.944aa6d~ppa772~ubuntu1
Running variety executable on console yields this:
$ variety
double free or corruption (out)
Aborted (core dumped)
Running the same through GDB yields this:
$ gdb /usr/bin/python3
...
(gdb) run /usr/bin/variety
Starting program: /usr/bin/python3 /usr/bin/variety
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7fffe5f77700 (LWP 26326)]
[New Thread 0x7fffe5776700 (LWP 26327)]
[New Thread 0x7fffe4f75700 (LWP 26328)]
[New Thread 0x7fffd7fff700 (LWP 26332)]
[New Thread 0x7fffd77fe700 (LWP 26333)]
double free or corruption (out)
Thread 1 "python3" received signal SIGABRT, Aborted.
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
51 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0 0x00007ffff7a22e97 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1 0x00007ffff7a24801 in __GI_abort () at abort.c:79
#2 0x00007ffff7a6d897 in __libc_message (action=action@entry=do_abort, fmt=fmt@entry=0x7ffff7b9ab9a "%s\n")
at ../sysdeps/posix/libc_fatal.c:181
#3 0x00007ffff7a7490a in malloc_printerr (str=str@entry=0x7ffff7b9c870 "double free or corruption (out)") at malloc.c:5350
#4 0x00007ffff7a7be75 in _int_free (have_lock=0, p=0xe14030, av=0x7ffff7dcfc40 <main_arena>) at malloc.c:4278
#5 0x00007ffff7a7be75 in __GI___libc_free (mem=0xe14040) at malloc.c:3124
#6 0x00007ffff4b3be10 in () at /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0
#7 0x00007ffff468df15 in () at /usr/lib/python3/dist-packages/gi/_gi.cpython-36m-x86_64-linux-gnu.so
#8 0x00007ffff4689ede in () at /usr/lib/python3/dist-packages/gi/_gi.cpython-36m-x86_64-linux-gnu.so
#9 0x00007ffff468ba48 in () at /usr/lib/python3/dist-packages/gi/_gi.cpython-36m-x86_64-linux-gnu.so
#10 0x00007ffff467fcb9 in () at /usr/lib/python3/dist-packages/gi/_gi.cpython-36m-x86_64-linux-gnu.so
#11 0x00000000005a730c in _PyObject_FastCallKeywords ()
#12 0x0000000000503073 in ()
#13 0x0000000000506859 in _PyEval_EvalFrameDefault ()
#14 0x0000000000501945 in _PyFunction_FastCallDict ()
#15 0x0000000000591461 in ()
#16 0x000000000059ebbe in PyObject_Call ()
#17 0x0000000000545068 in ()
#18 0x0000000000506b39 in _PyEval_EvalFrameDefault ()
#19 0x0000000000502209 in ()
#20 0x0000000000502f3d in ()
#21 0x0000000000506859 in _PyEval_EvalFrameDefault ()
#22 0x0000000000504c28 in ()
#23 0x0000000000502540 in ()
#24 0x0000000000502f3d in ()
#25 0x0000000000507641 in _PyEval_EvalFrameDefault ()
#26 0x0000000000504c28 in ()
#27 0x0000000000502540 in ()
#28 0x0000000000502f3d in ()
#29 0x0000000000506859 in _PyEval_EvalFrameDefault ()
#30 0x0000000000502209 in ()
#31 0x0000000000502f3d in ()
#32 0x0000000000506859 in _PyEval_EvalFrameDefault ()
#33 0x0000000000504c28 in ()
#34 0x0000000000506393 in PyEval_EvalCode ()
#35 0x0000000000634d52 in ()
#36 0x0000000000634e0a in PyRun_FileExFlags ()
#37 0x00000000006385c8 in PyRun_SimpleFileExFlags ()
#38 0x000000000063915a in Py_Main ()
#39 0x00000000004a6f10 in main ()
I've also run the same executable under valgrind:
$ valgrind --log-file=./variety.valgrind.log --leak-check=full --track-origins=yes --expensive-definedness-checks=yes variety
The result of the above command (file variety.valgrind.log is attached here (would be too long to paste as text).
Note: The crash does not reproduce while running under valgrind! The app starts and during the run, I manually exited it by right-clicking on the tray icon and selecting "exit".
However, an interesting message is received on console during this valgrind run:
(variety:27045): Gdk-CRITICAL **: 13:11:10.863: gdk_window_thaw_toplevel_updates: assertion 'window->update_and_descendants_freeze_count > 0' failed
Attaching the valgrind logs file: variety.valgrind.log.gz
Please, feel free to communicate with me for any additional information or debugging aid.
Thanks for maintaining the program!
Hello,
The gdb backtrace might be slightly more informative if you install the debug symbols for libglib-2.0 (package libglib2.0-0-dbgsym).
Aside from that, which wallpaper sources are you using? Did you tweak your config to do anything special? Could there potentially be corrupt images in one of your sources? (Possibly verify using https://stackoverflow.com/a/4780718)
Thanks. I already installed the debug info packages (for Python3 itself and libglib). Then it stopped reproducing. I don't think it is related to debug symbols, it's the nature of this kind of bug to only happen sometimes.
I will continue running the program under debug for as long as needed to try and find the cause. Will update here with my findings.
As for the sources, yes I played with them. I'll try to extract my config and share here when I find time for it...
Yeah.. sporadic bugs are way harder to fix than ones that happen consistently :(
Well, I checked all currently downloaded images using this script:
#!/usr/bin/python
from PIL import Image
import os
def check(fname):
try:
im = Image.open(fname)
im.verify()
return True
except Exception, e:
print "ERROR:", e
return False
def walk(path):
for name in os.listdir(path):
fullname = os.path.join(path, name)
if os.path.isfile(fullname):
yield fullname
elif os.path.isdir(fullname):
for subname in walk(fullname):
yield subname
variety_datadir = os.path.expanduser('~/.config/variety')
for fullname in walk(variety_datadir):
if fullname.endswith(".jpeg") or fullname.endswith(".jpg") or fullname.endswith(".gif") or fullname.endswith(".png"):
if not check(fullname):
print "BAD:", fullname
It shows that all currently downloaded images are considered fine.
On the other hand, the crash still didn't reproduce. So, when it does happen, I'll re-run this script to find out whether there are any corrupt images, in addition to looking at the gdb backtrace. That knowledge may help a bit deciphering this issue.
I'll also walk over the code maybe see something. Hope to find some time for that soon.
Will post any updates here.
Hi @jlu5
I have an additional thought about it. As I already mentioned, I use a laptop which is put into "sleep" several times a day (to conserve battery).
It could happen that if the system goes to sleep in the middle of variety downloading an image, the image may end up corrupted on disk somehow?
I did a quick look over variety Python code and didn't notice a validity check over downloaded images. I looked in the area of "save_locally" and "download_one" methods.
Maybe it's worth to add such a check in any case and skip (maybe even delete) image files that do not pass basic verification criteria. What do you think?
It might just be a nice feature anyways :) #145 is somewhat related
Yep. Although I'd also add some sort of "download failure counter" and give up after few attempts, because an image might be permanently unreachable for any number of reasons. Ok.
Regarding this crash (seemingly double free, or it could be buffer overrun/corruption), IMO it could be either corruption in downloaded data or a bug in underlying libraries such as GTK.
I started to notice these crashes about a month ago (maybe two). I use variety for much longer than that and the crash only started to happen recently.
Perhaps a difference in my setup is that I only use local folders when I'm not testing the downloaders. (This is just personal preference, I prefer curating my own collections)
Which sources are you using now?
Hi @jlu5
Attached files from my ~/.config/variety directory, including the variety.conf file that includes the list of sources I use.
I suggest diff-ing against your variety.conf to see the differences.
It also contains a recent log running with the -vvvvv verbose level. The log doesn't include any crashes yet, so not sure how useful it will be. Maybe you can spot something thought as I've seen several tracebacks there.
Note that I didn't include the actual image files because they take too much space and IMO not necessary at this stage.
Regarding what @jlu5 suggested, this solved the issue for me. I believe mine started to occur when I did an OS upgrade which might have caused a corrupted download during the upgrade.
I am also getting this when i run Variety 0.7.1: free(): invalid pointer Aborted
I have been using Variety for years. I wished that I new more, to help with resolving this. It seemed to quit working when some Ubuntu system updates took place.
I am willing to help, because this program is a big part of my daily desktop experience. Can someone suggest to this newbee, what details I can provide to assist with resolving this?
Much Thanks
I am also getting this when i run Variety 0.7.1: free(): invalid pointer Aborted
I have been using Variety for years. I wished that I new more, to help with resolving this. It seemed to quit working when some Ubuntu system updates took place.
I am willing to help, because this program is a big part of my daily desktop experience. Can someone suggest to this newbee, what details I can provide to assist with resolving this?
Much Thanks
I renamed $HOME/.config/variety/ to $HOME/.config/variety2/ Variety started working again for me.
I am also getting this when i run Variety 0.7.1: free(): invalid pointer Aborted I have been using Variety for years. I wished that I new more, to help with resolving this. It seemed to quit working when some Ubuntu system updates took place. I am willing to help, because this program is a big part of my daily desktop experience. Can someone suggest to this newbee, what details I can provide to assist with resolving this? Much Thanks
I renamed $HOME/.config/variety/ to $HOME/.config/variety2/ Variety started working again for me.
However, once I reboot - Variety stops working again...
(Copied from https://github.com/varietywalls/variety/issues/83#issuecomment-510909718)
Hi everyone,
What if you run Variety as python3 -q -X faulthandler /usr/bin/variety
in a terminal? This should print the last bits of Python code that are executed before Python crashes, which might narrow down which library calls are causing a crash.
(Copied from #83 (comment))
Hi everyone,
What if you run Variety as
python3 -q -X faulthandler /usr/bin/variety
in a terminal? This should print the last bits of Python code that are executed before Python crashes, which might narrow down which library calls are causing a crash.
It reproduced on me again today, so here we go!
$ python3 -q -X faulthandler /usr/bin/variety
free(): invalid pointer
Fatal Python error: Aborted
Thread 0x00007f99113e1700 (most recent call first):
File "/usr/lib/python3.6/threading.py", line 299 in wait
File "/usr/lib/python3.6/threading.py", line 551 in wait
File "/usr/lib/python3.6/threading.py", line 1180 in run
File "/usr/lib/python3.6/threading.py", line 916 in _bootstrap_inner
File "/usr/lib/python3.6/threading.py", line 884 in _bootstrap
Thread 0x00007f9911be2700 (most recent call first):
File "/usr/lib/python3/dist-packages/variety/VarietyWindow.py", line 857 in clock_thread_method
File "/usr/lib/python3.6/threading.py", line 864 in run
File "/usr/lib/python3.6/threading.py", line 916 in _bootstrap_inner
File "/usr/lib/python3.6/threading.py", line 884 in _bootstrap
Current thread 0x00007f9925710740 (most recent call first):
File "/usr/lib/python3/dist-packages/variety/Util.py", line 172 in __getitem__
File "/usr/lib/python3/dist-packages/variety/Util.py", line 477 in read_metadata
File "/usr/lib/python3/dist-packages/variety/VarietyWindow.py", line 663 in update_indicator
File "/usr/lib/python3/dist-packages/variety/VarietyWindow.py", line 177 in start
File "/usr/lib/python3/dist-packages/variety/__init__.py", line 198 in main
File "/usr/bin/variety", line 39 in <module>
Aborted (core dumped)
Which is, in class VarietyMetadata.get_tag_multiple()
:
class VarietyMetadata():
...
def __getitem__(self, key):
if self.has_tag(key):
if key in self.MULTIPLES:
return self.get_tag_multiple(key)
Here, in get_tag_multiple(key)
, which is defined in GExiv2.Metadata
: documented here
I've printed the value of key
just before this call. It was:
BEFORE_CRASH: calling get_tag_multiple(). key='Iptc.Application2.Keywords'
I've inspected the path (passed to VarietyMetadata.__init__()
, for the specific object that crashed).
The file was named /home/user/.config/variety/Downloaded/Desktoppr/02263_flash_1920x1080.jpg
. I am attaching that file here: 02263_flash_1920x1080.jpg.zip
I hope this would help you reproduce and fix the issue!
Also, if it helps, here are exact versions of exiv2-related libraries installed on my system:
# dpkg -l | grep exiv
ii gir1.2-gexiv2-0.10:amd64 0.10.8-1 amd64 GObject-based wrapper around the Exiv2 library - introspection data
ii libexiv2-14:amd64 0.25-3.1ubuntu0.18.04.3 amd64 EXIF/IPTC/XMP metadata manipulation library
ii libgexiv2-2:amd64 0.10.8-1 amd64 GObject-based wrapper around the Exiv2 library
I turned off - obtaining images from internet sources. I am using Variety just with images from local folders. And the problem went away....
OK YEP, I can reproduce this crash using the image @trackrx sent as well as the one linked in https://github.com/varietywalls/variety/issues/83#issuecomment-492881778
I'm using this little test case:
#!/usr/bin/env python3
import sys
from variety.Util import VarietyMetadata
try:
path = sys.argv[1]
except IndexError:
print('Error: need 1 argument: path to file', file=sys.stderr)
sys.exit(1)
x = VarietyMetadata(path)
print(x)
print(x.get_tag_multiple('Iptc.Application2.Keywords'))
and I get the following results:
$ python3 vrty-crash.py 02263_flash_1920x1080.jpg
<Util.VarietyMetadata object at 0x7f14cb76c120 (variety+Util+VarietyMetadata at 0xe1da20)>
free(): invalid pointer
Aborted
I'm curious. Does it reproduce if your further strip the VarietyMetadata
class code, and leave only the parent class GExiv2.Metadata
?
Actually no, GExiv2.Metadata seems capable of raising a proper exception when running on the images:
$ python3 vrty-crash2.py 02263_flash_1920x1080.jpg
vrty-crash2.py:4: PyGIWarning: GExiv2 was imported without specifying a version first. Use gi.require_version('GExiv2', '0.10') before import to ensure that the right version gets loaded.
from gi.repository import GExiv2
<GExiv2.Metadata object at 0x7fa548915ea0 (GExiv2Metadata at 0x1a3ba80)>
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x9a in position 1: invalid start byte
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "vrty-crash2.py", line 14, in <module>
print(x.get_tag_multiple('Iptc.Application2.Keywords'))
SystemError: <class 'gobject.Warning'> returned a result with an error set
The test code is as follows:
#!/usr/bin/env python3
import sys
from gi.repository import GExiv2
try:
path = sys.argv[1]
except IndexError:
print('Error: need 1 argument: path to file', file=sys.stderr)
sys.exit(1)
x = GExiv2.Metadata(path)
print(x)
print(x.get_tag_multiple('Iptc.Application2.Keywords'))
I could reproduce this using GExiv2 code alone, without any code from Variety sources:
#!/usr/bin/env python3
import sys
import gi
gi.require_version('GExiv2', '0.10')
gi.require_version('PangoCairo', '1.0')
gi.require_version('Gdk', '3.0')
from gi.repository import Gdk, Pango, GdkPixbuf, GLib, GExiv2
try:
path = sys.argv[1]
except IndexError:
print('Error: need 1 argument: path to file', file=sys.stderr)
sys.exit(1)
x = GExiv2.Metadata(path=path)
r = x.get_tag_multiple('Iptc.Application2.Keywords')
print(r)
$ ./test2.py 02263_flash_1920x1080.jpg
free(): invalid pointer
Aborted (core dumped)
So, this is either incorrect usage of the GExiv2 library by Variety, or a bug in GExiv2 library itself. I'm not experienced with the GExiv2 library. Maybe it's worth to ask their maintainers?
Hmm. Today I tried to reproduce it again (in order to try to find out the cause), and it didn't reproduce on me. Got:
$ ./test2.py 02263_flash_1920x1080.jpg
Traceback (most recent call last):
File "./test2.py", line 17, in <module>
r = x.get_tag_multiple('Iptc.Application2.Keywords')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x9a in position 1: invalid start byte
Maybe I was dreaming last time :( ... Sorry then. Or it just doesn't reproduce consistently. Don't know...
However, I found something else interesting here:
The test code that crashes is, basically:
from variety.Util import VarietyMetadata
x = VarietyMetadata("02263_flash_1920x1080.jpg")
y = x.get_tag_multiple('Iptc.Application2.Keywords')
I edited the /usr/lib/python3/dist-packages/variety/Util.py
file, adding a print
statement in VarietyMetadata
's __get_item__()
method.
Result: the __get__item__()
method implementation in VarietyMetadata
class is never called during the above test!
What this means is that the call to x.get_tag_multiple()
calls parent class (GExiv2.Metadata
's) implementation of get_tag_multiple()
, which doesn't call VarietyMetadata.__get__item__()
method.
This also means that the implementation of GExiv2.Metadata.get_tag_multiple()
doesn't have any clue about the key name 'Iptc.Application2.Keywords'
which only your subclass knows about (only your class knows what the field MULTIPLES
defined in your class means)...
This looks like a conceptual bug in Variety's code, as Variety defines:
class VarietyMetadata(GExiv2.Metadata)
immediately producing this custom list as a member of VarietyMetadata
class:
MULTIPLES = {
"Iptc.Application2.Headline",
"Iptc.Application2.Keywords",
"Xmp.dc.creator",
"Xmp.dc.subject",
}
However, it doesn't re-implement the get_tag_multiple()
method, but only __get_item__()
.
It still looks weird to me why original GExiv2.Metadata.get_tag_multiple()
function crashes with something ugly that looks like buffer overrun, but the way it is used by Variety also looks wrong to me.
I'm not sure how to fix this problem. Maybe override/re-implement get_tag_multiple()
method in your subclass will fix it?
What do you think about it? Does this make sense to you?
Thanks
The point of the __getitem__
and __setitem__
definitions is to allow easy key based access to the metadata fields. It is correct that we offload the get_tag_multiple()
call to the superclass, since we don't need to change its behaviour. Iptc.Application2.Keywords
is also a standardized keyword name: https://www.exiv2.org/iptc.html
The strangeness is how subclassing a GObject class causes exception handling to break, when this sort of thing seems to be supported?
The point of the
__getitem__
and__setitem__
definitions is to allow easy key based access to the metadata fields. It is correct that we offload theget_tag_multiple()
call to the superclass, since we don't need to change its behaviour.Iptc.Application2.Keywords
is also a standardized keyword name: https://www.exiv2.org/iptc.htmlThe strangeness is how subclassing a GObject class causes exception handling to break, when this sort of thing seems to be supported?
I see. You're absolutely right. In any case, I'm glad to help. I'll follow the linked issue you opened against gnome devs. Thank you again for developing this piece of software!
Just a note for anyone getting here: here's the latest discussion of this bug that I see. https://gitlab.gnome.org/GNOME/pygobject/issues/327
The reason is clearly related to the underlying library, not to Variety.
Hi, same issue for me I've got an invalid pointer error when starting variety in a terminal. I've SOLVED the issue by renaming the last loaded wallpaper image.
For that, I've ran
variety -v
to get the last loaded image filename. In my case the interesting line was at the end of the log :
INFO: 2019-12-22 14:17:12,065: update_indicator() 'Setting file info to: /home/laurent/Images/Wallpaper/Beautiful Mix HD Wallpapers/Googaback 161 pack (242).jpg'
then I've executed the following command:
mv "/home/laurent/Images/Wallpaper/Beautiful Mix HD Wallpapers/Googaback 161 pack (242).jpg" "/home/laurent/Images/Wallpaper/Beautiful Mix HD Wallpapers/Googaback 161 pack.jpg"
and start variety again... and it works !!
I've got the same error on Variety 0.8.2. @LaurentOngaro solution worked like charm for me too.
Same problem with fedora 32: ~/.config/variety $ inxi -S System: Host: grizzly Kernel: 5.8.13-200.fc32.x86_64 x86_64 bits: 64 Desktop: GNOME 3.36.6 Distro: Fedora release 32 (Thirty Two)
~/.config/variety $ python3 -q -X faulthandler /usr/bin/variety free(): invalid pointer Fatal Python error: Aborted
Current thread 0x00007f8eaafaa740 (most recent call first):
File "/usr/lib/python3.8/site-packages/variety/Util.py", line 179 in getitem
File "/usr/lib/python3.8/site-packages/variety/Util.py", line 531 in read_metadata
File "/usr/lib/python3.8/site-packages/variety/VarietyWindow.py", line 755 in update_indicator
File "/usr/lib/python3.8/site-packages/variety/VarietyWindow.py", line 549 in reload_config
File "/usr/lib/python3.8/site-packages/variety/VarietyWindow.py", line 194 in start
File "/usr/lib/python3.8/site-packages/variety/init.py", line 239 in main
File "/usr/bin/variety", line 66 in
~/.config/variety $ variety --version variety 0.8.4
It happens on old photos with, possibly, metadata not in utf-8 (renaming the photo with the same name works) Thank you for your job
$ inxi -S System: Host: madjoe Kernel: 4.19.32-1-MANJARO x86_64 bits: 64 Desktop: KDE Plasma 5.15.3 Distro: Manjaro Linux
$ pkg-config --modversion gtk+-3.0 3.24.7
$ variety
(variety:7093): Gtk-WARNING **: 10:38:12.389: Theme parsing error: gtk.css:68:35: The style property GtkButton:child-displacement-x is deprecated and shouldn't be used anymore. It will be removed in a future version
(variety:7093): Gtk-WARNING **: 10:38:12.389: Theme parsing error: gtk.css:69:35: The style property GtkButton:child-displacement-y is deprecated and shouldn't be used anymore. It will be removed in a future version
(variety:7093): Gtk-WARNING **: 10:38:12.389: Theme parsing error: gtk.css:73:46: The style property GtkScrolledWindow:scrollbars-within-bevel is deprecated and shouldn't be used anymore. It will be removed in a future version free(): invalid pointer Aborted (core dumped)