Crash in asc_locale_ctx_add_entry

[arrowd]

I'm using AppStreamCompose library via appstream-generator. I reproducibly get a crash when asgen processes the stellarium/1.2/freebsd:12:x86:64 package. Here's the backtrace:

#0  0x00000008009f3da3 in g_list_prepend () at /usr/local/lib/libglib-2.0.so.0
#1  0x0000000800d19de1 in asc_locale_ctx_add_entry (ctx=0x28092a8980, entry=0x2809202350) at ../compose/asc-utils-l10n.c:85
#2  0x0000000800d19d24 in asc_l10n_parse_data_qt (ctx=0x28092a8980, locale=0x28092a898d "ru", data=0x28097dc0da "\003", len=930096) at ../compose/asc-utils-l10n.c:274
#3  0x0000000800d19aba in asc_l10n_parse_file_qt (ctx=0x28092a8980, unit=0x80481fde0, locale=0x28092a898d "ru", filename=0x804820080 "/usr/local/share/stellarium/translations/stellarium-planetary-features/ru.qm", error=0x7fffdf3d43f0)
    at ../compose/asc-utils-l10n.c:322
#4  0x0000000800d193ba in asc_l10n_search_translations_qt (ctx=0x28092a8980, unit=0x80481fde0, prefix=0x8048f4870 "/usr/local", error=0x7fffdf3d43f0) at ../compose/asc-utils-l10n.c:397
#5  0x0000000800d18a52 in asc_read_translation_status (cres=0x2809283d00, unit=0x80481fde0, prefix=0x8048f4870 "/usr/local", min_percentage=25) at ../compose/asc-utils-l10n.c:450
#6  0x0000000800d07e9f in asc_compose_process_task_cb (ctask=0x2809220320, compose=0x2809212de0) at ../compose/asc-compose.c:1628
#7  0x0000000800d068a2 in asc_compose_run (compose=0x2809212de0, cancellable=0x0, error=0x7fffdf3d4968) at ../compose/asc-compose.c:2157
#8  0x00000000004f19e3 in _D9ascompose7ComposeQi3runMFC3gio11CancellableQnZC4glib8PtrArrayQj (this=0x28075105a0, cancellable=0x0) at ../build/girepo/ascompose/Compose.d:374
#9  0x00000000004223f3 in _D5asgen9extractor13DataExtractor14processPackageMFCQBy8backends10interfaces7PackageZSQDg6result15GeneratorResult (this=0x2807513480, pkg=0x280727ef80) at ../src/asgen/extractor.d:268
#10 0x000000000040221f in _D5asgen6engine6Engine15processPackagesMFKACQBq8backends10interfaces7PackageCQCx11iconhandler11IconHandlerCQEb12cptmodifiers21InjectedModificationsZ14__foreachbody6MFQEuZi (pkgsChunk=...) at ../src/asgen/engine.d:184
#11 0x000000000040b6ac in _D3std11parallelism__T15ParallelForeachTSQBn5range__T6ChunksTAC5asgen8backends10interfaces7PackageZQBuZQDd7opApplyMFMDFQCgZiZ4doItMFZv () at /usr/local/include/d/std/parallelism.d-mixin-4102:4148
#12 0x000000080139f33a in std.parallelism.run!(void() delegate).run(void() delegate) () at /usr/local/lib/libphobos2-ldc-debug-shared.so.102
#13 0x000000080139f315 in std.parallelism.Task!(std.parallelism.run, void() delegate).Task.impl(void*) () at /usr/local/lib/libphobos2-ldc-debug-shared.so.102
#14 0x000000080139c2ba in std.parallelism.AbstractTask.job() () at /usr/local/lib/libphobos2-ldc-debug-shared.so.102
#15 0x000000080139c7b7 in std.parallelism.TaskPool.doJob(std.parallelism.AbstractTask*) () at /usr/local/lib/libphobos2-ldc-debug-shared.so.102
#16 0x000000080139cbc1 in std.parallelism.TaskPool.executeWorkLoop() () at /usr/local/lib/libphobos2-ldc-debug-shared.so.102
#17 0x000000080139ca6a in std.parallelism.TaskPool.startWorkLoop() () at /usr/local/lib/libphobos2-ldc-debug-shared.so.102
#18 0x00000008016bf71d in core.thread.context.Callable.opCall() () at /usr/local/lib/libdruntime-ldc-debug-shared.so.102
#19 0x00000008016c2f39 in core.thread.threadbase.ThreadBase.run() () at /usr/local/lib/libdruntime-ldc-debug-shared.so.102
#20 0x00000008016c0645 in core.thread.osthread.Thread.run() () at /usr/local/lib/libdruntime-ldc-debug-shared.so.102
#21 0x00000008016c0c4f in thread_entryPoint () at /usr/local/lib/libdruntime-ldc-debug-shared.so.102
#22 0x0000000800eb0a78 in thread_start (curthread=0x804818900) at /usr/src/lib/libthr/thread/thr_create.c:292
#23 0x0000000000000000 in  ()

When I switch to the frame #1 and then do print ctx->data in the debugger I get:

data = 0x75722f73657275 = {<error reading variable: Cannot access memory at address 0x75722f73657275>

Printing the value as ASCII results in:

(gdb) print (char[8])ctx->data
$13 = "ures/ru"

There is clearly a memory corruption somewhere in the code. I'll keep on debugging, but if anyone have an idea on where the bug is, I'm all ears.

[ximion]

There is clearly a memory corruption somewhere in the code.

Yes, looks like it - running valgrind on appstream-generator is a nightmare though due to its large memory pool, LMDB and threading.

So, to investigate this, try the following: Compile AppStream with address sanitizer (pass -Db_sanitize=address,undefined -Dgir=false to Meson setup), extract stellarium into a directory and then run appstreamcli-compose directly on that directory. That will be a much smaller testcase :-)

[arrowd]

Thanks for your input. Both -Dgir=false and "run appstreamcli-compose` are really valueable advices! I'll try them out and report back.

[arrowd]

Ah, the pointer named locale is altered while being marked g_autofree. It is strange, though, that this bug has gone unnoticed.

I'll prepare a patch.