I can't search components by categories..

[matiasdelellis]

I am maintaining a humble software store based on the old gnome-packagekit. I upgraded from fedora 35 to fedora 37, and today I find some bugs.

It seems that there were several changes in appstream 0.15, but the particular api used, it claims to work the same.

• https://www.freedesktop.org/software/appstream/docs/api/appstream-AsPool.html#as-pool-get-components-by-categories

Theoretically it looks for all the components that are in any of the categories. 🤔 But in practice, it seems that by adding more search categories, it shows less components.

Well, I share a little test code.

#include <glib-object.h>
#include <appstream.h>

void
print_pkg_components_by_categories (AsPool *as_pool, gchar **categories)
{
    GPtrArray *components = NULL;
    AsComponent *component = NULL;
    const gchar *pkgname = NULL;
    guint i = 0;

    g_print("\n\nSearch by categories:\n");
    for (i = 0; categories[i] != NULL; i++) {
        g_print (" - Category: %s\n", categories[i]);
    }

    g_print("Components results:\n");
    components = as_pool_get_components_by_categories (as_pool, categories);
    for (i = 0; i < components->len; i++) {
        component = AS_COMPONENT (g_ptr_array_index (components, i));
        pkgname = as_component_get_pkgname (component);
        if (pkgname) {
            g_print (" - Package name: %s\n", pkgname);
        }
    }
}

int main( void )
{
    AsPool *as_pool;
    gchar **categories = NULL;

    as_pool = as_pool_new ();
    if (!as_pool_load (as_pool, NULL, NULL)) {
        g_print ("D'OH!: Cannot load AsPool");
        return -1;
    }

    categories = g_strsplit("AdventureGame", ",", -1);
    print_pkg_components_by_categories (as_pool, categories);
    g_strfreev(categories);

    categories = g_strsplit("BlocksGame", ",", -1);
    print_pkg_components_by_categories (as_pool, categories);
    g_strfreev(categories);

    categories = g_strsplit("AdventureGame,BlocksGame", ",", -1);
    print_pkg_components_by_categories (as_pool, categories);
    g_strfreev(categories);

    categories = g_strsplit("ActionGame,AdventureGame,ArcadeGame,Amusement,BlocksGame,BoardGame,CardGame,Game,KidsGame,LogicGame,RolePlaying,Shooter,Simulation,SportsGame,StrategyGame", ",", -1);
    print_pkg_components_by_categories (as_pool, categories);
    g_strfreev(categories);
}

Build with:

cc `pkg-config --cflags appstream` test.c -o test `pkg-config --libs appstream`

[matias@bangho-mdl xings-software]$ ./test 

Search by categories:
 - Category: AdventureGame
Components results:
 - Package name: naev
 - Package name: edgar
 - Package name: crossfire-client
 - Package name: scummvm
 - Package name: openarena
 - Package name: pioneer
 - Package name: freedink-engine
 - Package name: colobot

Search by categories:
 - Category: BlocksGame
Components results:
 - Package name: lbrickbuster2
 - Package name: flobopuyo
 - Package name: ballbuster
 - Package name: quadrapassel
 - Package name: BlockOutII
 - Package name: crystal-stacker

Search by categories:
 - Category: AdventureGame
 - Category: BlocksGame
Components results:

Search by categories:
 - Category: ActionGame
 - Category: AdventureGame
 - Category: ArcadeGame
 - Category: Amusement
 - Category: BlocksGame
 - Category: BoardGame
 - Category: CardGame
 - Category: Game
 - Category: KidsGame
 - Category: LogicGame
 - Category: RolePlaying
 - Category: Shooter
 - Category: Simulation
 - Category: SportsGame
 - Category: StrategyGame
Components results:

(process:36826): XbValueBindings-CRITICAL **: 15:39:52.495: xb_value_bindings_bind_str: assertion 'idx < G_N_ELEMENTS(_self->values)' failed

(process:36826): XbValueBindings-CRITICAL **: 15:39:52.495: xb_value_bindings_bind_str: assertion 'idx < G_N_ELEMENTS(_self->values)' failed

(process:36826): XbValueBindings-CRITICAL **: 15:39:52.496: xb_value_bindings_bind_str: assertion 'idx < G_N_ELEMENTS(_self->values)' failed

(process:36826): XbValueBindings-CRITICAL **: 15:39:52.496: xb_value_bindings_bind_str: assertion 'idx < G_N_ELEMENTS(_self->values)' failed

(process:36826): XbValueBindings-CRITICAL **: 15:39:52.496: xb_value_bindings_bind_str: assertion 'idx < G_N_ELEMENTS(_self->values)' failed

(process:36826): XbValueBindings-CRITICAL **: 15:39:52.496: xb_value_bindings_bind_str: assertion 'idx < G_N_ELEMENTS(_self->values)' failed

(process:36826): XbValueBindings-CRITICAL **: 15:39:52.496: xb_value_bindings_bind_str: assertion 'idx < G_N_ELEMENTS(_self->values)' failed

(process:36826): XbValueBindings-CRITICAL **: 15:39:52.496: xb_value_bindings_bind_str: assertion 'idx < G_N_ELEMENTS(_self->values)' failed

(process:36826): XbValueBindings-CRITICAL **: 15:39:52.496: xb_value_bindings_bind_str: assertion 'idx < G_N_ELEMENTS(_self->values)' failed

(process:36826): XbValueBindings-CRITICAL **: 15:39:52.496: xb_value_bindings_bind_str: assertion 'idx < G_N_ELEMENTS(_self->values)' failed

(process:36826): XbValueBindings-CRITICAL **: 15:39:52.496: xb_value_bindings_bind_str: assertion 'idx < G_N_ELEMENTS(_self->values)' failed

So, searching AdventureGame find 8 components, and searching BlocksGame show 6 components, and looking for both categories, does not find any.. 🙈

But my problem started when I search all the game subcategories, and it doesn't find anything either, but it also returns errors. 🤔

[ximion]

That you get less an less results is intended: Have a look at https://specifications.freedesktop.org/menu-spec/latest/apa.html - if you search for a main cetagory like Game you get all games, if you search for an additional category using Game;ActionGame you'll get all action games. Searching for ActionGame;AdventureGame;ArcadeGame;BlocksGame;BoardGame would mean "give me all games that are categorized as action, adventure, arcade, blocks and board game", and it's no wonder that you'd get few or no results for that. If you want to look for stuff that is in either category, you'll need a second query.

The critical warnings however are not normal, those should not exist.

[matiasdelellis]

Hello Ximion, Thanks for taking the trouble to answer.. 😃

I understand that it may seem logical, but it is not what the documentation says, and it is not how it behaved before.. This test is on a server where I have debian stretch installed.

matias@services:~$ ./test 

Search by categories:
 - Category: AdventureGame
Components results:
 - Package name: drascula-spanish
 - Package name: freedink-engine
 - Package name: mudlet
 - Package name: drascula
 - Package name: abe
 - Package name: drascula-italian
 - Package name: beneath-a-steel-sky
 - Package name: colobot
 - Package name: manaplus
 - Package name: freedink-engine
 - Package name: drascula-german
 - Package name: drascula-french
 - Package name: freedink-dfarc
 - Package name: passage
 - Package name: renpy-thequestion
 - Package name: moria
 - Package name: scummvm
 - Package name: sludge-engine
 - Package name: lure-of-the-temptress
 - Package name: gravitation
 - Package name: flight-of-the-amazon-queen
 - Package name: renpy
 - Package name: between
 - Package name: renpy-demo

Search by categories:
 - Category: BlocksGame
Components results:
 - Package name: petris
 - Package name: ghextris
 - Package name: cuyo
 - Package name: fltk1.3-games
 - Package name: gtetrinet
 - Package name: flobopuyo
 - Package name: angrydd
 - Package name: quadrapassel

Search by categories:
 - Category: AdventureGame
 - Category: BlocksGame
Components results:
 - Package name: drascula-spanish
 - Package name: freedink-engine
 - Package name: petris
 - Package name: mudlet
 - Package name: drascula
 - Package name: abe
 - Package name: drascula-italian
 - Package name: beneath-a-steel-sky
 - Package name: colobot
 - Package name: ghextris
 - Package name: cuyo
 - Package name: manaplus
 - Package name: fltk1.3-games
 - Package name: freedink-engine
 - Package name: drascula-german
 - Package name: drascula-french
 - Package name: freedink-dfarc
 - Package name: passage
 - Package name: gtetrinet
 - Package name: flobopuyo
 - Package name: renpy-thequestion
 - Package name: angrydd
 - Package name: moria
 - Package name: scummvm
 - Package name: sludge-engine
 - Package name: lure-of-the-temptress
 - Package name: gravitation
 - Package name: flight-of-the-amazon-queen
 - Package name: renpy
 - Package name: quadrapassel
 - Package name: between
 - Package name: renpy-demo
matias@services:~$ 

As you can see, the searches of various categories are added to the result. So, the search is OR between the categories.

I do not share the search of all categories because the answer is very long. It returns some duplicate results, which isn't good either, but I was already dealing with that (It's an old distribution, and I don't intend to fix libappstrem 0.10.6) 😅.

Now (At least in v0.15.5) the search is an AND as you well comment, and seems that I can check it. 🤔

I can easily adapt to the change, but I would like to know when was the behavior change, and perhaps the documentation should be updated. 😉

Thanks again,

[ximion]

Hi! You are right, and the correct way to address this would actually be to restore the previous behavior. However, annoyingly, this behavior is caused by libxmlb not supporting the features we need, so implementing this would only work with quite inefficient hacks. Implementing the needed features in libxmlb would take time, and even when it was done we would need to rely on a much newer version of libxmlb, which also isn't ideal.

So, I think even though it absolutely sucks, the thing to do here is to adjust the documentation, so it reflects reality. Is that okay for you? The behavior change will have happened with the switch to libxmlb in version 0.15.0.

[matiasdelellis]

Perfect. I can adapt. I was only worried about the behavior change. In fact this gives flexibility, since I have AND easier, and the OR is done just with multiple calls..

Thanks.

[ximion]

This (AND) is also, given how the XDG menu system works, the thing that you usually want, as you want to filter by category and there are master categories like "Game". When you run multiple calls, the only thing you need to do is to deduplicate the results, which could be a bit annoying.