Unable to generate metadata on Ubuntu 23.04

[jhonny-oliveira]

Hi!

I'm getting the error (crash) below, while attempting to generate the metadata for my repo on Ubuntu 23.04 (Generator version: 0.9.0).

Any idea about what might be happening?

std.file.FileException@std/file.d(4717): /home/xd/.aptly/public/ubuntu/cache/tmp/asgen-Xdhknp38/ubuntu/langpacks/var/lib/locales/supported.d: No such file or directory
----------------
??:? @safe bool std.file.DirIteratorImpl.stepIn(immutable(char)[]) [0x7f4cee0f673f]
??:? ref @safe std.file.DirIteratorImpl std.file.DirIteratorImpl.__ctor!(immutable(char)[]).__ctor(immutable(char)[], std.file.SpanMode, bool) [0x7f4cee0f7c0d]
??:? void std.typecons.RefCounted!(std.file.DirIteratorImpl, 0).RefCounted.RefCountedStore.initialize!(immutable(char)[], std.file.SpanMode, bool).initialize(ref immutable(char)[], ref std.file.SpanMode, ref bool) [0x7f4cee0137a4]
??:? std.file.DirIterator std.file.dirEntries(immutable(char)[], std.file.SpanMode, bool) [0x7f4cee0f60d7]
??:? void asgen.backends.ubuntu.ubupkg.LanguagePackProvider.extractLangpacks() [0x5584ffc10390]
??:? immutable(char)[][immutable(char)[]] asgen.backends.ubuntu.ubupkg.UbuntuPackage.getDesktopFileTranslations(glib.KeyFile.KeyFile, const(immutable(char)[])) [0x5584ffc117ac]
??:? extern (C) glib.c.types.GPtrArray* asgen.extractor.DataExtractor.translateDesktopTextCallback(glib.c.types.GKeyFile*, const(char)*, void*) [0x5584ffb8ddf5]
??:? [0x7f4ceeb4d9d8]
??:? as_desktop_entry_parse_data [0x7f4ceeb57f13]
??:? asc_parse_desktop_entry_data [0x7f4cee4bf1cb]
??:? [0x7f4cee4baa2f]
??:? asc_compose_run [0x7f4cee4bb5ac]
??:? glib.PtrArray.PtrArray ascompose.Compose.Compose.run(gio.Cancellable.Cancellable) [0x5584ffc1d034]
??:? asgen.result.GeneratorResult asgen.extractor.DataExtractor.processPackage(asgen.backends.interfaces.Package) [0x5584ffb8df55]
??:? [0x5584ffb75a5d]
??:? void std.parallelism.ParallelForeach!(std.range.Chunks!(asgen.backends.interfaces.Package[]).Chunks).ParallelForeach.opApply(scope int delegate(asgen.backends.interfaces.Package[])).doIt() [0x5584ffb7f639]
??:? void std.parallelism.TaskPool.executeWorkLoop() [0x7f4cee06cd3f]
??:? thread_entryPoint [0x7f4cedd8bd79]
??:? [0x7f4ceda8f189]
??:? [0x7f4cedb1dbcf]

You can find additional information and the full execution log here: appstream-generator_log.txt.

Thank you! Jhonny Oliveira

[Tuxman2]

@jhonny-oliveira:

Hello,

Do you try to generate the metadata with your local repository under Ubuntu 23.04 ? I'm trying to use my local repository with Gnome Software but it seems that I must use appstream-generator to have 'appstream catalog metadata' which Gnome Software requires. If you trying to do the same thing, can you give me advice for it ? I'm also on Ubuntu 23.04 version.

Thank you very much.

[ximion]

@jhonny-oliveira: Does your repository even contain language packs? If not, does the issue go away if you simply use the "debian" backend instead of the Ubuntu one?

[jhonny-oliveira]

@Tuxman2 ,

You can find below a raw dump of what I have done. I created a mirror of my PPAs with aptly excluding the packages that cause the crash and then run the appstream-generator. When/if I ever get this working I will create and publish a nice HowTo with it. Let me know if you can make anything out of it and/or if I missed something.

# Create appstream directory
mkdir ~/.aptly/public/appstream/ubuntu

# create metadata
tee ~/.aptly/public/appstream/ubuntu/asgen-config.json <<EOF
{
  "ProjectName": "XtraDeb",
  "ArchiveRoot": ".",
  "MediaBaseUrl": "https://192.168.122.53/ubuntu/appstream/ubuntu/media/",
  "HtmlBaseUrl": "https://192.168.122.53/ubuntu/appstream/ubuntu/",
  "Backend": "ubuntu",
  "Suites": {
    "lunar": {
      "sections": ["apps","play"],
      "architectures": ["amd64"]
    }
  }
}
EOF

cd ~/.aptly/public/appstream/ubuntu
appstream-generator process lunar

# link dep11 metadata inside the dists/<suite>/<section>/ directory (e.g.: dists/lunar/apps/dep11)
rDir=../../ubuntu
aDir=../../appstream/ubuntu
for mDir in $(find ${aDir}/export/data/ -maxdepth 2 -mindepth 2 -type d)
do
  suite=$(basename $(dirname $mDir))
  section=$(basename $mDir)
  rPATH=$(realpath --relative-to=${rDir}/dists/${suite}/${section} ${mDir})
  ln -sf ${rPATH} ${rDir}/dists/${suite}/${section}/dep11
done

# In the client testing VM
curl http://192.168.122.53:8080/xtradeb-prsk-pubkey.asc | sudo tee /etc/apt/trusted.gpg.d/xtradeb.net-prsk-pubkey.asc

echo "deb [arch=amd64] http://192.168.122.53:8080/ubuntu lunar apps play" | sudo tee /etc/apt/sources.list.d/xtradeb.net.list

sudo apt update

Cheers! Jhonny Oliveira @ https://xtradeb.net

[Tuxman2]

@jhonny-oliveira: Thanks for your help.

I've got a local repository made with the help of apt-mirror. I've got DEP-11 metadata but I can't install any software under Gnome-software with the help of my local repository. As I said on another topic, I can see the list of applications, do a research on a software, but when I push the install button, it's waiting but nothings happens. It seems that Gnome-software tries to contact stackedit.io and appstream.ubuntu.com websites but I'm not online. I try to find a way to install sotfware under Gnome-software offline with the help of my repository (available on external hard disk).

Thanks.

Regards.

Note: I'm using Ubuntu 23.04.

[jhonny-oliveira]

@Tuxman2 ,

If I understood correctly, you have it working (almost) but the install fails. Correct?

I didn't get that far. Can you please provide me more details, so I can understand what I'm missing.

I don't need to be offline. So, that's not a issue for me.

[Tuxman2]

@jhonny-oliveira: Yes the install fails, but it is very strange that I can see the list of applications and do research on a software. I define my local repository in /etc/apt/sources.list file. Like this: deb [arch=amd64 trusted=yes] file:/media/......./....../ubuntu/ lunar main restricted universe multiverse

In the terminal, I can install software with the sudo apt install command. I can do updates too. I tried with Ubuntu Software but it crashes with it. So, I installed Gnome-Software. When I push on the install button, another button appears: "Cancel" and after that an item is displayed below: "Ubuntu Unknow source Deb". I'm waiting but nothing happens. My local source repository is well defined in Gnome-Software.

Can it be a Gnome-Software bug ? When I'm online everything is ok.

[jhonny-oliveira]

Dear @Tuxman2, I just found out that aptly doesn't support dep11 (https://github.com/aptly-dev/aptly/pull/473).

Can it be a Gnome-Software bug ? When I'm online everything is ok. I don't know and I have no chance to test it at the moment.

Can you please share the recipe you followed using apt-mirror?

@ximion ,

I confirm https://github.com/ximion/appstream-generator/commit/8750a9251fdf90616e895ef8cc1e0d17f546bf80 fixed the problem.

Thank you!

[Tuxman2]

Hello,

I dropped this problem. I don't have time to deal with it anymore because of my job. I returned to Manjaro and its pamac tool.

Thanks for taking the time to help me.

Regards.