search

ximion / appstream-generator

A fast AppStream metadata generator
GNU Lesser General Public License v3.0
44 stars 29 forks source link

Appstream-generator tool with Fedora ? #117

Closed Tuxman2 closed 1 year ago

Tuxman2 commented 1 year ago

Hello,

A while ago I came here looking for a way to get my local Ubuntu mirror to work with Gnome Software. I had given up for lack of time. Now I have more time for that. Recently, I used Fedora 38 which I like. I created a local mirror repository on an external hard drive for Fedora. Again, I will want to use my local mirror with Gnome Software, but for that I need the metadata. Is appstream-generator tool suitable in my case? Can Fedora users here help me do what I want ? Of course, if they have a little time to devote to me.

Thank you so much.

ximion commented 1 year ago

It should work fine for generating Fedora data, however the RPM-MD backend of asgen has fallen into disrepair a bit and needs a new person taking care of it that knows Fedora better. So, there may be issues that I don't know about.

That said, AFAIK (and @hughsie can probably correct me if I'm wrong), Fedora still does not ship AppStream metadata as part of the regular repository metadata, but in a package instead. So, if that is still the case, if you purely mirror a Fedora repository, there would be no need to regenerate AppStream metadata, as you would have essentially mirrored that as well already (minus the screenshots and external media).

Tuxman2 commented 1 year ago

Hello,

I mirrored Fedora repository on my external hard drive. So, I can install packages without internet. With Dnfdragora or yumex-ng, it works well. When I try to use Gnome Software, it is a hassle. Gnome Software show me the updates, I see all the list of software, but I don't know why when I try to install a software, nothings happens. I see there is repodata directories on my external hard drive. repodata seems to be metadata files. When I use dnfdragora, it says that it checks metadata. So, I don't why I have some difficulties to use Gnome Software with my local mirror. That's why I try to find a solution to be able to use Gnome Software with my mirrored repository. But for now, I don't know how to do that.

Regards.

Note: Do I need to enable packagekit-offline or something similar under Fedora ?

Conan-Kudo commented 1 year ago

You just need to install appstream-data package on your system. Then it'll work.

Tuxman2 commented 1 year ago

@Conan-Kudo: Thanks for the tip, I will test it.

Regards.

Tuxman2 commented 1 year ago

Hello,

appstream-data was installed, but despite that, I don't know how to use my local mirror with Gnome Software. I see updates, list of programs, I know how to search, but when I try to install or update my system with the help of my mirror under Gnome Software, it doesn't work. I guess I need to create the metadata for my mirror because by default it seems to refer to rpmfusion-free and not free and Fedora repository. I also noticed, via the command gnome software --verbose, that the program tries to access websites to retrieve information and images. So, for now, I'm stuck on this topic because I don't know how to solve this problem.

Thanks.

Regards.

sidt4 commented 1 year ago

@Tuxman2: As explained above by ximion, Appstream works fundamentally different between Fedora and other distros ( e.g Debian ). Distros like Debian use appstream-generator and hosts the metadata live on https://appstream.debian.org, whereas Fedora uses appstream-generator and builds appstream data once every few months, and releases the metadata as part of appstream-data RPM package, which in most cases could contain old outdated metadata (iirc).

So, your issue doesn't have anything to do with appstream-generator, and if you cannot install / update in gnome-software via your mirror you should report it to https://gitlab.gnome.org/GNOME/gnome-software/-/issues.

sidt4 commented 1 year ago

Fedora still does not ship AppStream metadata as part of the regular repository metadata, but in a package instead.

Yes, they do ( https://src.fedoraproject.org/rpms/appstream-data )

Any idea why Fedora doesn't do it like Debian / Ubuntu etc ?

Because data from appstream-data RPM is months old ( due to their reduced frequency of release ) and is totally useless and confusing.

hughsie commented 1 year ago

Because data from appstream-data RPM is months old

hughsie commented 1 year ago

If it helps, I've build f37,f38,f39 this morning.

sidt4 commented 1 year ago

hughsie: Thanks for that. But, I'm not sure if that addresses the real issue.

appstream-data page mentions that it's built weekly, but I don't think that's true in reality.

appstream-data-fedora

I noticed bugs in https://bugzilla.redhat.com, where users complain that appstream-data is months old, which I think is a waste of their valuable time ( which can be used elsewhere ).

Due to this, I see quite a few annoying issues in gnome-software ( and maybe others too in other apps ), which I don't see in Debian (https://appstream.debian.org) / Ubuntu (https://appstream.ubuntu.com/).

There are already issues with appstream metadata as not all apps have switched to it, and even when apps have switched to provide valid appstream metadata, Fedora gives 2 months old metadata sometimes. This is more confusing, since sometimes I've to think how should a particular gnome-software issue be handled, particularly for Fedora due to its lagging appstream-data.

So, is there any real reason to use appstream-data RPM, and not host it live in a Fedora repository, which dnf can pickup during it's metadata refresh phase ?

hughsie commented 1 year ago

appstream-data page mentions that it's built weekly, but I don't think that's true in reality.

That's not been true for a long time.

which I think is a waste of their valuable time

Dude. I'm not the only one who's able to build the metadata.

So, is there any real reason to use appstream-data RPM, and not host it live in a Fedora repository

Yes, in that Fedora release engineering did not want to add the feature, mainly due to architectural issues.

ximion commented 1 year ago

Yeah, it took me some tough negotiations (and the introduction of the AppStream YAML variant, brr...) to get that feature into Debian, but it's working very well there and in its derivatives :-) I wonder if Fedora could at least provide infrastructure to auto-generate the appstream-data package and auto-upload it, that would be neat. But knowing @hughsie, it's definitely help wanted and patches welcome on that side ;-)

ximion commented 1 year ago

In other words,@sidt4, you could talk to Fedora's RelEng team ;-) I think other Fedora repos provide the metadata, and maybe opinions changed (there would for sure be extra work to do to maintain this though).

sidt4 commented 1 year ago

Dude. I'm not the only one who's able to build the metadata.

Please don't get me wrong. I'm not blaming anyone here, but blaming the appstream-data idea as a whole, which is difficult to get right.

Even if we update appstream-data daily, there is no guarantee it will be installed by the user in his system, unless it's pulled in as repo metadata during the dnf-metadata-refresh phase, which is performed before any dnf transaction.

Tuxman2 commented 1 year ago

Hello,

@Tuxman2: As explained above by ximion, Appstream works fundamentally different between Fedora and other distros ( e.g Debian ). Distros like Debian use appstream-generator and hosts the metadata live on https://appstream.debian.org, whereas Fedora uses appstream-generator and builds appstream data once every few months, and releases the metadata as part of appstream-data RPM package, which in most cases could contain old outdated metadata (iirc).

So, your issue doesn't have anything to do with appstream-generator, and if you cannot install / update in gnome-software via your mirror you should report it to https://gitlab.gnome.org/GNOME/gnome-software/-/issues.

Ok, I will report to Gnome-software gitlab. Thanks.