search

repology / repology-webapp

Repology web application
https://repology.org
GNU General Public License v3.0
200 stars 26 forks source link

Expose Links as an API Property #198

Open thecliguy opened 2 years ago

thecliguy commented 2 years ago

The results returned by the repology website includes "Links(s)" when querying a project like so: https://repology.org/project/<NAME>/packages

Would it be possible to expose "Links(s)" as a property of the results returned when querying via the API?

Also, what is the significance of the "(s)" in "Links(s)"? Is is just a typo? Is it supposed to say "Link(s)"?

AMDmi3 commented 2 years ago

Would it be possible to expose "Links(s)" as a property of the results returned when querying via the API?

What's your usecase? It's possiblt, but it'll require specific API, as including the links with existing API would be too heavy.

Also, what is the significance of the "(s)" in "Links(s)"? Is is just a typo? Is it supposed to say "Link(s)"?

Fixed, thank you!

thecliguy commented 2 years ago

What's your usecase? It's possiblt, but it'll require specific API, as including the links with existing API would be too heavy.

I was particularly interested in returning the "Upstream homepage" link as a means of deriving whether a package is aligned with an original repo or a fork.

AMDmi3 commented 2 years ago

This is often not reliable, as homepage link usually remains the same when package switches to the fork. Download links would be more reliable but not many repositories provide them. Do you need this for one time or you need continuous availability of data? Do you need this for specific repo?

This looks too specific for a dedicated API, but it's possible to revive https://github.com/repology/repology-exports and set up daily export.

thecliguy commented 2 years ago

This is often not reliable, as homepage link usually remains the same when package switches to the fork. Download links would be more reliable but not many repositories provide them.

Ahh, OK, that makes sense to use the Upstream Download link (when avaiable) as a more reliable source of data. Thank you for that tip.

Do you need this for one time or you need continuous availability of data? Do you need this for specific repo?

Occasionally I will check the state of packages for ssh-audit. I'm not a maintainer of the project, just an occasional contributor.

This is just a thought (and perhaps a naive one)... You mentioned that exposing links would make the API too heavy... What if when calling the API it was to accept an optional boolean showlinks=1 so that links are only returned if explicitly requested? Another thought... Could the links data that's served be cached on the backend to reduce DB load?

darkdragon-001 commented 1 year ago

I would be interested in Package Page links to display nice links for users with an easy way to install. As an alternative I could probably generate "Install links" directly. Which property is the name used to install a package? Is it visiblename? E.g. apt-get install PACKAGE.

AMDmi3 commented 1 year ago

@thecliguy unfortunately no, the very possibility to request links would allow excess load on the server. And no, links cannot be cached - it's the heaviest table in repology database, and different packages are requested each time, so it'd be a waste of memory.

AMDmi3 commented 1 year ago

@darkdragon-001 no, it is binname, sometimes srcname

kzndotsh commented 2 months ago

+1

AMDmi3 commented 2 months ago

@kzndotsh what's your use case?

kzndotsh commented 2 months ago

@kzndotsh what's your use case?

I run a discord community (https://discord.gg/linux) and we maintain a bot for our server https://github.com/allthingslinux/tux

Our goal is to add a lot of various utility features relating to linux so I started working on a Repology integration/wrapper. At first from scratch, then came across https://repology-client.sysrq.in/index.html.

I found it strange that no URL was provided but I could find the URLs on the package web pages themselves so I considered it as the library just being undeveloped.

Looking further, I saw in your source code that you do have in fact links available but no API spec showed them in use so I came to find this issue now :)

AMDmi3 commented 2 months ago

OK, but what's the specific use case for links?

kzndotsh commented 2 months ago

OK, but what's the specific use case for links?

Sorry, sorta left that out specifically lol. Basically to provide links on the embeds generated from either individual package queries or package search lists.

e.g.

AMDmi3 commented 2 months ago

Hmm, that makes sense. I guess with migration to rust we have some additional performance budget so we can expose more info through additional endpoint. Do you expect the same format (e.g. per-package info) or some kind of aggregated format?

kzndotsh commented 2 months ago

Hmm, that makes sense. I guess with migration to rust we have some additional performance budget so we can expose more info through additional endpoint. Do you expect the same format (e.g. per-package info) or some kind of aggregated format?

To be fair, anything is better than nothing.

I'm not entirely sure what you imagine for an aggregated format though. Explain further?

Also I am curious - what of these URL/URL categories are generated dynamically vs not e.g. them being "manually curated/added"? https://repology.org/project/bash/information

AMDmi3 commented 2 months ago

I'm not entirely sure what you imagine for an aggregated format though. Explain further?

Something akin to what is shown on package 'information' tab, e.g. all known links without their relation to specific packages.

Also I am curious - what of these URL/URL categories are generated dynamically vs not e.g. them being "manually curated/added"?

Links related to upstream are obviously manual, links related to repositories are generated (on Repology or in a few cases on repository side).