search

commercialhaskell / stackage-server

Server for stable, curated Haskell package sets
MIT License
106 stars 27 forks source link

"See all snapshot X appears in" should exclude old `nightly`s by default #311

Open andreasabel opened 2 years ago

andreasabel commented 2 years ago

E.g. https://www.stackage.org/package/Cabal/snapshots times out for me, likely because there are too many snapshots Cabal appears in, so the page construction takes too long.

But really, I am usually interested in all LTS snapshots X appears in. (And even for LTSs, I am only interested in the latest LTS for each GHC version.)

Maybe restricting the query to the LTSs and the latest nightly should be the default, and one could try to really list all snapshots in a second step.

(Moved from https://github.com/commercialhaskell/stackage-content/issues/100.)

juhp commented 2 years ago

Maybe it makes sense to split it into all LTS snapshots and all Nightly snapshots, at least as a first step. (I tend to agree with you on latest ghc minors but Stackage/stack hasn't really designed that way, but maybe something can be done.)

(Admittedly for ghc libraries I usually just go to https://gitlab.haskell.org/ghc/ghc/-/wikis/commentary/libraries/version-history but yes this ought to work better.)

juhp commented 2 years ago

Hmm but I am not sure if this will help to fix the root problem - looks like the db queries may be struggling now. :-( I see the same problem if I try with my own packages...

juhp commented 2 years ago

I am wondering if queries like the one in this report trigger the heavy server db query problems we are seeing. At least after trying it again today, I am seeing terrible performance again: @snoyberg any thoughts?

Well I could imagine limiting to LTS snapshots could reduce the problem substantially.

snoyberg commented 2 years ago

Seems like a reasonable first step here

snoyberg commented 2 years ago

It turns out to be a bit more complicated to switch to "LTS only", but limiting to only X entries is simple. I've currently limited responses to 50 only. If this actually fixes the issues, we can do the overhaul necessary to make this an LTS-only change instead.

CC @psibi

andreasabel commented 1 year ago

@snoyberg wrote:

I've currently limited responses to 50 only.

This fixes the load timeout, but makes the page entirely useless.

E.g. I want to know which version of a package appeared first in which LTS. The page is useless in determining this: https://www.stackage.org/package/HsOpenSSL/snapshots

How about turning the map around and give for each version of a package the range of snapshots it is in, and separate LTS from nightly?

Something like this?

Package xyz

       LTS           nightly
1.3.4  19.3  - 20.4  2022-01-04 - 2022-12-22
1.3.3  18.27 - 19.2  2021-10-12 - 2022-01-01
...

CC @juhp

juhp commented 1 year ago

What if we just listed on the main package page:

Nightly: 
LTS 20:
LTS 19:
LTS 18:
:

Maybe we should also drop nightly from the snapshots page?

@andreasabel I am not sure, but I think your idea could burden the server quite a bit?

andreasabel commented 1 year ago

What if we just listed on the main package page:

Nightly: 
LTS 20:
LTS 19:
LTS 18:
:

That would be great to have!

Maybe we should also drop nightly from the snapshots page?

That would make room for more LTSs there, but there are still hundreds of LTSs, so condensing the report would still be called for.

@andreasabel I am not sure, but I think your idea could burden the server quite a bit?

This might likely require an index (said reverse map) to work efficiently.