search

phar-io / phive

The Phar Installation and Verification Environment (PHIVE)
https://phar.io
BSD 3-Clause "New" or "Revised" License
581 stars 45 forks source link

phive is only scanning first page of GitHub release api #274

Open lippok opened 4 years ago

lippok commented 4 years ago

I want to install the old version 0.11.16 of phpstan, but the installation fails when the release is not in the local cache.

phive.xml:

<?xml version="1.0" encoding="UTF-8"?>
<phive xmlns="https://phar.io/phive">
  <phar name="phpstan" version="^0.11.16" installed="0.11.16" location="./tools/phpstan" copy="true"/>
</phive>

Output:

Phive 0.14.4 - Copyright (C) 2015-2020 by Arne Blankerts, Sebastian Heuer and Contributors Downloading https://api.github.com/repos/phpstan/phpstan/releases [ERROR] No matching release found!

The release is not listed at https://api.github.com/repos/phpstan/phpstan/releases but on https://api.github.com/repos/phpstan/phpstan/releases?page=2 but only the first page seems to be read.

theseer commented 4 years ago

That is apparently correct but not a trivial fix.

As github doesn't seem to provide an explicit API to find out how many releases actually exist, this information has to be deducted from the link header's last attribute.

Given the most common use case is likely to request the installation of a somewhat recent version, we probably do not want to always get the full list of all releases from all pages and with that have pointless requests hitting against the github api rate limit.

We thus need to implement a sort of streaming, lazy-loading kind of lookup that fetches additional release infos when no match is found and not all releases have been read yet.

That's completely different from the current implementation which consumes the downloaded json. Also, the downloader code doesn't yet support looking at the link header.

theseer commented 3 years ago

Starting phive 0.14.5 we're requesting 100 releases per "page". While that of course doesn't fix the actual issue, it at least mitigates the problem a little bit on the cost of some additional traffic.

I'm still unhappy with the github api ;)

Xethron commented 2 years ago

Hello @theseer, it seems like this problem is back.

I've been having problems downloading phpstan 0.12.99, which looks like it just dropped out of the last 30. Looking through the code, I see the ?per_page=100 is still in the GithubAliasResolver, however, it appears that this is not being used for phpstan, and instead, the URL in https://phar.io/data/repositories.xml is being used.

theseer commented 2 years ago

@Xethron Which version if phive are you using?

Xethron commented 2 years ago

Phive 0.15.0. I just managed to get it to work by changing

  <phar name="phpstan" version="^0.12.92" installed="0.12.99" location="./tools/phpstan" copy="false"/>

to

  <phar name="phpstan/phpstan" version="^0.12.92" installed="0.12.99" location="./tools/phpstan" copy="false"/>

I assume this bypasses the "alias" lookup, and then uses the GithubAliasResolver instead of the url in repositories.xml

theseer commented 2 years ago

Can you try using phive 0.15.1 and if the problem persists with it?

theseer commented 2 years ago 
  <phar name="phpstan/phpstan" version="^0.12.92" installed="0.12.99" location="./tools/phpstan" copy="false"/>

I assume this bypasses the "alias" lookup, and then uses the GithubAliasResolver instead of the url in repositories.xml

Yes, that's a different resolver and I somehow managed to miss the other resolver when adding the per_page option to the URL.

Xethron commented 2 years ago

Just tested it with 0.15.1 and it works!

Thank you for your help, and thank you for the extremely fast replies :)

PS: Installing phive with curl -LsS -o phive.phar https://phar.io/releases/phive.phar still installs 0.15.0

theseer commented 2 years ago

Fixed that. Thanks.