search

silverstripe / addons.silverstripe.org

Website hosting Silverstripe Framework extensions
BSD 3-Clause "New" or "Revised" License
13 stars 16 forks source link

Could not parse version constraint ^: Invalid version string "^" #257

Open chillu opened 3 years ago

chillu commented 3 years ago

AddonsUpdater started failing around 30th of Oct in prod, without us changing anything obvious in code. Composer v2 was announced 24th of Oct. The timing is too close to be coincidence.

Theory A: Packagist made subtle changes to their API responses Theory B: Packagist introduced harsher rate limit or connection terminations on large volume API requests

Job log error (note that this happens without a precending connection error):

[2020-11-18 12:47:57][INFO] Updating addon silverstripe/homepagefordomain:
- Processed version 0.1.9999999.9999999-dev
- Processed version 9999999-dev
- Will not queue a rebuild
PackagistService: Retrieved silverstripe/kickassets
Updating addon silverstripe/kickassets:
- Processed version dev-master
[2020-11-18 12:47:59][WARNING] dylangrech92/seotoolbox 298 Could not parse version constraint ^: Invalid version string "^"
[2020-11-18 12:47:59][WARNING] dylangrech92/seotoolbox 298 Could not parse version constraint ^: Invalid version string "^"
[2020-11-18 12:47:59][WARNING] dylangrech92/seotoolbox 298 Could not parse version constraint ^: Invalid version string "^"
[2020-11-18 12:47:59][WARNING] dylangrech92/seotoolbox 298 Could not parse version constraint ^: Invalid version string "^"
[2020-11-18 12:47:59][WARNING] dylangrech92/seotoolbox 298 Could not parse version constraint ^: Invalid version string "^"
[2020-11-18 12:47:59][WARNING] dylangrech92/seotoolbox 298 Could not parse version constraint ^: Invalid version string "^"
[2020-11-18 12:47:59][WARNING] dylangrech92/seotoolbox 298 Could not parse version constraint ^: Invalid version string "^"
[2020-11-18 12:47:59][WARNING] dylangrech92/seotoolbox 298 Could not parse version constraint ^: Invalid version string "^"
[2020-11-18 12:47:59][WARNING] dylangrech92/seotoolbox 298 Could not parse version constraint ^: Invalid version string "^"
[2020-11-18 12:47:59][WARNING] dylangrech92/seotoolbox 298 Could not parse version constraint ^: Invalid version string "^"
[2020-11-18 12:47:59][WARNING] dylangrech92/seotoolbox 298 Could not parse version constraint ^: Invalid version string "^"
[2020-11-18 12:47:59][WARNING] dylangrech92/seotoolbox 298 Could not parse version constraint ^: Invalid version string "^"
[2020-11-18 12:47:59][WARNING] dylangrech92/seotoolbox 298 Could not parse version constraint ^: Invalid version string "^"
[2020-11-18 12:47:59][WARNING] silverstripe/kickassets 1196 Could not parse version constraint ^: Invalid version string "^"
[2020-11-18 12:47:59][WARNING] silverstripe/kickassets 1196 Could not parse version constraint ^: Invalid version string "^"
[2020-11-18 12:47:59][WARNING] silverstripe/kickassets 1196 Could not parse version constraint ^: Invalid version string "^"

Raygun error: https://app.raygun.com/crashreporting/1jsbyna/errors/4786298554?correlationId=426936499138&occurredOn=2020-11-18T00:06:13.0000000Z

Message: GuzzleHttp\Exception\ConnectException: cURL error 7: Failed to connect to packagist.org port 443: Connection refused (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
#0 GuzzleHttp\Handler\CurlFactory::createRejection called at [/var/www/mysite/releases/4a2e9a2990e431935f1213f643fdab1274de8cf7/vendor/guzzlehttp/guzzle/src/Handler/CurlFactory.php:155]
#1 GuzzleHttp\Handler\CurlFactory::finishError called at [/var/www/mysite/releases/4a2e9a2990e431935f1213f643fdab1274de8cf7/vendor/guzzlehttp/guzzle/src/Handler/CurlFactory.php:105]
#2 GuzzleHttp\Handler\CurlFactory::finish called at [/var/www/mysite/releases/4a2e9a2990e431935f1213f643fdab1274de8cf7/vendor/guzzlehttp/guzzle/src/Handler/CurlHandler.php:43]
#3 GuzzleHttp\Handler\CurlHandler::__invoke called at [/var/www/mysite/releases/4a2e9a2990e431935f1213f643fdab1274de8cf7/vendor/guzzlehttp/guzzle/src/Handler/Proxy.php:28]
#4 GuzzleHttp\Handler\Proxy::GuzzleHttp\Handler\{closure} called at [/var/www/mysite/releases/4a2e9a2990e431935f1213f643fdab1274de8cf7/vendor/guzzlehttp/guzzle/src/Handler/Proxy.php:51]
#5 GuzzleHttp\Handler\Proxy::GuzzleHttp\Handler\{closure} called at [/var/www/mysite/releases/4a2e9a2990e431935f1213f643fdab1274de8cf7/vendor/guzzlehttp/guzzle/src/PrepareBodyMiddleware.php:37]
#6 GuzzleHttp\PrepareBodyMiddleware::__invoke called at [/var/www/mysite/releases/4a2e9a2990e431935f1213f643fdab1274de8cf7/vendor/guzzlehttp/guzzle/src/Middleware.php:29]
#7 GuzzleHttp\Middleware::GuzzleHttp\{closure} called at [/var/www/mysite/releases/4a2e9a2990e431935f1213f643fdab1274de8cf7/vendor/guzzlehttp/guzzle/src/RedirectMiddleware.php:70]
#8 GuzzleHttp\RedirectMiddleware::__invoke called at [/var/www/mysite/releases/4a2e9a2990e431935f1213f643fdab1274de8cf7/vendor/guzzlehttp/guzzle/src/Middleware.php:59]
#9 GuzzleHttp\Middleware::GuzzleHttp\{closure} called at [/var/www/mysite/releases/4a2e9a2990e431935f1213f643fdab1274de8cf7/vendor/guzzlehttp/guzzle/src/HandlerStack.php:71]
#10 GuzzleHttp\HandlerStack::__invoke called at [/var/www/mysite/releases/4a2e9a2990e431935f1213f643fdab1274de8cf7/vendor/guzzlehttp/guzzle/src/Client.php:351]
#11 GuzzleHttp\Client::transfer called at [/var/www/mysite/releases/4a2e9a2990e431935f1213f643fdab1274de8cf7/vendor/guzzlehttp/guzzle/src/Client.php:162]
#12 GuzzleHttp\Client::requestAsync called at [/var/www/mysite/releases/4a2e9a2990e431935f1213f643fdab1274de8cf7/vendor/guzzlehttp/guzzle/src/Client.php:182]
#13 GuzzleHttp\Client::request called at [/var/www/mysite/releases/4a2e9a2990e431935f1213f643fdab1274de8cf7/vendor/knplabs/packagist-api/src/Packagist/Api/Client.php:201]
#14 Packagist\Api\Client::request called at [/var/www/mysite/releases/4a2e9a2990e431935f1213f643fdab1274de8cf7/vendor/knplabs/packagist-api/src/Packagist/Api/Client.php:181]
#15 Packagist\Api\Client::respond called at [/var/www/mysite/releases/4a2e9a2990e431935f1213f643fdab1274de8cf7/vendor/knplabs/packagist-api/src/Packagist/Api/Client.php:105]
#16 Packagist\Api\Client::get called at [/var/www/mysite/releases/4a2e9a2990e431935f1213f643fdab1274de8cf7/app/src/services/PackagistService.php:59]
#17 PackagistService::getPackages called at [/var/www/mysite/releases/4a2e9a2990e431935f1213f643fdab1274de8cf7/app/src/services/AddonUpdater.php:75]
#18 AddonUpdater::update called at [/var/www/mysite/releases/4a2e9a2990e431935f1213f643fdab1274de8cf7/app/src/tasks/UpdateAddonsTask.php:47]
#19 UpdateAddonsTask::run called at [/var/www/mysite/releases/4a2e9a2990e431935f1213f643fdab1274de8cf7/vendor/symbiote/silverstripe-queuedjobs/src/Jobs/RunBuildTaskJob.php:80]

Working through this in https://github.com/silverstripe/addons.silverstripe.org/pull/255. We might need to update to composer v2, although it's unclear what fails.

chillu commented 3 years ago

Looks like we're creating around 400MB of QueuedJobDescriptor entries per day :( There's a Symbiote\QueuedJobs\Jobs\CleanupJob configured to clean out after three days though, so I can't see how we've ended up accumulating 20GB of data there. Will need to keep an eye on the growth.

SELECT COUNT(*) FROM Addon WHERE Name IS NULL;
+----------+
| COUNT(*) |
+----------+
|       86 |
+----------+
1 row in set (0.00 sec)

That's really weird, and potentially a contributor to the log accumulation (hard to tell cause and effect with queuedjobs). I've added a guard for this in the pull request (not live yet), and cleaned out those entries for now. Might have been faulty or empty packagist responses that didn't get picked up by Guzzle as an HTTP error before processing them?

DELETE FROM Addon WHERE Name IS NULL;

I've also created an empty slate again on the production jobs to see how it behaves (TRUNCATE QueuedJobDescriptor).

With the PR running in UAT, it appears that we've been rate limited quite heavily by packagist (2req/min), which makes it impossible to diagnose the long running task issues now. Will keep monitoring the situation.

chillu commented 3 years ago

The UAT server seems to repeatedly run out of memory, it only has 1GB. But the PR introduces a generator approach for the memory intensive packagist data. My hope is that it'll fix the issue, and allow us to perform a full UpdateAddonsTask run on UAT.

image
robbieaverill commented 3 years ago

But the PR introduces a generator approach for the memory intensive packagist data

Do you have any suggestions for improving this at the Packagist PHP SDK level? I'm happy to have a look at that

chillu commented 3 years ago

Thanks Robbie! I think I've gotten through it now with the latest commits in https://github.com/silverstripe/addons.silverstripe.org/pull/255.

We could still be more efficient in our Packagist API usage in two ways:

  1. Use HTTP caching in guzzle, separate from build job throttling, e.g. for 24h. It's another layer of complexity though.
  2. In AddonUpdater, use the statically generated files in https://packagist.org/p/silverstripe/framework.json rather than the larger/richer API responses in https://packagist.org/packages/silverstripe/framework.json. Technically we only need the full data in AddonBuilder. The Packagist API docs say they prefer consumers to stick to those static files.

If we're still getting rate limited after the latest round of fixes, I might need to look at that second option there.

chillu commented 3 years ago

Still happening on the same two repos, even without any interference by rate limiting and HTTP errors. The weird thing is that I can't really see what version constraint would be invalid there. It's these two payloads:

https://packagist.org/packages/dylangrech92/seotoolbox.json https://packagist.org/packages/unclecheese/silverstripe-kickassets.json

When "extracting" a composer.json out of a version there (mainly for the require statements) and running composer validate, you don't get any errors on the constraints. Example:

{
  "name": "dylangrech92/seotoolbox",
  "description": "This plugin was created to facilitate SEO work by automating most of the manual labor",
  "keywords": [
      "seo",
      "reporting",
      "silverstripe",
      "Toolbox",
      "automated links",
      "seo testing"
  ],
  "homepage": "https://github.com/dylangrech92/seotoolbox",
  "version": "dev-develop",
  "license": [
      "BSD-3-Clause"
  ],
  "authors": [
      {
          "name": "Dylan Grech",
          "email": "dylangrech92@gmail.com",
          "homepage": "https://innovativecodes.com/websites-marketing-and-more/dylan-grech",
          "role": "Developer"
      }
  ],
  "type": "silverstripe-module",
  "support": {
      "issues": "https://github.com/dylangrech92/seotoolbox/issues",
      "email": "dylangrech92@gmail.com",
      "docs": "https://github.com/dylangrech92/seotoolbox/tree/master/docs",
      "source": "https://github.com/dylangrech92/seotoolbox"
  },
  "time": "2016-10-14T05:22:22+00:00",
  "extra": {
      "installer-name": "seotoolbox"
  },
  "require": {
      "php": "^5.3.0",
      "silverstripe/cms": "^3.2",
      "silverstripe/framework": "^3.2",
      "undefinedoffset/sortablegridfield": "^0.6.2",
      "silverstripe/html5": "^1.0.4"
  }
}