silverstripe / addons.silverstripe.org

Website hosting Silverstripe Framework extensions
BSD 3-Clause "New" or "Revised" License
13 stars 16 forks source link

BuildAddonJob pulls the wrong package?! #258

Open chillu opened 3 years ago

chillu commented 3 years ago

From UAT, but have seen this in prod as well. It's building a2nt/silverstripe-dropzone here, but failing on a request for https://packagist.org/packages/silverstripe/frameworktest.json. I think the 502 is packagist rate limiting us, since a standard GET (even from the same IP a while later) works fine. My theory is that the job data isn't passed through properly via queuedjobs.

SELECT * FROM QueuedJobDescriptor WHERE SavedJobMessages LIKE '%502%';
*************************** 3. row ***************************
                ID: 2700
         ClassName: Symbiote\QueuedJobs\DataObjects\QueuedJobDescriptor
        LastEdited: 2020-11-19 13:06:38
           Created: 2020-11-19 09:37:07
          JobTitle: Build Addon: a2nt/silverstripe-dropzone
         Signature: b7f1153b1b91689787c7a3e9c4a08b9a
    Implementation: BuildAddonJob
        StartAfter: NULL
        JobStarted: 2020-11-19 11:01:00
      JobRestarted: NULL
       JobFinished: 2020-11-19 11:01:08
        TotalSteps: 1
    StepsProcessed: 1
LastProcessedCount: -1
      ResumeCounts: 0
      SavedJobData: O:8:"stdClass":1:{s:9:"PackageID";i:2;}
  SavedJobMessages: a:2:{i:0;s:28:"[2020-11-19 11:01:08][INFO] ";i:1;s:290:"[2020-11-19 13:06:38][ERROR] Server error: `GET https://packagist.org/packages/silverstripe/frameworktest.json` resulted in a `502 Bad Gateway` response:
<html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
<hr><center>ngin (truncated...)
";}
         JobStatus: Complete
           JobType: 2
           RunAsID: 7
            Worker: 6da3b86f1fa7c1e147af891ad2dad38b
            Expiry: 2020-11-19 13:01:00
    NotifiedBroken: 0
       WorkerCount: 1
chillu commented 3 years ago

More research (after a single run through of UpdateAddons (which creates thousands of BuildAddonsJob)

SELECT COUNT(*) FROM QueuedJobDescriptor WHERE SavedJobMessages LIKE '%Could not parse version constraint%'\G;
*************************** 1. row ***************************
COUNT(*): 577

SELECT COUNT(*) FROM QueuedJobDescriptor WHERE SavedJobMessages LIKE '%Connection refused%'\G;
*************************** 1. row ***************************
COUNT(*): 345

SELECT COUNT(*) FROM QueuedJobDescriptor WHERE SavedJobMessages LIKE '%502 Bad Gateway%'\G;
*************************** 1. row ***************************
COUNT(*): 1091

At least the last two are likely related to rate limiting. It appears that silverstripe/frameworktest is the first package which triggered one of these failures above. It's unclear how that influences subsequent job runs for completely different packages though. Might be some weird singleton reuse inside queuedjobs