search

mozilla / delivery-console

Normandy recipe editor
Mozilla Public License 2.0
13 stars 22 forks source link

The Delivery Console Stage website is displayed as a blank page #1130

Closed Softvision-PatriciuPop closed 4 years ago

Softvision-PatriciuPop commented 4 years ago

[Affected versions]:

[Affected Platforms]:

[Steps to reproduce]:

  1. Open the browser and navigate to the "https://delivery-console.stage.mozaws.net/" website.
  2. Observe the page.

[Expected result]:

[Actual result]:

[Notes]:

rehandalal commented 4 years ago

Looking at the console I see:

image

It appears that the static bundle (https://delivery-console.stage.mozaws.net/static/js/main.712a2b3f.js) has disappeared or somehow the "static" folder is no longer pointed at the correct location.

@sciurus thoughts?

sciurus commented 4 years ago 
bpitts39260:~ bpitts$ aws s3 ls --recursive s3://$BUCKET
2020-05-31 12:53:54        242 __version__
2020-05-31 12:53:47        262 asset-manifest.json
2020-05-31 12:53:20       5430 favicon.ico
2020-05-31 12:53:47        535 index.html
2020-05-31 12:53:20        317 manifest.json
2020-05-31 12:53:47       3136 service-worker.js
2020-05-31 12:53:47     540404 static/css/main889a9d21.css
2020-05-31 12:53:47     747429 static/css/main889a9d21.css.map
2020-05-31 12:53:53    2400336 static/js/main.1a1cb3a9.js
2020-05-31 12:53:53    7032611 static/js/main.1a1cb3a9.js.map

main.1a1cb3a9.js is the bundle from the latest build, but it's not the one Firefox is trying to download. Firefox is trying to download one from the previous build, which the latest build deleted.

bpitts39260:~ bpitts$ aws s3 cp s3:/$BUCKET/index.html .
download: s3://$BUCKET/index.html to ./index.html
bpitts39260:~ bpitts$ grep 'static/js/' index.html 
<!doctype html><html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1,shrink-to-fit=no"><meta name="theme-color" content="#000000"><link rel="manifest" href="/manifest.json"><link rel="shortcut icon" href="/favicon.ico?v=1"><title>Delivery Console</title><link href="/static/css/main889a9d21.css" rel="stylesheet"></head><body><noscript>You need to enable JavaScript to run this app.</noscript><div id="root"></div><script src="/static/js/main.712a2b3f.js"></script></body></html>

That shows the files in S3 are out of sync. The assets are from the latest build, but the index is from the previous build.

Checking built history, Jenkins has decided to deploy v1.19.0 to stage 6 times!

4/10 16:46 5/10 17:51 5/27 16:11 5/31 10:34 5/31 12:49 5/31 12:50

I believe those last two builds ran their uploads concurrently, which is how we ended up in this mixed state. I've reconfigured Jenkins to disallow concurrent builds for this job, which will prevent this from happening again in the future.

I don't understand why Jenkins is forgetting it has seen this tag and retriggering the build, but I don't plan to investigate this.

sciurus commented 4 years ago

I've redeployed stage and it's working again.

Softvision-PatriciuPop commented 4 years ago

I've verified that this issue is no longer reproducible using Firefox Beta 78.0b1 and FIrefox Nightly 79.0a1 on Windows 10 x64.