[ERR] Auto-update raises access denied when acl is set to private for S3

[sProject]

• 19.47.1:

• 2.16.3

• Linux, AppImage, x64, Fedora 27:

Hi! I'm working on getting publishing for linux + auto update for AppImage to work with private s3 bucket and private AppImage. I have a very simple app that print its version... and that's about it. When I use S3Options and I set acl : public-read everything works perfect. I'm also able to publish to the private s3 bucket and I'm able to set acl: private. However, I get an error access denied when I run the app and the app wants to check for updates. I have configured ~/.aws/credentials, I even tried setting env vars, nothing helps. Here is my log

[fedora@localhost electron-update-example]$ dist/simple-electron-0.0.1-x86_64.AppImage 
installed: X-AppImage-BuildId=eb49d8b0-d51e-11a7-213a-0b6d859a326c image: X-AppImage-BuildId=bda05140-d8da-11a7-1864-9feaf57d15d2
[10:22:32.809] [info] App starting...
[10:22:33.480] [info] Checking for update
[10:22:33.481] [info] Checking for update...
[10:22:34.684] [error] Error: HttpError: 403 Forbidden
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>CBC7496C38F832C2</RequestId><HostId>ss8I01KR+SqYTft6Rcq78XrSdLPwAqPrW4T9FU7UykqGk6a0Dvdj49nzv2a/zwcRsD8tyTNfM7A=</HostId></Error>"
Headers: {
  "content-type": [
    "application/xml"
  ],
  "date": [
    "Mon, 04 Dec 2017 10:34:29 GMT"
  ],
  "server": [
    "AmazonS3"
  ],
  "transfer-encoding": [
    "chunked"
  ],
  "x-amz-id-2": [
    "ss8I01KR+SqYTft6Rcq78XrSdLPwAqPrW4T9FU7UykqGk6a0Dvdj49nzv2a/zwcRsD8tyTNfM7A="
  ],
  "x-amz-request-id": [
    "CBC7496C38F832C2"
  ]
}
    at createHttpError (/tmp/.mount_simplejtlBEG/app/resources/app.asar/node_modules/builder-util-runtime/src/httpExecutor.ts:29:10)
    at IncomingMessage.response.on (/tmp/.mount_simplejtlBEG/app/resources/app.asar/node_modules/builder-util-runtime/src/httpExecutor.ts:146:18)
    at emitNone (events.js:86:13)
    at IncomingMessage.emit (events.js:188:7)
    at endReadableNT (_stream_readable.js:975:12)
    at _combinedTickCallback (internal/process/next_tick.js:80:11)
    at process._tickCallback (internal/process/next_tick.js:104:9)
From previous event:
    at CancellationToken.createPromise (/tmp/.mount_simplejtlBEG/app/resources/app.asar/node_modules/builder-util-runtime/src/CancellationToken.ts:51:5)
    at ElectronHttpExecutor.doApiRequest (/tmp/.mount_simplejtlBEG/app/resources/app.asar/node_modules/builder-util-runtime/src/httpExecutor.ts:79:30)
    at ElectronHttpExecutor.request (/tmp/.mount_simplejtlBEG/app/resources/app.asar/node_modules/builder-util-runtime/src/httpExecutor.ts:71:17)
    at /tmp/.mount_simplejtlBEG/app/resources/app.asar/node_modules/electron-updater/src/GenericProvider.ts:19:55
    at Generator.next (<anonymous>)
From previous event:
    at GenericProvider.getLatestVersion (/tmp/.mount_simplejtlBEG/app/resources/app.asar/node_modules/electron-updater/out/GenericProvider.js:72:11)
    at /tmp/.mount_simplejtlBEG/app/resources/app.asar/node_modules/electron-updater/src/AppUpdater.ts:251:37
From previous event:
    at AppImageUpdater.doCheckForUpdates (/tmp/.mount_simplejtlBEG/app/resources/app.asar/node_modules/electron-updater/out/AppUpdater.js:326:11)
    at /tmp/.mount_simplejtlBEG/app/resources/app.asar/node_modules/electron-updater/src/AppUpdater.ts:228:25
    at Generator.next (<anonymous>)
    at runCallback (timers.js:672:20)
    at tryOnImmediate (timers.js:645:5)
    at processImmediate [as _immediateCallback] (timers.js:617:5)
From previous event:
    at AppImageUpdater._checkForUpdates (/tmp/.mount_simplejtlBEG/app/resources/app.asar/node_modules/electron-updater/out/AppUpdater.js:280:11)
    at AppImageUpdater.checkForUpdates (/tmp/.mount_simplejtlBEG/app/resources/app.asar/node_modules/electron-updater/src/AppUpdater.ts:177:35)
    at App.<anonymous> (/tmp/.mount_simplejtlBEG/app/resources/app.asar/main.js:139:15)
    at emitTwo (events.js:111:20)
    at App.emit (events.js:194:7)
[10:22:34.698] [info] Error in auto-updater. HttpError: 403 Forbidden
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>CBC7496C38F832C2</RequestId><HostId>ss8I01KR+SqYTft6Rcq78XrSdLPwAqPrW4T9FU7UykqGk6a0Dvdj49nzv2a/zwcRsD8tyTNfM7A=</HostId></Error>"
Headers: {
  "content-type": [
    "application/xml"
  ],
  "date": [
    "Mon, 04 Dec 2017 10:34:29 GMT"
  ],
  "server": [
    "AmazonS3"
  ],
  "transfer-encoding": [
    "chunked"
  ],
  "x-amz-id-2": [
    "ss8I01KR+SqYTft6Rcq78XrSdLPwAqPrW4T9FU7UykqGk6a0Dvdj49nzv2a/zwcRsD8tyTNfM7A="
  ],
  "x-amz-request-id": [
    "CBC7496C38F832C2"
  ]
}

[develar]

Auto-update downloads file using public HTTP urls, not using API. Do you understand that your users will need to set credentials also?

[sProject]

Yes, I understand this. I have aws cli credentials set up on my machine but auto update ignores them. Can you give me an advise on how to use aws module for auto updater instead of normal HTTPs.

Thanks

[develar]

As far I see, you need to set Authorization header. You can simply set it using autoUpdater.requestHeaders property.

See http://docs.aws.amazon.com/AmazonS3/latest/dev/RESTAuthentication.html#ConstructingTheAuthenticationHeader

https://www.npmjs.com/package/aws4 can be used to generate such header. Please try :)

autoUpdater.requestHeaders = {Authorization: "secret"}

[sProject]

Awesome man! Thanks for the help! I'll try it and report back

[sProject]

OK here it is - the ultimate guide to having private s3 bucket, for anyone else who is interested in this.

HOWTO:

Dependencies aws-sign aws-sdk

Main.js All modifications are done in the main.js electron file - the file where you implement auto-update.

Step 1 - Import libraries

const aws = require('aws-sdk');
const AwsSign = require('aws-sign');

Step 2 - Set constants You'll need a copy of your package.json build section, the one that looks like this

{
  "build": {
    "publish": {
      "bucket": "BUCKET_NAME",
      "path": "PATH/TO/FILES"
    }
  }
}

Using it we define the following constants

const release_path = config.get('build').publish.path;
const latest_yml_path = `/${release_path}/latest-${process.platform}.yml`;
const s3_bucket = config.get('build').publish.bucket;

Step 3 - Create signer

const credentials = new aws.SharedIniFileCredentials();
const signer = new AwsSign({
  accessKeyId: credentials.accessKeyId,
  secretAccessKey: credentials.secretAccessKey
});

Step 4 - Update checking-for-update

autoUpdater.on('checking-for-update', () => {
  var opts = {
    method: 'GET',
    host: `${s3_bucket}.s3.amazonaws.com`,
    path: latest_yml_path
  };
  signer.sign(opts);
  autoUpdater.requestHeaders = opts.headers
})

Step 5 - Update update-available

autoUpdater.on('update-available', (info) => {
  let update_path = `/${release_path}/${info.path}`;
  let opts = {
    method: 'GET',
    host: `${s3_bucket}.s3.amazonaws.com`,
    path: update_path
  };
  signer.sign(opts);
  autoUpdater.requestHeaders = opts.headers
  autoUpdater.downloadUpdate()
  }
})

Hope this helps!

[marcusjwhelan]

@sProject I currently have a private S3 without all of this work. I simply have permissions set to getObject only for public but no write access. And have the aws credentials on my build laptop in my .aws folder. You could I presume have these credentials in CodeShip as well and use docker in there to build the project but I haven't done that yet.

[marcolink]

For us, hosting our tools (electron apps) on public accessible buckets is not an option. The given example didn't work with our region eu-central-1 which only allows V4 signing. Therefore we had to change the signing part a bit:

Solution:

We use aws4 for request signing.

and

autoUpdater.on('checking-for-update', () => {
  var opts = {
    method: 'GET',
    host: `${s3_bucket}.s3.amazonaws.com`,
    path: latest_yml_path
  };
  signer.sign(opts);
  autoUpdater.requestHeaders = opts.headers
})

becomes

import * as aws4 from 'aws4';
import * as path from 'path'
const pkg = require('../../package');

autoUpdater.on('checking-for-update', () => {
  const opts = {
    service: 's3',
      region: pkg.build.publish.region,
      method: 'GET',
      host: `s3-${pkg.build.publish.region}.amazonaws.com`,
      path: path.join('/', pkg.build.publish.bucket, latest_yml_path)
  };

  aws4.sign(opts, {
    accessKeyId: <AWS_ACCESS_KEY>,
    secretAccessKey: <AWS_SECRET_ACCESS_KEY>
  });
  signer.sign(opts);
  autoUpdater.requestHeaders = opts.headers
})

The same also works for the actual update.

Question:

Is there a way to write a custom Provider, or extends the existing GenericProvider to apply signing where it should actually happen

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stuartcusackie]

@marcolink

How come you have signer.sign(opts) after aws4.sign? Seems like you are signing with two different aws sdk versions?

I'm having trouble with autoUpdater on my private repo:

at downloadBlockMap
status 403: Forbidden

Very reluctant to switch to a public repo...

I've opened an issue here: https://github.com/electron-userland/electron-builder/issues/4030

[marcolink]

@stuartcusackie

sorry, my code had a bug, it's actually:

import * as aws4 from 'aws4';
import * as path from 'path'
const pkg = require('../../package');

autoUpdater.on('checking-for-update', () => {
  const opts = {
    service: 's3',
      region: pkg.build.publish.region,
      method: 'GET',
      host: `s3-${pkg.build.publish.region}.amazonaws.com`,
      path: path.join('/', pkg.build.publish.bucket, latest_yml_path)
  };

  aws4.sign(opts, {
    accessKeyId: <AWS_ACCESS_KEY>,
    secretAccessKey: <AWS_SECRET_ACCESS_KEY>
  });
  // signer.sign(opts); --remove this line --
  autoUpdater.requestHeaders = opts.headers
})

[stuartcusackie]

Thanks @marcolink !

Still having trouble with these blockmaps...

[stuartcusackie]

@marcolink @sproject

Did you ever have trouble with the blockmap requests? I seem to need to create two more signing requests for the old blockmap and the new blockmap?

Can I see your builder configs?

[JTInfinite]

Hi all, sorry to resurrect a closed issue (I can of course open a new one if preferred) but using the above configuration I receive the error: Error: net::ERR_TOO_MANY_REDIRECTS.

Any ideas on what I might be doing wrong here? Thanks

[JTInfinite]

Referencing my above code - Error: net::ERR_TOO_MANY_REDIRECTS is the error in the development environment.

When I build the app I get the following:

"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message>

This leads me to believe I am constructing the opts object wrong. Super annoying!

[scorring]

@JTInfinite, I was having the same SignatureDoesNotMatch issue with the code provided by @marcolink. Here is my working code sample:

autoUpdater.on("checking-for-update", async () => {
  let opts = {
    region: "eu-west-1",
    protocol: "https:",
    hostname: "BUCKET_NAME.s3.amazonaws.com",
    path: "/PATH/TO/latest.yml",
    host: "s3-eu-west-1.amazonaws.com"
  };

  await aws4.sign(opts, {
    accessKeyId: "XXX",
    secretAccessKey: "YYY"
  });

  autoUpdater.requestHeaders = opts.headers;
});

Change host and region following your values.

[Cyrillius]

@scorring

I am having this exact issue but I didn't find a working solution.

What is strange is that to find an existing version the Host is correctly set:

11:35:04.067 > Checking for update
11:35:04.118 > checking-for-update options: {
  service: 's3',
  region: 'eu-west-1',
  host: '{MYBUCKET}.s3-eu-west-1.amazonaws.com',
  path: '/4/latest.yml'
}
11:35:04.254 > checking-for-update header: {
  Host: '{MYBUCKET}.s3-eu-west-1.amazonaws.com',
  'X-Amz-Content-Sha256': 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855',
  'X-Amz-Date': '20201118T103504Z',
  Authorization: 'AWS4-HMAC-SHA256 Credential={MYKEYID}/20201118/eu-west-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=fcf7d3b39bae40e56f748f5a3a64f6b366e0f23941a5dd047783007cabe69ce8'
}
11:35:06.359 > Found version 0.1.5 (url: {MYAPP} 0.1.5.exe)

But to download the actual block map the host is not set anymore:

11:35:06.512 > Download block maps (old: "https://{MYBUCKET}.s3.amazonaws.com/4/Festina%20Support%20Setup%200.1.4.exe.blockmap", new: https://{MYBUCKET}.s3.amazonaws.com/4/Festina%20Support%20Setup%200.1.5.exe.blockmap)
header of the failed request : {
  accept: '*/*',
  'X-Amz-Content-Sha256': 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855',
  'X-Amz-Date': '20201118T103504Z',
  Authorization: 'AWS4-HMAC-SHA256 Credential={MYKEYID}/20201118/eu-west-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=fcf7d3b39bae40e56f748f5a3a64f6b366e0f23941a5dd047783007cabe69ce8',
  'User-Agent': 'electron-builder',
  'Cache-Control': 'no-cache'
}

It seems like electron is copying signature information but not the Host information.

Maybe this fix is not totally working: https://github.com/electron-userland/electron-builder/pull/4848/files

Still searching where is the issue....

[scorring]

@Cyrillius : Did you try with exactly these options:

region: "eu-west-1", protocol: "https:", hostname: "BUCKET_NAME.s3.amazonaws.com", path: "/PATH/TO/latest.yml", host: "s3-eu-west-1.amazonaws.com"

I had to play around before finding these fields (less, more or distinct properties caused error).

[Cyrillius]

my options were:

  const opts = {
    service: 's3',
    region: api.amazon.s3Bucket.region,
    host: `${api.amazon.s3Bucket.name}.${api.amazon.s3Bucket.host}`,
    path:
      process.platform === 'win32'
        ? `${api.amazon.s3Bucket.releasePath}/latest.yml`
        : `${api.amazon.s3Bucket.releasePath}/latest-${process.platform}.yml`,
  };
  log.info('checking-for-update', opts);
  aws4.sign(opts, {
    accessKeyId: api.amazon.production.keyId,
    secretAccessKey: api.amazon.production.accessKey,
  });
  autoUpdater.requestHeaders = opts.headers;

with parameters:

      "s3Bucket": {
      "name": "kronabyflashstationupdatebucket",
      "host": "s3.amazonaws.com",
      "region": "eu-west-1",
      "releasePath": "/4"
    }

and still have this kind of errors:

[2020-11-19 00:15:52.727] [error] Cannot download differentially, fallback to full download: Error: Cannot download "https://{bucketname}.s3.amazonaws.com/4/xxxx-support-0.1.6.exe.blockmap", status 403: Forbidden
    at ClientRequest.<anonymous> (C:\Users\{user}\AppData\Local\Programs\xxxx-support\resources\app.asar\main.prod.js:2:36762)
    at ClientRequest.emit (events.js:210:5)
    at SimpleURLLoaderWrapper.<anonymous> (electron/js2c/browser_init.js:2559:12)
    at SimpleURLLoaderWrapper.emit (events.js:210:5)

I have given up as on Linux platform there is no error and on windows it will the app completely but will work at the end.

I tried your solution which is for me:

  const opts = {
    service: 's3',
    protocol: "https:",
    region: api.amazon.s3Bucket.region,
    hostname: `${api.amazon.s3Bucket.name}.${api.amazon.s3Bucket.host}`,
    host: "s3-eu-west-1.amazonaws.com",
    path:
      process.platform === 'win32'
        ? `${api.amazon.s3Bucket.releasePath}/latest.yml`
        : `${api.amazon.s3Bucket.releasePath}/latest-${process.platform}.yml`,
  };
  log.info('checking-for-update', opts);
  aws4.sign(opts, {
    accessKeyId: api.amazon.production.keyId,
    secretAccessKey: api.amazon.production.accessKey,
  });
  autoUpdater.requestHeaders = opts.headers;

and still have this issue:

[2020-12-31 14:24:19.213] [error] Cannot download differentially, fallback to full download: Error: Cannot download "https://{bucketname}.s3.amazonaws.com/4/xxxx-support-0.1.9.exe.blockmap", status 403: Forbidden
    at ClientRequest.<anonymous> (C:\Users\{user}\Documents\Git\xxx-support\release\win-unpacked\resources\app.asar\main.prod.js:2:36989)
    at ClientRequest.emit (events.js:210:5)
    at SimpleURLLoaderWrapper.<anonymous> (electron/js2c/browser_init.js:2559:12)
    at SimpleURLLoaderWrapper.emit (events.js:210:5)

[reddybhavanish]

Does anthing worked on this ?

[fabiobsantosprogrow]

Nowadays we still facing the same problem with version 6.1.4. I don't understand why this issue is closed when people still having issues... We will try the download manually on private s3 approach but I would like to have some guidance how to this using the auto-update without extra work.