Access Denied (403) executing downloadData() despite allow.guest.to(['read'])

gpavlov2016 commented 6 months ago

Before opening, please confirm:

[X] I have searched for duplicate or closed issues and discussions.
[X] I have read the guide for submitting bug reports.
[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.

JavaScript Framework

Next.js

Amplify APIs

Storage

Amplify Version

v6

Amplify Categories

storage

Backend

Amplify Gen 2 (Preview)

Environment information

``` # Put output below this line System: OS: Windows 11 10.0.22631 CPU: (20) x64 13th Gen Intel(R) Core(TM) i9-13900H Memory: 1.84 GB / 31.68 GB Binaries: Node: 18.19.1 - C:\Program Files\nodejs\node.EXE Yarn: 1.22.19 - ~\AppData\Roaming\npm\yarn.CMD npm: 10.2.4 - C:\Program Files\nodejs\npm.CMD pnpm: 8.15.4 - ~\AppData\Local\pnpm\pnpm.EXE Browsers: Edge: Chromium (123.0.2420.97) Internet Explorer: 11.0.22621.1 npmPackages: %name%: 0.1.0 @ampproject/toolbox-optimizer: undefined () @aws-amplify/backend: ^0.13.0 => 0.13.0 @aws-amplify/backend-cli: ^0.12.0 => 0.12.0 @aws-amplify/ui-react: ^6.1.8 => 6.1.8 @aws-amplify/ui-react-internal: undefined () @aws-sdk/s3-presigned-post: ^3.556.0 => 3.556.0 @babel/core: undefined () @babel/runtime: 7.22.5 @edge-runtime/cookies: 4.1.0 @edge-runtime/ponyfill: 2.4.2 @edge-runtime/primitives: 4.1.0 @hapi/accept: undefined () @heroicons/react: ^1.0.6 => 1.0.6 @mswjs/interceptors: undefined () @napi-rs/triples: undefined () @next/font: undefined () @next/react-dev-overlay: undefined () @opentelemetry/api: undefined () @types/node: ^20 => 20.12.7 @types/react: ^18.2.79 => 18.2.79 @types/react-dom: ^18.2.25 => 18.2.25 @vercel/nft: undefined () @vercel/og: 0.6.2 acorn: undefined () amphtml-validator: undefined () anser: undefined () arg: undefined () assert: undefined () async-retry: undefined () async-sema: undefined () autoprefixer: ^10.4.19 => 10.4.19 aws-amplify: ^6.0.28 => 6.0.28 aws-amplify/adapter-core: undefined () aws-amplify/analytics: undefined () aws-amplify/analytics/kinesis: undefined () aws-amplify/analytics/kinesis-firehose: undefined () aws-amplify/analytics/personalize: undefined () aws-amplify/analytics/pinpoint: undefined () aws-amplify/api: undefined () aws-amplify/api/server: undefined () aws-amplify/auth: undefined () aws-amplify/auth/cognito: undefined () aws-amplify/auth/cognito/server: undefined () aws-amplify/auth/enable-oauth-listener: undefined () aws-amplify/auth/server: undefined () aws-amplify/data: undefined () aws-amplify/data/server: undefined () aws-amplify/datastore: undefined () aws-amplify/in-app-messaging: undefined () aws-amplify/in-app-messaging/pinpoint: undefined () aws-amplify/push-notifications: undefined () aws-amplify/push-notifications/pinpoint: undefined () aws-amplify/storage: undefined () aws-amplify/storage/s3: undefined () aws-amplify/storage/s3/server: undefined () aws-amplify/storage/server: undefined () aws-amplify/utils: undefined () aws-cdk: ^2.138.0 => 2.138.0 aws-cdk-lib: ^2.138.0 => 2.138.0 aws-sdk: ^2.1603.0 => 2.1603.0 babel-packages: undefined () browserify-zlib: undefined () browserslist: undefined () buffer: undefined () bytes: undefined () ci-info: undefined () cli-select: undefined () client-only: 0.0.1 comment-json: undefined () compression: undefined () conf: undefined () constants-browserify: undefined () constructs: ^10.3.0 => 10.3.0 content-disposition: undefined () content-type: undefined () cookie: undefined () cross-spawn: undefined () crypto-browserify: undefined () css.escape: undefined () data-uri-to-buffer: undefined () debug: undefined () devalue: undefined () domain-browser: undefined () edge-runtime: undefined () esbuild: ^0.20.2 => 0.20.2 (0.19.12) eslint: ^8 => 8.57.0 eslint-config-next: 14.1.4 => 14.1.4 events: undefined () find-cache-dir: undefined () find-up: undefined () fresh: undefined () get-orientation: undefined () glob: undefined () gzip-size: undefined () http-proxy: undefined () http-proxy-agent: undefined () https-browserify: undefined () https-proxy-agent: undefined () icss-utils: undefined () ignore-loader: undefined () image-size: undefined () is-animated: undefined () is-docker: undefined () is-wsl: undefined () jest-worker: undefined () json5: undefined () jsonwebtoken: undefined () loader-runner: undefined () loader-utils: undefined () lodash.curry: undefined () lru-cache: undefined () micromatch: undefined () mini-css-extract-plugin: undefined () nanoid: undefined () native-url: undefined () neo-async: undefined () next: 14.1.4 => 14.1.4 node-fetch: undefined () node-html-parser: undefined () ora: undefined () os-browserify: undefined () p-limit: undefined () path-browserify: undefined () platform: undefined () postcss: ^8.4.38 => 8.4.38 (8.4.31) postcss-flexbugs-fixes: undefined () postcss-modules-extract-imports: undefined () postcss-modules-local-by-default: undefined () postcss-modules-scope: undefined () postcss-modules-values: undefined () postcss-preset-env: undefined () postcss-safe-parser: undefined () postcss-scss: undefined () postcss-value-parser: undefined () process: undefined () punycode: undefined () querystring-es3: undefined () raw-body: undefined () react: ^18 => 18.2.0 react-builtin: undefined () react-dom: ^18 => 18.2.0 react-dom-builtin: undefined () react-dom-experimental-builtin: undefined () react-experimental-builtin: undefined () react-is: 18.2.0 react-refresh: 0.12.0 react-server-dom-turbopack-builtin: undefined () react-server-dom-turbopack-experimental-builtin: undefined () react-server-dom-webpack-builtin: undefined () react-server-dom-webpack-experimental-builtin: undefined () regenerator-runtime: 0.13.4 sass-loader: undefined () scheduler-builtin: undefined () scheduler-experimental-builtin: undefined () schema-utils: undefined () semver: undefined () send: undefined () server-only: 0.0.1 setimmediate: undefined () shell-quote: undefined () source-map: undefined () stacktrace-parser: undefined () stream-browserify: undefined () stream-http: undefined () string-hash: undefined () string_decoder: undefined () strip-ansi: undefined () superstruct: undefined () tailwindcss: ^3.4.3 => 3.4.3 tar: undefined () terser: undefined () text-table: undefined () timers-browserify: undefined () tsx: ^4.7.1 => 4.7.2 tty-browserify: undefined () typescript: ^5.4.3 => 5.4.5 (4.4.4, 4.9.5) ua-parser-js: undefined () unistore: undefined () util: undefined () vm-browserify: undefined () watchpack: undefined () web-vitals: undefined () webpack: undefined () webpack-sources: undefined () ws: undefined () zod: undefined () npmGlobalPackages: @aws-amplify/cli: 12.10.3 corepack: 0.22.0 npm: 10.2.4 ```

Describe the bug

storage\resource.ts:

import { defineStorage } from '@aws-amplify/backend';
import { generateThumbnail } from '../functions/thumbnail/resource';
import { getUploadUrl } from '../functions/get-upload-url/resource';
import { getPresignedUrl } from '../functions/presigned-url/resource';

export const storage = defineStorage({
  name: 'Videos',

  access: (allow) => ({
    'user-uploaded/*': [
        allow.authenticated.to(['write']),
        allow.resource(getUploadUrl).to(['write']),
        allow.resource(getPresignedUrl).to(['read']),
        allow.resource(generateThumbnail).to(['read'])
    ],
    'thumbnails/*': [
        allow.guest.to(['read']),
        allow.resource(generateThumbnail).to(['write'])
    ]
  }),
  triggers: {
    onUpload: generateThumbnail
  }
});

downloadData from another file:

import {downloadData} from '@aws-amplify/storage'

const image = downloadData({key: 'thumbnails/6d946476-4c41-4124-92b1-1f05b26c7080.jpg'});

Response from server:

<Error>
<Code>
AccessDenied
</Code>
<Message>
Access Denied
</Message>
<RequestId>
DWSECSC2RY8JG2GA
</RequestId>
<HostId>
mVqZ8sD0FHjIWf/wYQNQxUM0n2VLC9QLK2l4jbLSMiA5WW3XZeWDfYtwTBINRXPxdg1crm4bW+M=
</HostId>
</Error>

Request headers:

GET /public/thumbnails/6d946476-4c41-4124-92b1-1f05b26c7080.jpg HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate, br, zstd
Accept-Language: en-US,en;q=0.9,he;q=0.8,ru;q=0.7
Cache-Control: no-cache
Connection: keep-alive
Host: amplify-nextamplifygen2-guypa-videosbucketd93e5bfa-7ccjbkw4ic1y.s3.us-east-1.amazonaws.com
Origin: http://localhost:3000
Pragma: no-cache
Referer: http://localhost:3000/
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: cross-site
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36
authorization: AWS4-HMAC-SHA256 Credential=ASIA5R2IBV3JS6INALH3/20240423/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date;x-amz-security-token;x-amz-user-agent, Signature=86f5c7e2784f304bef4955366290851c771163b844dea9000542525f6f0733fa
sec-ch-ua: "Google Chrome";v="123", "Not:A-Brand";v="8", "Chromium";v="123"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "Windows"
x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date: 20240423T031604Z
x-amz-security-token: IQoJb3JpZ2luX2VjECMaCXVzLWVhc3QtMSJHMEUCIGvxUT+ntUSAI7vng0dDVpoWFQSmeB+jU89vXaCsKXcVAiEA75QbUCb3R90Z6yphpOhFkU0XjjYK4PT4syLUrcl1OBEqxAQIbBAEGgw5MzE2MjIxMzc1NTUiDMfD9gepLSlmenzM/yqhBEH7QLeKCPK74MijYhI1vPz23WKc6ZV0Qz27K1CgninA4mwrkF18GYQu6xOpA9jOiO2o5G0Iabfe0gzwlvV2LJ6OfoFPvSpS3vus0mGIz5tczRyxO0ZOF5XK7qrB5pdcCIUdba53cR+M0Oo4QHxKBH5Y0lESCO2k2qsJBF95IRgN/kNxuqVAgaC7TG6OAURr/zJ9bKDfTOu29M/hJV4hZrIqX6MOg4OHpehPJbwmZTQ6in3B455XbRwgHxqDcdmZd7aEyajQE/18T54nzjVdFWoBXoaY6Iq/pD0gZ/Ff2PglH7wGSizhsfGFa83ZT67LB0V5scEJAFoOvvEy8Mvf+iGJtGIywgq3802XJ1HWhZjU/eR+6vdpatmyd5Un9cJp+Mvu56NmhdbEW4L9aG/oLlMXB+dTjaaXGSyTbigbPsf+SA7fbOSz7dmEwCVqJmZGIKxlmiNAIVGLpEEgHleC6jhhq87R/3eEzBnZQi/asB3wJejxNjRYQGmsYSz+i4eREhpUvjVXIEEjUKYVdAizlqppGAoWNhiLsCstS0SDPiteHXA3VIsiBpCIExoqGAnUjJIodjvd/HIXOl3DVDejC58IsO+7fnV0L3xu1b/wF/2P+Rc6lKy+qjjKPHMWcs3vjvGEJvx9b3js+j5VjTQXmB8kpl4jQiHgnu9isBiYEPKCUw/eYFiGkvkMY4C57IQU+SNU/OqWMmUtLHZamuspu0yPMMC8nLEGOoUCRGc1EyzseZMiwMrsJiJ3kztwH/Ss5hgWgfR688i0D94VG+yJloo4Hach7joeTOJOFs6wL6ts30MeH3tMPZYgq9vXeh9L3lESfRITWVrVGoQRvUFcqPPblzeeqPqR5EVJiOdvYZR/2//M8yfGcZEyW1I0oCjZIUoUShSgXE3VzM7MrX4yDsK03WlJL1uAjt2UwDOU2oB/wwec42ZQJcHv/tZa/JJBYYnjBvhoDpnkLVhtA37Sc+afJs/bRBd2z4/kXIHMBidom9vOt1TsxkpGb53POqVZAhQ6XlSd6o0kkH7NXqKJLFw+AZTB56YL9KKtcNg4zEz4sq93zjWTRXvln8CRyzWq
x-amz-user-agent: aws-amplify/6.0.27 storage/2 framework/2

Expected behavior

Expected to get the file

Reproduction steps

Define the storage resources mentioned above and run downloadData for any object key under the authorized key prefix for allow.guest.to(['read'])

Code Snippet

// Put your code below this line.

Log output

``` // Put your logs below this line ```

aws-exports.js

No response

Manual configuration

No response

Additional configuration

No response

Mobile Device

No response

Mobile Operating System

No response

Mobile Browser

No response

Mobile Browser Version

No response

Additional information and screenshots

No response

ashika112 commented 6 months ago

Hi 👋 Thanks for opening up the issue. The feature to support the backend is not yet released since we are actively working on it. There is a tagged release gen2-storage which should help access this new backend.

Install npm install aws-amplify@gen2-storage

Update code to


import {downloadData} from 'aws-amplify/storage'

const image = downloadData({path: 'thumbnails/6d946476-4c41-4124-92b1-1f05b26c7080.jpg'});



More on this will come out with docs

strickon commented 6 months ago

I am seeing something with getUrl. I am trying to do a test where I upload a file , generate a link on a page, then download the file. The upload is working but the link that I am getting back using getUrl keeps giving me an access denied 403 error.

it successfully works for public access but not private.

I have added 'private/{entity_id}/*': [allow.entity('identity').to(['read', 'write', 'delete'])] to the access of my defineStorage

uploads work fine but downloads don't

cwomack commented 6 months ago

@gpavlov2016 and @strickon, you should now be able to verify/test this on v6.2.0 of Amplify. Can you upgrade to the latest version and let us know if you still experience this issue?

ashika112 commented 6 months ago

@strickon can you provide us with a code sample. The permission you have should let you read & write into private/CURRENT_USER_ID/*

strickon commented 6 months ago

I have figured out what I think is causing the problem but not sure how to address is. So I am using the amplify ui storage manger component to upload the file. It is going into a private folder in the bucket. I am then using getUrl to generate a presigned url using the server context. The problem I believe is that the url being generated doesn't match the s3 bucket url for the identity id portion. I am a logged in user uploading to a private folder.

I see the files in the bucket with a path.

Now I generate a url with getUrl using the server context and the path in the url is different from the path in the bucket. the identityId portion of the url is different.

I don't know which part is incorrect. Is the StorageManager putting the file in the wrong private folder with the wrong identity id or is the getUrl generating the wrong download url?

Everything works fine if I upload and download using public access level which puts it into the public folder.

here is the upload component from amplify-ui . it requires "use client"; (could this be an issue?)

export default function Upload() { return ( <StorageManager accessLevel='private' acceptedFileTypes={['.doc','.pdf','.docx']} autoUpload={false} maxFileCount={1} path={"upload/"} processFile={processFile} onUploadSuccess={({ key }) => { //Store files console.log('Upload Success '+key) CreateUpload(key) }}

); }

Download getUrl

Here is the function to create teh download url

const CreateDownload = async (key: string, access: string) => { return (await runWithAmplifyServerContext({ nextServerContext: null, operation: async (contextSpec) => (await getUrl(contextSpec, { key: key, options: { accessLevel: access, // can be 'private', 'protected', or 'guest' but defaults to guest }, })).url.toString()})) }

Let me know any thoughts.

strickon commented 6 months ago

I updated my app and it wasn't working but found the source of the issue and it works now. The documentation was a bit sparse.

The problem was that the the server context was null in the geturl example so it only worked for public files. the cookies needed to be passed in to generate the proper url for private access.

nextServerContext: { cookies },

ashika112 commented 6 months ago

Could you point out where u had difficulty with docs? We can update it to give clarity.

strickon commented 6 months ago

https://docs.amplify.aws/gen2/build-a-backend/server-side-rendering/#pageMain

The particular example with getUrl was a bit confusing with regards to the server context. It would be good to have some more detailed examples of the storage component and browser based uploads and downloads.

ashika112 commented 6 months ago

Thanks appreciate the feedback. It is in works and should start seeing more indepth docs soon :)

gpavlov2016 commented 6 months ago

@gpavlov2016 and @strickon, you should now be able to verify/test this on v6.2.0 of Amplify. Can you upgrade to the latest version and let us know if you still experience this issue?

@cwomack Upgrading to v6.2 solved the issue, thank you! I did have to use the field name path in the argument object to getUrl while the value is called key in S3 console so its a bit confusing because there is another field called key in the param description. In summary, it worked as follows: getUrl({ path: 'thumbnails/4081c322-0086-4035-9181-02c124e99350.jpg' })

ashika112 commented 6 months ago

@gpavlov2016 Thanks for the feedback and we will work on it. We have updated our docs site to reflect the new usage. key usage is going to be deprecated in future version and all API using key should have a warning for it.

https://docs.amplify.aws/react/build-a-backend/storage/set-up-storage/

Hopefully this helps clarify the storage API usage. Let us know if there is anything else we can help with or we can close the issue.

danCoda commented 2 months ago

Hi guys, apologies for bringing this one up again but I think this is the best place.

I'm new to AWS and am learning Amplify, and following this: https://docs.amplify.aws/react/build-a-backend/storage/set-up-storage/#upload-your-first-file.

At that step, when I try to actually upload a file, I get a 403 AccessDenied Amplify error when trying to use the uploadData function.

I think I have set everything right.

//backend.ts:
defineBackend({
  auth,
  data,
  sayHello,
  storage
});

// storage/resource.ts:
export const storage = defineStorage({
  name: "photosStorage",
  access: (allow) => ({
    "experiences/{entity_id}/*": [
      allow.entity("identity").to(["read", "write", "delete"]),
      allow.guest.to(["read"]),
    ],
  }),
});

            <button
              onClick={() =>
                uploadData({
                  path: `experiences/${file.name}`,
                  data: file,
                })
              }
            >
              Upload
            </button>

Anyone else in my situation / AccessDenied error when following the tutorial? Not sure where I went wrong.

chrisbonifacio commented 2 months ago

Hi @danCoda 👋

It looks like your access rules only allows guest (unauthenticated) users to read from the bucket but you are trying to write to the bucket.

To write to the bucket, your access rules allow authenticated users with an entity id. You have to pass the user's entity id into the file path like so:

import { uploadData } from 'aws-amplify/storage';

try {
  const result = await uploadData({
    path: ({identityId}) => `experiences/${identityId}/${file.name}.jpg`
    data: file,
  }).result;
  console.log('Succeeded: ', result);
} catch (error) {
  console.log('Error : ', error);
}

danCoda commented 2 months ago

Appreciate your assistance @chrisbonifacio !

(Tip for anyone - I feel that the AWS sandbox needs to be restarted every now and again)

I got it working with your suggestion. I was also referencing this Amplify Storage tutorial and I there is some misunderstanding. Why did I have to remove the bucket option below to get it working (along with your suggestion)?

  async function tryDownload() {
    try {
      const result = await downloadData({
        path: ({ identityId }) => `profile-pictures/${identityId}/image.jpg`, // User's entity id has been passed.
        options: {
          // bucket: "amplifyTeamDrive", // NOTE: No good, I had to remove this to work.
        },
      }).result;
      console.log("Result", result);
    } catch (error) {
      console.log(`Error: ${error}`);
    }
  }

// My storage configuration:
export const storage = defineStorage({
  name: "amplifyTeamDrive", // NOTE: This is the bucket name?
  access: (allow) => ({
    "profile-pictures/{entity_id}/*": [
        allow.entity("identity").to(["read", "write", "delete"]),
    ],
  }),
});

aws-amplify / amplify-js