search

aws-amplify / amplify-category-api

The AWS Amplify CLI is a toolchain for simplifying serverless web and mobile development. This plugin provides functionality for the API category, allowing for the creation and management of GraphQL and REST based backends for your amplify project.
https://docs.amplify.aws/
Apache License 2.0
89 stars 79 forks source link

On adding a secondary index, my data gets dropped from my table #2795

Open tjma2001 opened 3 months ago

tjma2001 commented 3 months ago

Before opening, please confirm:

JavaScript Framework

React

Amplify APIs

DataStore

Amplify Version

v6

Amplify Categories

storage

Backend

Amplify Gen 2 (Preview)

Environment information

``` # Put output below this line System: OS: Windows 11 10.0.22631 CPU: (16) x64 AMD Ryzen 7 7840HS w/ Radeon 780M Graphics Memory: 10.43 GB / 31.29 GB Binaries: Node: 18.17.1 - C:\Program Files\nodejs\node.EXE Yarn: 1.22.19 - C:\Program Files\nodejs\yarn.CMD npm: 9.6.7 - C:\Program Files\nodejs\npm.CMD Browsers: Edge: Chromium (127.0.2651.74) Internet Explorer: 11.0.22621.3527 npmPackages: %name%: 0.1.0 @aws-amplify/backend: 1.1.1 => 1.1.1 @aws-amplify/backend-cli: 1.2.3 => 1.2.3 @aws-amplify/ui-react: ^6.1.5 => 6.1.8 @aws-amplify/ui-react-internal: undefined () @aws-lambda-powertools/logger: ^2.1.0 => 2.1.0 @aws-sdk/client-sqs: ^3.569.0 => 3.569.0 @emotion/react: ^11.8.2 => 11.11.4 @emotion/styled: ^11.8.1 => 11.11.5 @mui/icons-material: ^5.8.3 => 5.15.15 @mui/lab: ^5.0.0-alpha.85 => 5.0.0-alpha.170 @mui/material: ^5.8.6 => 5.15.15 @mui/x-data-grid: ^6.6.0 => 6.19.11 @refinedev/cli: ^2.16.21 => 2.16.29 @refinedev/core: ^4.46.1 => 4.49.0 @refinedev/devtools: ^1.1.29 => 1.1.37 @refinedev/graphql: ^6.4.8 => 6.5.0 @refinedev/inferencer: ^4.5.16 => 4.6.0 @refinedev/kbar: ^1.3.5 => 1.3.8 @refinedev/mui: ^5.14.1 => 5.14.6 @refinedev/react-hook-form: ^4.8.13 => 4.8.16 @refinedev/react-router-v6: ^4.5.5 => 4.5.7 @refinedev/simple-rest: ^5.0.1 => 5.0.4 @sentry/react: ^8.22.0 => 8.22.0 @types/aws-lambda: ^8.10.137 => 8.10.137 @types/graphql: ^14.5.0 => 14.5.0 @types/node: ^18.16.2 => 18.19.31 (20.16.1) @types/react: ^18.0.0 => 18.2.79 @types/react-dom: ^18.0.0 => 18.2.25 @types/rollup: ^0.54.0 => 0.54.0 @typescript-eslint/eslint-plugin: ^5.57.1 => 5.62.0 @typescript-eslint/parser: ^5.57.1 => 5.62.0 @vitejs/plugin-react: ^4.0.0 => 4.2.1 aws-amplify: ^6.3.6 => 6.3.6 aws-amplify/adapter-core: undefined () aws-amplify/analytics: undefined () aws-amplify/analytics/kinesis: undefined () aws-amplify/analytics/kinesis-firehose: undefined () aws-amplify/analytics/personalize: undefined () aws-amplify/analytics/pinpoint: undefined () aws-amplify/api: undefined () aws-amplify/api/server: undefined () aws-amplify/auth: undefined () aws-amplify/auth/cognito: undefined () aws-amplify/auth/cognito/server: undefined () aws-amplify/auth/enable-oauth-listener: undefined () aws-amplify/auth/server: undefined () aws-amplify/data: undefined () aws-amplify/data/server: undefined () aws-amplify/datastore: undefined () aws-amplify/in-app-messaging: undefined () aws-amplify/in-app-messaging/pinpoint: undefined () aws-amplify/push-notifications: undefined () aws-amplify/push-notifications/pinpoint: undefined () aws-amplify/storage: undefined () aws-amplify/storage/s3: undefined () aws-amplify/storage/s3/server: undefined () aws-amplify/storage/server: undefined () aws-amplify/utils: undefined () aws-cdk: ^2.138.0 => 2.138.0 aws-cdk-lib: ^2.138.0 => 2.138.0 aws-sdk: ^2.1677.0 => 2.1677.0 axios: ^1.6.2 => 1.6.8 concurrently: ^8.2.2 => 8.2.2 constructs: ^10.3.0 => 10.3.0 dotenv: ^16.4.5 => 16.4.5 esbuild: ^0.20.2 => 0.20.2 (0.19.12) eslint: ^8.38.0 => 8.57.0 eslint-plugin-react-hooks: ^4.6.0 => 4.6.0 eslint-plugin-react-refresh: ^0.4.5 => 0.4.6 i18next: ^20.1.0 => 20.6.1 i18next-browser-languagedetector: ^6.1.1 => 6.1.8 i18next-xhr-backend: ^3.2.2 => 3.2.2 qrcode.react: ^3.1.0 => 3.1.0 qrcodejs: ^1.0.0 => 1.0.0 react: ^18.0.0 => 18.2.0 react-dom: ^18.0.0 => 18.2.0 react-hook-form: ^7.30.0 => 7.51.3 react-i18next: ^11.8.11 => 11.18.6 react-router-dom: ^6.8.1 => 6.23.0 tsx: ^4.7.2 => 4.7.2 typescript: ^5.4.5 => 5.4.5 (4.4.4, 4.9.5) vite: ^5.1.3 => 5.2.10 ```

Describe the bug

When I add a global secondary index to a model, .secondaryIndexes((index) => [index('apiKey').sortKeys(['dateStamp'])]),, it drops all that data from my dynamodb tables.

Expected behavior

I expect it to work, the same way, that adding or removing fields from a model should work. I know if I add a GSI on the AWS web interface, that it will not drop my data at all.

Reproduction steps

  1. Create a model
  2. Add data to the model
  3. Add a GSI via resource.ts

Code Snippet

// Put your code below this line.

Log output

``` // Put your logs below this line ```

aws-exports.js

No response

Manual configuration

No response

Additional configuration

No response

Mobile Device

No response

Mobile Operating System

No response

Mobile Browser

No response

Mobile Browser Version

No response

Additional information and screenshots

No response

chrisbonifacio commented 2 months ago

Hi @tjma2001, thanks for raising this issue 👋

This sounds like expected behavior though. By default, in sandbox environments, destructive updates are allowed on the API resources. This is not the case in production environments.

Are you observing this in a sandbox or production environment?

tjma2001 commented 2 months ago

Hi @tjma2001, thanks for raising this issue 👋

This sounds like expected behavior though. By default, in sandbox environments, destructive updates are allowed on the API resources. This is not the case in production environments.

Are you observing this in a sandbox or production environment?

I am observing this in sandbox environment. I haven't tried doing this on production yet. Perhaps I can spin up a dummy table, add some data, add the GSI and see what the effect is.

I have not seen this behaviour in a sandbox environment which is why I opened up the ticket. Could this be because it takes quite a while to create a GSI on a table that already exists?

tjma2001 commented 2 months ago

Hi @tjma2001, thanks for raising this issue 👋

This sounds like expected behavior though. By default, in sandbox environments, destructive updates are allowed on the API resources. This is not the case in production environments.

Are you observing this in a sandbox or production environment?

I just performed a test in production.

thanks for the assist.

tjma2001 commented 2 months ago

The other thing to note is that it significantly increases your deployment time. I would suggest only adding one GSI per deployment unless these updates are performed in parallel.

AnilMaktala commented 1 month ago

Hey @tjma2001, Thank you for bringing this up. I was able to replicate the issue in the sandbox environment but not in production. This is expected behavior in the sandbox to minimize deployment time. However, I’ll flag this as a feature request for the team to explore options for retaining data during sandbox deployments.

j0kerZ commented 1 month ago

+1 for this feature. I don't want to seed the data every time we make changes to the schema 😥 (And this freaked me out so I didn't even dare to test on production 😢)

jaison2007 commented 1 month ago

I agree - this is very surprising behavior but glad that it only affects the sandbox. Is it possible to document this somewhere?