search

aws-amplify / amplify-js

A declarative JavaScript library for application development using cloud services.
https://docs.amplify.aws/lib/q/platform/js
Apache License 2.0
9.42k stars 2.12k forks source link

Unable to generate client schema when adding external RDS database #13885

Closed outraday closed 2 weeks ago

outraday commented 2 weeks ago

Before opening, please confirm:

JavaScript Framework

React

Amplify APIs

GraphQL API, DataStore

Amplify Version

v6

Amplify Categories

api

Backend

Amplify Gen 2 (Preview)

Environment information

``` # Put output below this line System: OS: Windows 10 10.0.19045 CPU: (4) x64 AMD Ryzen 3 3200G with Radeon Vega Graphics Memory: 1.74 GB / 15.95 GB Binaries: Node: 18.20.3 - C:\Program Files\nodejs\node.EXE npm: 10.8.3 - C:\Program Files\nodejs\npm.CMD Browsers: Edge: Chromium (127.0.2651.74) Internet Explorer: 11.0.19041.4355 npmPackages: %name%: 0.1.0 @aws-amplify/backend: ^1.2.1 => 1.2.1 @aws-amplify/backend-cli: ^1.2.5 => 1.2.5 @aws-amplify/ui-react: ^6.1.12 => 6.1.13 @aws-amplify/ui-react-internal: undefined () @aws-amplify/ui-react-storage: ^3.1.4 => 3.1.4 @aws-sdk/client-cognito-identity-provider: ^3.614.0 => 3.614.0 @babel/preset-env: ^7.24.7 => 7.24.7 @babel/preset-react: ^7.24.7 => 7.24.7 @babel/preset-typescript: ^7.24.7 => 7.24.7 @dnd-kit/core: ^6.1.0 => 6.1.0 @dnd-kit/modifiers: ^7.0.0 => 7.0.0 @dnd-kit/sortable: ^8.0.0 => 8.0.0 @dnd-kit/utilities: ^3.2.2 => 3.2.2 @emoji-mart/react: ^1.1.1 => 1.1.1 @emotion/styled: ^11.11.0 => 11.11.5 @floating-ui/dom: ^1.5.3 => 1.6.7 (0.5.4) @floating-ui/react: ^0.26.5 => 0.26.19 (0.19.2) @fluentui/react-context-selector: ^9.1.65 => 9.1.65 @lexical/clipboard: ^0.12.6 => 0.12.6 @lexical/code: ^0.12.5 => 0.12.6 @lexical/hashtag: ^0.12.6 => 0.12.6 @lexical/link: ^0.12.5 => 0.12.6 @lexical/list: ^0.12.5 => 0.12.6 @lexical/mark: ^0.12.5 => 0.12.6 @lexical/markdown: ^0.12.5 => 0.12.6 @lexical/react: ^0.12.5 => 0.12.6 @lexical/rich-text: ^0.12.5 => 0.12.6 @lexical/selection: ^0.12.5 => 0.12.6 @lexical/utils: ^0.12.5 => 0.12.6 @lingui/cli: ^4.11.2 => 4.11.2 @lingui/core: ^4.5.0 => 4.11.2 @lingui/detect-locale: ^4.5.0 => 4.11.2 @lingui/format-po: ^4.5.0 => 4.11.2 @lingui/macro: ^4.5.0 => 4.11.2 @lingui/react: ^4.5.0 => 4.11.2 @lingui/vite-plugin: ^4.11.2 => 4.11.2 @reduxjs/toolkit: ^2.2.6 => 2.2.6 @reduxjs/toolkit-query: 1.0.0 @reduxjs/toolkit-query-react: 1.0.0 @reduxjs/toolkit-react: 1.0.0 @tailwindcss/line-clamp: ^0.4.4 => 0.4.4 @tanstack/eslint-plugin-query: ^5.35.6 => 5.50.1 @tanstack/query-codemods: 4.24.3 @tanstack/react-query: ^5.40.0 => 5.50.1 @tanstack/react-table: ^8.12.0 => 8.19.2 @tanstack/react-virtual: ^3.3.0 => 3.8.2 @types/emoji-mart: ^3.0.14 => 3.0.14 @types/ip: ^1.1.3 => 1.1.3 @types/lodash: ^4.14.200 => 4.17.6 @types/node: ^20 => 20.14.10 @types/react-resizable: ^3.0.8 => 3.0.8 @types/ua-parser-js: ^0.7.39 => 0.7.39 @typescript-eslint/eslint-plugin: ^7.13.1 => 7.16.0 (5.62.0) @typescript-eslint/parser: ^7.13.1 => 7.16.0 (5.62.0, 6.21.0) @vitejs/plugin-react: ^4.3.1 => 4.3.1 alphanumeric-id: ^1.0.1 => 1.0.1 antd: ^5.11.1 => 5.19.1 aws-amplify: ^6.6.0 => 6.6.0 aws-amplify/adapter-core: undefined () aws-amplify/analytics: undefined () aws-amplify/analytics/kinesis: undefined () aws-amplify/analytics/kinesis-firehose: undefined () aws-amplify/analytics/personalize: undefined () aws-amplify/analytics/pinpoint: undefined () aws-amplify/api: undefined () aws-amplify/api/server: undefined () aws-amplify/auth: undefined () aws-amplify/auth/cognito: undefined () aws-amplify/auth/cognito/server: undefined () aws-amplify/auth/enable-oauth-listener: undefined () aws-amplify/auth/server: undefined () aws-amplify/data: undefined () aws-amplify/data/server: undefined () aws-amplify/datastore: undefined () aws-amplify/in-app-messaging: undefined () aws-amplify/in-app-messaging/pinpoint: undefined () aws-amplify/push-notifications: undefined () aws-amplify/push-notifications/pinpoint: undefined () aws-amplify/storage: undefined () aws-amplify/storage/s3: undefined () aws-amplify/storage/s3/server: undefined () aws-amplify/storage/server: undefined () aws-amplify/utils: undefined () aws-cdk: ^2.148.0 => 2.148.0 aws-cdk-lib: ^2.148.0 => 2.148.0 aws-sdk: ^2.1667.0 => 2.1667.0 axios: ^1.7.2 => 1.7.2 babel-plugin-macros: ^3.1.0 => 3.1.0 clsx: ^2.1.0 => 2.1.1 (1.1.1, 1.2.1) constructs: ^10.3.0 => 10.3.0 dayjs: ^1.11.10 => 1.11.11 emoji-mart: ^5.5.2 => 5.6.0 emoji-picker-react: ^4.5.16 => 4.11.1 esbuild: ^0.23.0 => 0.23.0 (0.17.19, 0.21.5) eslint: ^8.57.0 => 8.57.0 eslint-config-airbnb: ^19.0.4 => 19.0.4 eslint-config-airbnb-typescript: ^18.0.0 => 18.0.0 eslint-config-prettier: ^8.10.0 => 8.10.0 eslint-config-react-app: ^7.0.1 => 7.0.1 eslint-import-resolver-typescript: ^3.6.1 => 3.6.1 eslint-plugin-css-modules: ^2.11.4 => 2.12.0 eslint-plugin-filename-rules: ^1.3.1 => 1.3.1 eslint-plugin-flowtype: ^8.0.3 => 8.0.3 eslint-plugin-html: ^7.1.0 => 7.1.0 eslint-plugin-import: ^2.28.1 => 2.29.1 eslint-plugin-jsx-a11y: ^6.7.1 => 6.9.0 eslint-plugin-lingui: ^0.2.0 => 0.2.2 eslint-plugin-no-barrel-files: ^1.1.1 => 1.1.1 eslint-plugin-prettier: ^5.1.3 => 5.1.3 eslint-plugin-react: ^7.33.2 => 7.34.3 eslint-plugin-react-hooks: ^4.6.2 => 4.6.2 eslint-plugin-react-refresh: ^0.4.7 => 0.4.7 eslint-plugin-simple-import-sort: ^10.0.0 => 10.0.0 fast-xml-parser: ^4.4.1 => 4.4.1 (4.2.5) flexlayout-react: ^0.7.15 => 0.7.15 framer-motion: ^12.0.0-alpha.0 => 12.0.0-alpha.0 history: ^5.3.0 => 5.3.0 immer: ^10.0.4 => 10.1.1 (9.0.21, 9.0.6) iso3166-2-db: ^2.3.10 => 2.3.10 jotai: ^2.8.0 => 2.9.0 jotai-devtools: ^0.8.0 => 0.8.0 jotai-effect: ^1.0.0 => 1.0.0 jotai-optics: ^0.4.0 => 0.4.0 jotai-scope: ^0.5.1 => 0.5.2 lexical: ^0.12.5 => 0.12.6 lexical-beautiful-mentions: ^0.1.36 => 0.1.36 lodash: ^4.17.21 => 4.17.21 optics-ts: ^2.4.1 => 2.4.1 prettier: ^3.2.5 => 3.3.2 (2.3.2, 2.8.8, 1.19.1) prettier-eslint: ^16.1.1 => 16.3.0 prettier-plugin-tailwindcss: ^0.5.12 => 0.5.14 react: ^19.0.0-rc-163365a0-20240717 => 19.0.0-rc-163365a0-20240717 react-dom: ^19.0.0-rc-163365a0-20240717 => 19.0.0-rc-163365a0-20240717 react-draggable: ^4.4.6 => 4.4.6 react-dropzone: ^14.2.3 => 14.2.3 react-error-boundary: ^4.0.13 => 4.0.13 (3.1.4) react-hot-toast: ^2.4.1 => 2.4.1 react-icons: ^5.0.1 => 5.2.1 react-image-file-resizer: ^0.4.8 => 0.4.8 react-loading-skeleton: ^3.3.1 => 3.4.0 react-redux: ^9.1.2 => 9.1.2 react-resizable: ^3.0.5 => 3.0.5 react-resize-detector: ^9.1.0 => 9.1.1 react-router-dom: ^6.20.0 => 6.24.1 react-spinners: ^0.14.1 => 0.14.1 react-table: ^7.8.0 => 7.8.0 react-tag-autocomplete: ^7.2.0 => 7.3.0 react-textarea-autosize: ^8.5.3 => 8.5.3 (8.3.4) react-virtuoso: ^4.10.3 => 4.10.3 (2.19.1) redux-first-history: ^5.2.0 => 5.2.0 redux-persist: ^6.0.0 => 6.0.0 redux-persist/integration/react: undefined () redux-thunk: ^3.1.0 => 3.1.0 reselect: ^5.1.1 => 5.1.1 stream-chat: ^8.14.4 => 8.37.0 stream-chat-react: ^11.2.1 => 11.23.1 stylelint: ^16.3.1 => 16.6.1 stylelint-config-recess-order: ^5.0.0 => 5.0.1 stylelint-config-recommended: ^13.0.0 => 13.0.0 (14.0.1) stylelint-config-standard-scss: ^13.0.0 => 13.1.0 stylelint-config-tailwindcss: ^0.0.7 => 0.0.7 stylelint-order: ^6.0.4 => 6.0.4 stylelint-prettier: ^5.0.0 => 5.0.0 tailwind-merge: ^2.4.0 => 2.4.0 tailwind-scrollbar-hide: ^1.1.7 => 1.1.7 tailwindcss: ^3.4.1 => 3.4.4 ts-enum-util: ^4.1.0 => 4.1.0 tsx: ^4.16.2 => 4.16.2 types-react: ^19.0.0-rc.1 => 19.0.0-rc.1 types-react-dom: ^19.0.0-rc.1 => 19.0.0-rc.1 (19.0.0) typescript: ^5.2.2 => 5.5.3 (4.4.4, 4.9.5) typescript-eslint: ^7.15.0 => 7.16.0 typescript-plugin-css-modules: ^5.0.2 => 5.1.0 typesense: ^1.8.2 => 1.8.2 ua-parser-js: ^1.0.38 => 1.0.38 vite: ^5.3.1 => 5.3.3 vite-plugin-babel: ^1.2.0 => 1.2.0 vite-plugin-babel-macros: ^1.0.6 => 1.0.6 vite-plugin-eslint: ^1.8.1 => 1.8.1 vite-tsconfig-paths: ^4.3.2 => 4.3.2 npmGlobalPackages: aws-cdk: 2.144.0 corepack: 0.28.0 create-vite: 5.3.0 dpdm: 3.14.0 eslint: 9.9.1 firebase-tools: 13.11.0 npm: 10.8.3 rimraf: 5.0.10 ```

Describe the bug

When adding an external postgresql schema with combine I get an error in function excludeDisabledOps in clientUtils.ts file:

TypeError: coarseToFineDict[key] is not iterable (cannot read property undefined)

I digged into it and found out that models that are generated after configuring the external database have an attribute of type model where the only key in the properties object is timestamp. The function iterates over the keys and tries to spread out coarseToFineDict[key]. The only keys in this dict are queries, mutations, subscriptions. No key timestamp exists, therefore the value is undefined and trying to spread it gives the error. On regular schema models the properties object is empty and no error occurs.

Expected behavior

The external database schema from postgres can be genereated without any errors and then be combined with the regular datastore schema.

Reproduction steps

Following the steps from the guide: https://docs.amplify.aws/react/build-a-backend/data/connect-to-existing-data-sources/connect-postgres-mysql-database/#step-2---generate-typescript-representation-of-your-database-schema

  1. Typescript representation of the database schema: npx ampx generate schema-from-database --connection-uri-secret SQL_CONNECTION_STRING --out amplify/data/schema.sql.ts

  2. Combining the schemas with the combine method

  3. Run ampx sandbox

  4. Start the application

Code Snippet

The schema.sql.file

export const generatedSqlSchema = configure({
    database: {
        identifier: "XXX",
        engine: "mysql",
        connectionUri: secret("PSQL_BACKEND_CONNECTION_STRING"),
        vpcConfig: {
            vpcId: "vpc-XXX",
            securityGroupIds: ["sg-XXX"],
            subnetAvailabilityZones: [
                {
                    subnetId: "subnet-XXX",
                    availabilityZone: "eu-central-1b"
                },
                {
                    subnetId: "subnet-XXX",
                    availabilityZone: "eu-central-1c"
                },
                {
                    subnetId: "subnet-XXX",
                    availabilityZone: "eu-central-1a"
                }
            ]
        }
    }
}).schema({
    Test: a
        .model({
            id: a.string().required()
        })
        .identifier(["id"])
});

Combining the schemas:

const dataBackendSchema = generatedSqlSchema.authorization((allow) => allow.guest());

...<regular schema>...

const combinedSchema = a.combine([schema, dataBackendSchema]);

export type Schema = ClientSchema<typeof combinedSchema>;

export const data = defineData({
    schema: combinedSchema,
    authorizationModes: {
        defaultAuthorizationMode: "userPool"
    }
});

Log output

TypeError: coarseToFineDict[key] is not iterable (cannot read property undefined) at excludeDisabledOps (clientUtils.ts:47:29) at generateModelsProperty (generateModelsProperty.ts:17:33) at addSchemaToClient (addSchemaToClient.ts:6:21) at generateClient (generateClient.ts:35:9) at generateClient2 (API.ts:10:12) at amplify-client.ts:10:30

aws-exports.js

No response

Manual configuration

No response

Additional configuration

No response

Mobile Device

No response

Mobile Operating System

No response

Mobile Browser

No response

Mobile Browser Version

No response

Additional information and screenshots

No response

outraday commented 2 weeks ago

For everybody else having this error. It is not a bug but a missing package.

The autogenerated .sql file has eslint disabled which did not show me that I missed @aws-amplify/data-schema package. Installed it as dev dependency and then it worked.