aws-amplify / amplify-category-api

The AWS Amplify CLI is a toolchain for simplifying serverless web and mobile development. This plugin provides functionality for the API category, allowing for the creation and management of GraphQL and REST based backends for your amplify project.
https://docs.amplify.aws/
Apache License 2.0
89 stars 79 forks source link

Auto-generation of Data Schema for PostgreSQL Ignores Non-Public Schemas #2640

Open TazChoudhury opened 5 months ago

TazChoudhury commented 5 months ago

Before opening, please confirm:

JavaScript Framework

React

Amplify APIs

Storage

Amplify Version

v6

Amplify Categories

storage

Backend

Amplify Gen 2 (Preview)

Environment information

``` # Put output below this line System: OS: macOS 14.5 CPU: (10) arm64 Apple M1 Pro Memory: 359.33 MB / 16.00 GB Shell: 5.9 - /bin/zsh Binaries: Node: 22.2.0 - ~/.nvm/versions/node/v22.2.0/bin/node Yarn: 1.22.22 - ~/.nvm/versions/node/v22.2.0/bin/yarn npm: 10.8.1 - ~/.nvm/versions/node/v22.2.0/bin/npm pnpm: 9.4.0 - ~/.nvm/versions/node/v22.2.0/bin/pnpm Browsers: Chrome: 126.0.6478.62 Edge: 126.0.2592.61 Safari: 17.5 npmPackages: %name%: 0.1.0 @aws-amplify/backend: ^1.0.0 => 1.0.0 @aws-amplify/backend-cli: ^1.0.1 => 1.0.1 @aws-amplify/ui-react: ^6.1.9 => 6.1.9 @aws-amplify/ui-react-internal: undefined () @types/react: ^18.2.66 => 18.3.1 @types/react-dom: ^18.2.22 => 18.3.0 @typescript-eslint/eslint-plugin: ^7.2.0 => 7.8.0 @typescript-eslint/parser: ^7.2.0 => 7.8.0 @vitejs/plugin-react: ^4.2.1 => 4.2.1 aws-amplify: ^6.2.0 => 6.2.0 aws-amplify/adapter-core: undefined () aws-amplify/analytics: undefined () aws-amplify/analytics/kinesis: undefined () aws-amplify/analytics/kinesis-firehose: undefined () aws-amplify/analytics/personalize: undefined () aws-amplify/analytics/pinpoint: undefined () aws-amplify/api: undefined () aws-amplify/api/server: undefined () aws-amplify/auth: undefined () aws-amplify/auth/cognito: undefined () aws-amplify/auth/cognito/server: undefined () aws-amplify/auth/enable-oauth-listener: undefined () aws-amplify/auth/server: undefined () aws-amplify/data: undefined () aws-amplify/data/server: undefined () aws-amplify/datastore: undefined () aws-amplify/in-app-messaging: undefined () aws-amplify/in-app-messaging/pinpoint: undefined () aws-amplify/push-notifications: undefined () aws-amplify/push-notifications/pinpoint: undefined () aws-amplify/storage: undefined () aws-amplify/storage/s3: undefined () aws-amplify/storage/s3/server: undefined () aws-amplify/storage/server: undefined () aws-amplify/utils: undefined () aws-cdk: ^2.138.0 => 2.140.0 aws-cdk-lib: ^2.138.0 => 2.140.0 constructs: ^10.3.0 => 10.3.0 esbuild: ^0.20.2 => 0.20.2 eslint: ^8.57.0 => 8.57.0 eslint-plugin-react-hooks: ^4.6.0 => 4.6.2 eslint-plugin-react-refresh: ^0.4.6 => 0.4.6 react: ^18.2.0 => 18.3.1 react-dom: ^18.2.0 => 18.3.1 rsuite: ^5.64.1 => 5.64.1 rsuite/Accordion: undefined () rsuite/AccordionPanel: undefined () rsuite/Affix: undefined () rsuite/Animation: undefined () rsuite/AutoComplete: undefined () rsuite/Avatar: undefined () rsuite/AvatarGroup: undefined () rsuite/Badge: undefined () rsuite/Bounce: undefined () rsuite/Breadcrumb: undefined () rsuite/BreadcrumbItem: undefined () rsuite/Button: undefined () rsuite/ButtonGroup: undefined () rsuite/ButtonToolbar: undefined () rsuite/Calendar: undefined () rsuite/Carousel: undefined () rsuite/CascadeTree: undefined () rsuite/Cascader: undefined () rsuite/CheckPicker: undefined () rsuite/CheckTree: undefined () rsuite/CheckTreePicker: undefined () rsuite/Checkbox: undefined () rsuite/CheckboxGroup: undefined () rsuite/Col: undefined () rsuite/Collapse: undefined () rsuite/Container: undefined () rsuite/Content: undefined () rsuite/CustomProvider: undefined () rsuite/DOMHelper: undefined () rsuite/DateInput: undefined () rsuite/DatePicker: undefined () rsuite/DateRangeInput: undefined () rsuite/DateRangePicker: undefined () rsuite/Divider: undefined () rsuite/Drawer: undefined () rsuite/DrawerActions: undefined () rsuite/DrawerBody: undefined () rsuite/DrawerFooter: undefined () rsuite/DrawerHeader: undefined () rsuite/DrawerTitle: undefined () rsuite/Dropdown: undefined () rsuite/DropdownItem: undefined () rsuite/DropdownMenu: undefined () rsuite/DropdownSeparator: undefined () rsuite/Fade: undefined () rsuite/FlexboxGrid: undefined () rsuite/FlexboxGridItem: undefined () rsuite/Footer: undefined () rsuite/Form: undefined () rsuite/FormControl: undefined () rsuite/FormControlLabel: undefined () rsuite/FormErrorMessage: undefined () rsuite/FormGroup: undefined () rsuite/FormHelpText: undefined () rsuite/Grid: undefined () rsuite/Header: undefined () rsuite/Heading: undefined () rsuite/HeadingGroup: undefined () rsuite/Highlight: undefined () rsuite/IconButton: undefined () rsuite/InlineEdit: undefined () rsuite/Input: undefined () rsuite/InputGroup: undefined () rsuite/InputGroupAddon: undefined () rsuite/InputGroupButton: undefined () rsuite/InputNumber: undefined () rsuite/InputPicker: undefined () rsuite/List: undefined () rsuite/ListItem: undefined () rsuite/Loader: undefined () rsuite/MaskedInput: undefined () rsuite/Message: undefined () rsuite/Modal: undefined () rsuite/ModalBody: undefined () rsuite/ModalDialog: undefined () rsuite/ModalFooter: undefined () rsuite/ModalHeader: undefined () rsuite/ModalTitle: undefined () rsuite/MultiCascadeTree: undefined () rsuite/MultiCascader: undefined () rsuite/Nav: undefined () rsuite/NavItem: undefined () rsuite/NavMenu: undefined () rsuite/Navbar: undefined () rsuite/NavbarBrand: undefined () rsuite/Notification: undefined () rsuite/Pagination: undefined () rsuite/Panel: undefined () rsuite/PanelGroup: undefined () rsuite/Placeholder: undefined () rsuite/PlaceholderGraph: undefined () rsuite/PlaceholderGrid: undefined () rsuite/PlaceholderParagraph: undefined () rsuite/Popover: undefined () rsuite/Progress: undefined () rsuite/ProgressCircle: undefined () rsuite/ProgressLine: undefined () rsuite/Radio: undefined () rsuite/RadioGroup: undefined () rsuite/RadioTile: undefined () rsuite/RadioTileGroup: undefined () rsuite/RangeSlider: undefined () rsuite/Rate: undefined () rsuite/Row: undefined () rsuite/SafeAnchor: undefined () rsuite/Schema: undefined () rsuite/SelectPicker: undefined () rsuite/Sidebar: undefined () rsuite/Sidenav: undefined () rsuite/SidenavBody: undefined () rsuite/SidenavHeader: undefined () rsuite/SidenavToggle: undefined () rsuite/Slide: undefined () rsuite/Slider: undefined () rsuite/Stack: undefined () rsuite/StackItem: undefined () rsuite/StepItem: undefined () rsuite/Steps: undefined () rsuite/Tab: undefined () rsuite/Table: undefined () rsuite/TableCell: undefined () rsuite/TableColumn: undefined () rsuite/TableColumnGroup: undefined () rsuite/TableHeaderCell: undefined () rsuite/Tabs: undefined () rsuite/Tag: undefined () rsuite/TagGroup: undefined () rsuite/TagInput: undefined () rsuite/TagPicker: undefined () rsuite/Text: undefined () rsuite/Timeline: undefined () rsuite/TimelineItem: undefined () rsuite/Toggle: undefined () rsuite/Tooltip: undefined () rsuite/Transition: undefined () rsuite/Tree: undefined () rsuite/TreePicker: undefined () rsuite/Uploader: undefined () rsuite/VisuallyHidden: undefined () rsuite/Whisper: undefined () rsuite/locales: undefined () rsuite/locales/ar_EG: undefined () rsuite/locales/cs_CZ: undefined () rsuite/locales/da_DK: undefined () rsuite/locales/de_DE: undefined () rsuite/locales/default: undefined () rsuite/locales/en_GB: undefined () rsuite/locales/en_US: undefined () rsuite/locales/es_AR: undefined () rsuite/locales/es_ES: undefined () rsuite/locales/fa_IR: undefined () rsuite/locales/fi_FI: undefined () rsuite/locales/fr_FR: undefined () rsuite/locales/hu_HU: undefined () rsuite/locales/it_IT: undefined () rsuite/locales/ja_JP: undefined () rsuite/locales/kk_KZ: undefined () rsuite/locales/ko_KR: undefined () rsuite/locales/ne_NP: undefined () rsuite/locales/nl_NL: undefined () rsuite/locales/pt_BR: undefined () rsuite/locales/ru_RU: undefined () rsuite/locales/sv_SE: undefined () rsuite/locales/tr_TR: undefined () rsuite/locales/zh_CN: undefined () rsuite/locales/zh_TW: undefined () rsuite/toaster: undefined () rsuite/useBreakpointValue: undefined () rsuite/useMediaQuery: undefined () rsuite/useToaster: undefined () sass: ^1.77.6 => 1.77.6 tsx: ^4.7.2 => 4.9.1 typescript: ^5.4.5 => 5.4.5 (4.4.4, 4.9.5) vite: ^5.2.0 => 5.2.11 npmGlobalPackages: @aws-amplify/cli: 12.12.2 corepack: 0.28.2 ncu: 0.2.1 npm-check-updates: 16.14.20 npm: 10.8.1 ```

Describe the bug

When running the command:

npx ampx generate schema-from-database --connection-uri-secret SQL_CONNECTION_STRING --out amplify/data/schema.sql.ts

The auto-generation process does not pick up tables that are in non-public schemas. If the table_schema is not 'public' or belongs to another category, it will not be included by the auto-generation command.

Expected behavior

All tables within the 'public' schema or any other user-defined schemas should be picked up by the auto-generation process. Tables within the pg_catalog or information_schema schemas should be ignored.

Reproduction steps

Having run the step 1 from this guide. Then run step 2 on a postgresql database that has tables under a non public schema.

Thomas-QCD commented 1 month ago

Also having this issue. I have tried changing the connection string all sorts of ways. node_modules/@aws-amplify/schema-generator/lib/generate-schema.js shows this code:

export const parseDatabaseUrl = (databaseUrl) => {
    try {
        const parsedDatabaseUrl = new URL(databaseUrl);
        const { username: encodedUsername, password: encodedPassword, hostname: encodedHost, } = parsedDatabaseUrl;
        const username = decodeURIComponent(encodedUsername);
        const password = decodeURIComponent(encodedPassword);
        const host = decodeURIComponent(encodedHost);
        const database = decodeURIComponent(parsedDatabaseUrl?.pathname?.slice(1));
        // Default engine is MySQL
        const engine = constructDBEngine(parsedDatabaseUrl?.protocol?.slice(0, -1) ?? DEFAULT_ENGINE);
        const port = parsedDatabaseUrl?.port
            ? parseInt(parsedDatabaseUrl?.port, 10)
            : getDefaultPort(engine);
        const config = {
            engine,
            username,
            password,
            database,
            host,
            port,
        };

I would assume the schema should be here but is not included. Edit: I should also note that this works with another database name that has a table under the public schema, but for this case, the public schema does not contain any tables, thus returning the Imported SQL schema is empty. error.

Thomas-QCD commented 1 month ago

I was able to modify my database to fix this issue. This may not be an option in some cases however. CREATE VIEW public.table AS SELECT * FROM otherSchema.table; I was able to get a auto generated schema from amplify after making this change