`atlas schema diff` with enum inside a schema does not emit a statement to create said enum

[swlynch99]

I have a database schema that has an enum and some tables within a postgresql schema. When I run atlas schema diff or atlas schema apply it fails to emit the CREATE TYPE enum definition. Everything works fine if the enum is in the default schema, this only happens if the enum is part of another schema.

Steps to reproduce:

First, create repro.sql with the contents below

CREATE SCHEMA test;
CREATE TYPE test.state AS ENUM ('a', 'b', 'c' );
CREATE TABLE test.the_table (
    id int4 NOT NULL,
    value test.state NOT NULL
);

Then run the following

touch empty.sql
atlas schema diff --dev-url docker://postgres/15/test --from file://empty.sql --to file://repro.sql --format '{{ sql . "  " }}'

This will print out

-- Add new schema named "test"
CREATE SCHEMA "test";
-- Create "the_table" table
CREATE TABLE "test"."the_table" (
  "id" integer NOT NULL,
  "value" "test"."state" NOT NULL
);

... which does not have the definition for the test.state enum even though test.the_table refers to it.

Extra notes:

• atlas schema inspect does emit HCL that includes the test.state enum.

[DreamwareDevelopment]

Also encountered enums not being created.

schema "stocks" {}

enum "exchange" {
  schema = schema.stocks
  values = ["AMEX", "ARCA", "BATS", "CRYPTO", "NASDAQ", "NYSE", "NYSEARCA", "OTC"]
}

enum "status" {
  schema = schema.stocks
  values = ["active", "inactive"]
}

table "assets" {
  schema = schema.stocks
  column "id" {
    null = false
    type = uuid
  }
  column "symbol" {
    null = false
    type = text
  }
  column "exchange" {
    null = false
    type = enum.exchange
  }
  column "name" {
    null = false
    type = text
  }
  column "status" {
    null = false
    type = enum.status
  }
  primary_key  {
    columns = [column.id]
  }
  ...
}

And here's the output from the operator:

2023-10-19T03:23:33Z    DEBUG   events  atlasexec: create "assets" table: pq: type "stocks.exchange" does not exist, {"Driver":"postgres","URL":{"Scheme":"postgres","Opaque":"","User":{},"Host":"postgresql.default:3300","Path":"/stocks","RawPath":"","OmitHost":false,"ForceQuery":false,"RawQuery":"sslmode=disable","Fragment":"","RawFragment":"","Schema":""},"Changes":{"Applied":["CREATE SCHEMA \"stocks\""],"Pending":["CREATE TABLE \"stocks\".\"assets\" (\"id\" uuid NOT NULL, \"symbol\" text NOT NULL, \"exchange\" \"stocks\".\"exchange\" NOT NULL, \"name\" text NOT NULL, \"status\" \"stocks\".\"status\" NOT NULL, \"tradeable\" boolean NOT NULL, \"marginable\" boolean NOT NULL, \"shortable\" boolean NOT NULL, \"easy_to_borrow\" boolean NOT NULL, \"fractionable\" boolean NOT NULL, \"maintenance_margin_requirement\" integer NOT NULL, PRIMARY KEY (\"id\"))","COMMENT ON TABLE \"stocks\".\"assets\" IS 'List of all requested traded assets'","CREATE TABLE \"stocks\".\"active_ticker_requests\" (\"from\" timestamptz NOT NULL, \"asset_id\" uuid NOT NULL, \"user_id\" bigint NOT NULL, PRIMARY KEY (\"asset_id\", \"user_id\"), CONSTRAINT \"asset_id\" FOREIGN KEY (\"asset_id\") REFERENCES \"stocks\".\"assets\" (\"id\") ON UPDATE CASCADE ON DELETE CASCADE)","CREATE INDEX \"asset_index\" ON \"stocks\".\"active_ticker_requests\" (\"asset_id\", \"from\")","COMMENT ON TABLE \"stocks\".\"active_ticker_requests\" IS '*:* users-\u003eassets relationship table of requests, for restarting streams when recovering from query service failures'","COMMENT ON COLUMN \"stocks\".\"active_ticker_requests\" .\"user_id\" IS 'Unfortunately we can''t have a foreign key reference between databases.'","CREATE TABLE \"stocks\".\"stock_prices\" (\"time\" timestamptz NOT NULL, \"price\" money NOT NULL, \"asset_id\" uuid NOT NULL, CONSTRAINT \"asset_id\" FOREIGN KEY (\"asset_id\") REFERENCES \"stocks\".\"assets\" (\"id\") ON UPDATE CASCADE ON DELETE CASCADE) PARTITION BY RANGE (\"time\")","CREATE UNIQUE INDEX \"price_index\" ON \"stocks\".\"stock_prices\" (\"asset_id\", \"time\") INCLUDE (\"price\")","COMMENT ON TABLE \"stocks\".\"stock_prices\" IS 'Stock prices, partitioned by time, ingested using ticker_requests'","CREATE TABLE \"stocks\".\"ticker_tracker\" (\"minimum_recorded_at\" timestamptz NOT NULL, \"maximum_recorded_at\" timestamptz NOT NULL, \"asset_id\" uuid NOT NULL, CONSTRAINT \"asset_id\" FOREIGN KEY (\"asset_id\") REFERENCES \"stocks\".\"assets\" (\"id\") ON UPDATE CASCADE ON DELETE CASCADE)","CREATE UNIQUE INDEX \"unique_constraint_index\" ON \"stocks\".\"ticker_tracker\" (\"asset_id\")","COMMENT ON TABLE \"stocks\".\"ticker_tracker\" IS 'List of all tracked tickers, with minimum recorded_at time, overwritten by ticker_requests'"],"Error":{"Stmt":"CREATE TABLE \"stocks\".\"assets\" (\"id\" uuid NOT NULL, \"symbol\" text NOT NULL, \"exchange\" \"stocks\".\"exchange\" NOT NULL, \"name\" text NOT NULL, \"status\" \"stocks\".\"status\" NOT NULL, \"tradeable\" boolean NOT NULL, \"marginable\" boolean NOT NULL, \"shortable\" boolean NOT NULL, \"easy_to_borrow\" boolean NOT NULL, \"fractionable\" boolean NOT NULL, \"maintenance_margin_requirement\" integer NOT NULL, PRIMARY KEY (\"id\"))","Text":"create \"assets\" table: pq: type \"stocks.exchange\" does not exist"}}}   {"type": "Warning", "object": {"kind":"AtlasSchema","namespace":"default","name":"stocks-schema","uid":"0221429e-555b-4f96-a686-093075704a52","apiVersion":"db.atlasgo.io/v1alpha1","resourceVersion":"715"}, "reason": "TransientErr"}

[jerluc]

I'd be happy to take a look at this if someone could point me to where in the code TYPE generation might be occurring.

In the meantime, I'll try to turn one of these examples into a failing integration test in TestPostgres_Enums, which looks like a reasonable place to start.