TonicAI / condenser

Condenser is a database subsetting tool
https://www.tonic.ai
MIT License
312 stars 48 forks source link

Error on UNIQUE CONSTRAINT creation #5

Closed gayan415 closed 5 years ago

gayan415 commented 5 years ago

Hi,

My source DB has a table called employer, here is the schema.

CREATE TABLE "employer" (
    "id" BIGSERIAL PRIMARY KEY,
    "organizationId" BIGINT REFERENCES "organization" ("id") ON DELETE CASCADE NOT NULL,
    "networkId" BIGINT REFERENCES "network" ("id") ON DELETE SET NULL UNIQUE,
    "name" TEXT NOT NULL,
    "externalId" TEXT,
    "address" JSONB DEFAULT('{}') NOT NULL,
    "contact" JSONB DEFAULT('{}') NOT NULL,
    "canonicalEmployerId" BIGINT REFERENCES "employer" ("id") ON DELETE SET NULL,
    "created" TIMESTAMP WITH TIME ZONE DEFAULT(NOW()) NOT NULL,
    "modified" TIMESTAMP WITH TIME ZONE,
    UNIQUE ("organizationId", "externalId")
);
CREATE INDEX "employer_networkId_idx" ON "employer" ("networkId");
CREATE INDEX "employer_name_idx" ON "employer" ("name");
CREATE INDEX "employer_sortByVerifiedThenName_idx" ON "employer" ((COALESCE(("employer"."networkId" < 0)::INT, 1) || '_' || "employer"."name"));
CREATE INDEX "employer_name_trgm_idx" ON "employer" USING gin("name" gin_trgm_ops);
CREATE INDEX "employer_address_idx" ON "employer" USING gin("address");
COMMENT ON COLUMN "employer"."contact" IS 'Employer work email, phone number, or any contact-related data';
COMMENT ON COLUMN "employer"."externalId" IS 'ID used for external application to identify a BLN employer record';
COMMENT ON COLUMN "employer"."canonicalEmployerId" IS 'Canonical employer ID of the employer';
COMMENT ON CONSTRAINT "employer_organizationId_externalId_key" ON "employer" IS 'Combination of "organizaionId" and "externalId" has to be unique';

I'm getting an error like below

 File "main.py", line 30, in func_base
    database.validate_database_create()
  File "/xxx/xxx/xxx/condenser/database_creator.py", line 82, in validate_database_create
    raise Exception(f'Creating tables failed.  See {self.create_error_path} for details')
Exception: Creating tables failed.  See /xxx/xxx/xxx/condenser/SQL/create_error.txt for details

Then I checked SQL/create_error.txt and it has

constraint "employer_organizationId_externalId_key" for table "employer" does not exist

I have checked generated dump_create.sql file and I can not see UNIQUE ("organizationId", "externalId") . Here is the snippet from dump_create.sql .

CREATE TABLE public.employer (
    id bigint NOT NULL,
    "externalId" text,
    address jsonb DEFAULT '{}'::jsonb NOT NULL,
    created timestamp with time zone DEFAULT now() NOT NULL,
    modified timestamp with time zone,
    "networkId" bigint,
    contact jsonb DEFAULT '{}'::jsonb NOT NULL,
    "organizationId" bigint NOT NULL,
    name text NOT NULL,
    "canonicalEmployerId" bigint
);
COMMENT ON COLUMN public.employer."externalId" IS 'ID used for external application to identify a BLN employer record';
COMMENT ON COLUMN public.employer.contact IS 'Employer work email, phone number, or any contact-related data';
COMMENT ON COLUMN public.employer."canonicalEmployerId" IS 'Canonical employer ID of the employer';
CREATE SEQUENCE public.employer_id_seq
    START WITH 1
    INCREMENT BY 1
    NO MINVALUE
    NO MAXVALUE
    CACHE 1;
ALTER SEQUENCE public.employer_id_seq OWNED BY public.employer.id;

I tried to fix it by changing config.json and never success, What kind of setting am I missing?

Thanks Gayan

akamor commented 5 years ago

Hi, thanks for trying out Condenser and reporting this bug. We found the issue and have fixed it. If you pull the latest master your issue should be resolved. In short, we weren't properly handling 'COMMENT ON CONSTRAINT' commands.

On a side note, your one of the first people to give Condenser a try. Would you mind reaching out to me at adam@tonic.ai? Would love to chat a bit about what you are working on and to hear more of your thoughts on Condenser.