Open waypointsoftware opened 2 years ago
Please, drop the default value from ID column.
For example:
drop table if exists mytable_02; drop sequence if exists seq_mytable_02;
create sequence seq_mytable_02;
create table mytable_02( id int default nextval('seq_mytable_02'), name varchar );
insert into mytable_02(name) values('teste 1'); insert into mytable_02(name) values('teste 2'); insert into mytable_02(name) values('teste 3'); insert into mytable_02(name) values('teste 4'); insert into mytable_02(name) values('teste 5');
alter table public.mytable_02 alter column id drop default; alter table public.mytable_02 alter column id type uuid using cast(lpad(to_hex(id), 32, '0') as uuid);
select * from mytable_02;
Please, let me know. Clayton
Hello. I am having issues with the FKs. Also tried to remove the default value for id like you said, but didnt work.
This is very odd because the code seems to delete the FKs before trying to convert stuff.
The db state is.
Plan table -> id (integer)
Account table -> plan (integer)
account.plan points to plan.id
I am using a backup database for test and tried to remove the FK, the error proceeds to the next FK column/table.
[ 22:22:08.486186 ] ...changing FK datatype public.accounts.plan => varchar [ 22:22:08.710170 ] Enabling trigger [ 22:22:08.710170 ] ...enable trigger, if exists, on public.accounts Traceback (most recent call last): File "C:\UIID Replace\int_pk2uuid_pk_1\src\replace_id.py", line 122, in execute self._change_fk_column_to_datatype(conn, *args, **kwargs, data_type='varchar') File "C:\UIID Replace\int_pk2uuid_pk_1\src\replace_id.py", line 322, in _change_fk_column_to_datatype utils.execute(connection, sql) File "C:\UIID Replace\int_pk2uuid_pk_1\src\replace_id.py", line 70, in execute cursor.execute(sql_command) psycopg2.errors.DatatypeMismatch: foreign key constraint "FK_accounts_plans_plan" cannot be implemented DETAIL: Key columns "plan" and "id" are of incompatible types: character varying and integer.
Full Log:
[ 22:21:29.866911 ] Disabling trigger [ 22:21:29.866911 ] ...disable trigger, if exists, on public.accounts [ 22:21:30.084837 ] ...disable trigger, if exists, on public.accounts_users [ 22:21:30.301143 ] ...disable trigger, if exists, on public.alteration_history [ 22:21:30.516211 ] ...disable trigger, if exists, on public.applications [ 22:21:30.731982 ] ...disable trigger, if exists, on public.cities [ 22:21:30.948362 ] ...disable trigger, if exists, on public.companies [ 22:21:31.165491 ] ...disable trigger, if exists, on public.companies_modules [ 22:21:31.382637 ] ...disable trigger, if exists, on public.cultivars [ 22:21:31.598163 ] ...disable trigger, if exists, on public.cultures [ 22:21:31.813487 ] ...disable trigger, if exists, on public.farms [ 22:21:32.029865 ] ...disable trigger, if exists, on public.farms_users [ 22:21:32.246215 ] ...disable trigger, if exists, on public.modules [ 22:21:32.462109 ] ...disable trigger, if exists, on public.modules_applications [ 22:21:32.678063 ] ...disable trigger, if exists, on public.permissions [ 22:21:32.894875 ] ...disable trigger, if exists, on public.pesticides [ 22:21:33.110338 ] ...disable trigger, if exists, on public.plans [ 22:21:33.326938 ] ...disable trigger, if exists, on public.plans_applications [ 22:21:33.542955 ] ...disable trigger, if exists, on public.plantings [ 22:21:33.759303 ] ...disable trigger, if exists, on public.plantings_growth_previsions [ 22:21:33.976715 ] ...disable trigger, if exists, on public.plots [ 22:21:34.193810 ] ...disable trigger, if exists, on public.plots_fertilizations [ 22:21:34.413849 ] ...disable trigger, if exists, on public.plots_phytosanitaries_managements [ 22:21:34.629325 ] ...disable trigger, if exists, on public.plots_points [ 22:21:34.848926 ] ...disable trigger, if exists, on public.plots_rains [ 22:21:35.065479 ] ...disable trigger, if exists, on public.plots_soil_analysis [ 22:21:35.284632 ] ...disable trigger, if exists, on public.profiles [ 22:21:35.500380 ] ...disable trigger, if exists, on public.states [ 22:21:35.716118 ] ...disable trigger, if exists, on public.unities [ 22:21:35.931624 ] ...disable trigger, if exists, on public.users [ 22:21:36.148945 ] Creating temporary pk column [ 22:21:36.148945 ] ...adding public.accounts.id_temp2replace [ 22:21:36.365790 ] ...adding public.accounts_users.id_temp2replace [ 22:21:36.582226 ] ...adding public.alteration_history.id_temp2replace [ 22:21:36.797748 ] ...adding public.applications.id_temp2replace [ 22:21:37.012995 ] ...adding public.cities.id_temp2replace [ 22:21:37.229391 ] ...adding public.companies.id_temp2replace [ 22:21:37.445419 ] ...adding public.companies_modules.id_temp2replace [ 22:21:37.661667 ] ...adding public.cultivars.id_temp2replace [ 22:21:37.877911 ] ...adding public.cultures.id_temp2replace [ 22:21:38.094527 ] ...adding public.farms.id_temp2replace [ 22:21:38.310202 ] ...adding public.farms_users.id_temp2replace [ 22:21:38.526039 ] ...adding public.modules.id_temp2replace [ 22:21:38.741593 ] ...adding public.modules_applications.id_temp2replace [ 22:21:38.957694 ] ...adding public.permissions.id_temp2replace [ 22:21:39.174787 ] ...adding public.pesticides.id_temp2replace [ 22:21:39.390782 ] ...adding public.plans.id_temp2replace [ 22:21:39.606535 ] ...adding public.plans_applications.id_temp2replace [ 22:21:39.822454 ] ...adding public.plantings.id_temp2replace [ 22:21:40.037522 ] ...adding public.plantings_growth_previsions.id_temp2replace [ 22:21:40.253880 ] ...adding public.plots.id_temp2replace [ 22:21:40.469386 ] ...adding public.plots_fertilizations.id_temp2replace [ 22:21:40.685069 ] ...adding public.plots_phytosanitaries_managements.id_temp2replace [ 22:21:40.901903 ] ...adding public.plots_points.id_temp2replace [ 22:21:41.118060 ] ...adding public.plots_rains.id_temp2replace [ 22:21:41.333417 ] ...adding public.plots_soil_analysis.id_temp2replace [ 22:21:41.549284 ] ...adding public.profiles.id_temp2replace [ 22:21:41.765680 ] ...adding public.states.id_temp2replace [ 22:21:41.981648 ] ...adding public.unities.id_temp2replace [ 22:21:42.197703 ] ...adding public.users.id_temp2replace [ 22:21:42.414420 ] Assign values to temporary pk column [ 22:21:42.414420 ] ...assiging public.accounts.id_temp2replace [ 22:21:42.638205 ] ...assiging public.accounts_users.id_temp2replace [ 22:21:42.857072 ] ...assiging public.alteration_history.id_temp2replace [ 22:21:43.073777 ] ...assiging public.applications.id_temp2replace [ 22:21:43.292200 ] ...assiging public.cities.id_temp2replace [ 22:21:43.527853 ] ...assiging public.companies.id_temp2replace [ 22:21:43.745482 ] ...assiging public.companies_modules.id_temp2replace [ 22:21:43.965065 ] ...assiging public.cultivars.id_temp2replace [ 22:21:44.184265 ] ...assiging public.cultures.id_temp2replace [ 22:21:44.403206 ] ...assiging public.farms.id_temp2replace [ 22:21:44.623104 ] ...assiging public.farms_users.id_temp2replace [ 22:21:44.841796 ] ...assiging public.modules.id_temp2replace [ 22:21:45.059066 ] ...assiging public.modules_applications.id_temp2replace [ 22:21:45.279492 ] ...assiging public.permissions.id_temp2replace [ 22:21:45.503503 ] ...assiging public.pesticides.id_temp2replace [ 22:21:45.720284 ] ...assiging public.plans.id_temp2replace [ 22:21:45.938299 ] ...assiging public.plans_applications.id_temp2replace [ 22:21:46.157666 ] ...assiging public.plantings.id_temp2replace [ 22:21:46.376600 ] ...assiging public.plantings_growth_previsions.id_temp2replace [ 22:21:46.595064 ] ...assiging public.plots.id_temp2replace [ 22:21:46.813759 ] ...assiging public.plots_fertilizations.id_temp2replace [ 22:21:47.031822 ] ...assiging public.plots_phytosanitaries_managements.id_temp2replace [ 22:21:47.250593 ] ...assiging public.plots_points.id_temp2replace [ 22:21:47.469164 ] ...assiging public.plots_rains.id_temp2replace [ 22:21:47.687144 ] ...assiging public.plots_soil_analysis.id_temp2replace [ 22:21:47.905612 ] ...assiging public.profiles.id_temp2replace [ 22:21:48.123155 ] ...assiging public.states.id_temp2replace [ 22:21:48.340818 ] ...assiging public.unities.id_temp2replace [ 22:21:48.558207 ] ...assiging public.users.id_temp2replace [ 22:21:48.777199 ] Creating serial column [ 22:21:48.777199 ] ...adding serial column public.accounts [ 22:21:48.993832 ] ...adding serial column public.accounts_users [ 22:21:49.210622 ] ...adding serial column public.alteration_history [ 22:21:49.428970 ] ...adding serial column public.applications [ 22:21:49.644927 ] ...adding serial column public.cities [ 22:21:49.861758 ] ...adding serial column public.companies [ 22:21:50.077623 ] ...adding serial column public.companies_modules [ 22:21:50.293372 ] ...adding serial column public.cultivars [ 22:21:50.509485 ] ...adding serial column public.cultures [ 22:21:50.726478 ] ...adding serial column public.farms [ 22:21:50.941930 ] ...adding serial column public.farms_users [ 22:21:51.158400 ] ...adding serial column public.modules [ 22:21:51.374878 ] ...adding serial column public.modules_applications [ 22:21:51.591434 ] ...adding serial column public.permissions [ 22:21:51.806806 ] ...adding serial column public.pesticides [ 22:21:52.023527 ] ...adding serial column public.plans [ 22:21:52.238893 ] ...adding serial column public.plans_applications [ 22:21:52.454997 ] ...adding serial column public.plantings [ 22:21:52.670962 ] ...adding serial column public.plantings_growth_previsions [ 22:21:52.887429 ] ...adding serial column public.plots [ 22:21:53.104566 ] ...adding serial column public.plots_fertilizations [ 22:21:53.320243 ] ...adding serial column public.plots_phytosanitaries_managements [ 22:21:53.536346 ] ...adding serial column public.plots_points [ 22:21:53.752250 ] ...adding serial column public.plots_rains [ 22:21:53.969026 ] ...adding serial column public.plots_soil_analysis [ 22:21:54.185443 ] ...adding serial column public.profiles [ 22:21:54.401417 ] ...adding serial column public.states [ 22:21:54.616649 ] ...adding serial column public.unities [ 22:21:54.833036 ] ...adding serial column public.users [ 22:21:55.048915 ] Copying pk to serial column [ 22:21:55.048915 ] ...copying pk to serial column public.accounts [ 22:21:55.266419 ] ...copying pk to serial column public.accounts_users [ 22:21:55.488794 ] ...copying pk to serial column public.alteration_history [ 22:21:55.704583 ] ...copying pk to serial column public.applications [ 22:21:55.922628 ] ...copying pk to serial column public.cities [ 22:21:56.152407 ] ...copying pk to serial column public.companies [ 22:21:56.368478 ] ...copying pk to serial column public.companies_modules [ 22:21:56.584998 ] ...copying pk to serial column public.cultivars [ 22:21:56.800993 ] ...copying pk to serial column public.cultures [ 22:21:57.017145 ] ...copying pk to serial column public.farms [ 22:21:57.233216 ] ...copying pk to serial column public.farms_users [ 22:21:57.449333 ] ...copying pk to serial column public.modules [ 22:21:57.667301 ] ...copying pk to serial column public.modules_applications [ 22:21:57.884582 ] ...copying pk to serial column public.permissions [ 22:21:58.103976 ] ...copying pk to serial column public.pesticides [ 22:21:58.319716 ] ...copying pk to serial column public.plans [ 22:21:58.535579 ] ...copying pk to serial column public.plans_applications [ 22:21:58.752186 ] ...copying pk to serial column public.plantings [ 22:21:58.967572 ] ...copying pk to serial column public.plantings_growth_previsions [ 22:21:59.183508 ] ...copying pk to serial column public.plots [ 22:21:59.400148 ] ...copying pk to serial column public.plots_fertilizations [ 22:21:59.615875 ] ...copying pk to serial column public.plots_phytosanitaries_managements [ 22:21:59.831745 ] ...copying pk to serial column public.plots_points [ 22:22:00.047372 ] ...copying pk to serial column public.plots_rains [ 22:22:00.263905 ] ...copying pk to serial column public.plots_soil_analysis [ 22:22:00.479951 ] ...copying pk to serial column public.profiles [ 22:22:00.696555 ] ...copying pk to serial column public.states [ 22:22:00.912355 ] ...copying pk to serial column public.unities [ 22:22:01.130013 ] ...copying pk to serial column public.users [ 22:22:01.346218 ] Dropping fk constraints [ 22:22:01.346218 ] ...dropping FK constraint public.accounts FK_accounts_plans_plan [ 22:22:01.562207 ] ...dropping FK constraint public.accounts_users FK_accounts_users_accounts_account [ 22:22:01.779991 ] ...dropping FK constraint public.accounts_users FK_accounts_users_users_user [ 22:22:01.999037 ] ...dropping FK constraint public.cities FK_cities_states_state [ 22:22:02.213840 ] ...dropping FK constraint public.companies_modules FK_companies_modules_companies_company [ 22:22:02.429586 ] ...dropping FK constraint public.companies_modules FK_companies_modules_modules_module [ 22:22:02.652521 ] ...dropping FK constraint public.cultivars FK_cultivars_cultures_culture_id [ 22:22:02.868725 ] ...dropping FK constraint public.farms FK_farms_accounts_account [ 22:22:03.083563 ] ...dropping FK constraint public.farms FK_farms_cities_city [ 22:22:03.299264 ] ...dropping FK constraint public.farms FK_farms_states_state [ 22:22:03.515413 ] ...dropping FK constraint public.farms_users FK_farms_users_farms_farm [ 22:22:03.731213 ] ...dropping FK constraint public.farms_users FK_farms_users_users_user [ 22:22:03.946992 ] ...dropping FK constraint public.modules_applications FK_modules_applications_applications_application [ 22:22:04.163246 ] ...dropping FK constraint public.modules_applications FK_modules_applications_modules_module [ 22:22:04.378245 ] ...dropping FK constraint public.permissions FK_permissions_applications_application [ 22:22:04.594920 ] ...dropping FK constraint public.permissions FK_permissions_profiles_profile [ 22:22:04.809958 ] ...dropping FK constraint public.plans FK_plans_companies_company [ 22:22:05.031634 ] ...dropping FK constraint public.plans_applications FK_plans_applications_applications_application [ 22:22:05.247322 ] ...dropping FK constraint public.plans_applications FK_plans_applications_plans_plan [ 22:22:05.462650 ] ...dropping FK constraint public.plantings FK_plantings_cultivars_cultivar [ 22:22:05.678380 ] ...dropping FK constraint public.plantings FK_plantings_plots_plot [ 22:22:05.894433 ] ...dropping FK constraint public.plantings_growth_previsions FK_plantings_growth_previsions_plantings_planting [ 22:22:06.109981 ] ...dropping FK constraint public.plantings_growth_previsions FK_plantings_growth_previsions_plots_plot [ 22:22:06.325528 ] ...dropping FK constraint public.plots FK_plots_farms_farm [ 22:22:06.541257 ] ...dropping FK constraint public.plots_fertilizations FK_plots_fertilizations_plots_plot [ 22:22:06.756350 ] ...dropping FK constraint public.plots_phytosanitaries_managements FK_plots_phytosanitaries_managements_pesticides_pesticide [ 22:22:06.972589 ] ...dropping FK constraint public.plots_phytosanitaries_managements FK_plots_phytosanitaries_managements_plots_plot [ 22:22:07.187347 ] ...dropping FK constraint public.plots_phytosanitaries_managements FK_plots_phytosanitaries_managements_unities_unity [ 22:22:07.402832 ] ...dropping FK constraint public.plots_points FK_plots_points_plots_plot [ 22:22:07.618866 ] ...dropping FK constraint public.plots_rains FK_plots_rains_plots_plot [ 22:22:07.834348 ] ...dropping FK constraint public.plots_soil_analysis FK_plots_soil_analysis_plots_plot [ 22:22:08.051858 ] ...dropping FK constraint public.users FK_users_companies_company [ 22:22:08.269574 ] ...dropping FK constraint public.users FK_users_profiles_profile [ 22:22:08.485186 ] Changing fk to varchar [ 22:22:08.486186 ] ...changing FK datatype public.accounts.plan => varchar [ 22:22:08.710170 ] Enabling trigger [ 22:22:08.710170 ] ...enable trigger, if exists, on public.accounts Traceback (most recent call last): File "C:\UIID Replace\int_pk2uuid_pk_1\src\replace_id.py", line 122, in execute self._change_fk_column_to_datatype(conn, *args, **kwargs, data_type='varchar') File "C:\UIID Replace\int_pk2uuid_pk_1\src\replace_id.py", line 322, in _change_fk_column_to_datatype utils.execute(connection, sql) File "C:\UIID Replace\int_pk2uuid_pk_1\src\replace_id.py", line 70, in execute cursor.execute(sql_command) psycopg2.errors.DatatypeMismatch: foreign key constraint "FK_accounts_plans_plan" cannot be implemented DETAIL: Key columns "plan" and "id" are of incompatible types: character varying and integer.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\UIID Replace\int_pk2uuid_pk_1\src\main.py", line 12, in
I will try to reproduce the error.
Please, send to me the complete "create table" commands for:
Thank you, Clayton
Accounts points directly to Plans. (column plan)
CREATE TABLE public.plans ( id integer NOT NULL DEFAULT nextval('plans_id_seq'::regclass), name text COLLATE pg_catalog."default" NOT NULL, company integer NOT NULL, "default" boolean NOT NULL, updated_at timestamp without time zone, updated_by integer, deleted_at timestamp without time zone, deleted_by integer, created_at timestamp without time zone, created_by integer, CONSTRAINT "PK_plans" PRIMARY KEY (id), CONSTRAINT "FK_plans_companies_company" FOREIGN KEY (company) REFERENCES public.companies (id) MATCH SIMPLE ON UPDATE NO ACTION ON DELETE CASCADE ) CREATE INDEX "IX_plans_company" ON public.plans USING btree (company) TABLESPACE pg_default;
CREATE TABLE public.accounts ( id integer NOT NULL DEFAULT nextval('accounts_id_seq'::regclass), name text COLLATE pg_catalog."default" NOT NULL, company integer NOT NULL, cpf_cnpj text COLLATE pg_catalog."default" NOT NULL, phone text COLLATE pg_catalog."default" NOT NULL, email text COLLATE pg_catalog."default" NOT NULL, active boolean NOT NULL, updated_at timestamp without time zone, updated_by integer, deleted_at timestamp without time zone, deleted_by integer, created_at timestamp without time zone, created_by integer, plan integer, CONSTRAINT "PK_accounts" PRIMARY KEY (id), CONSTRAINT "FK_accounts_companies_company" FOREIGN KEY (company) REFERENCES public.companies (id) MATCH SIMPLE ON UPDATE NO ACTION ON DELETE CASCADE, CONSTRAINT "FK_accounts_plans_plan" FOREIGN KEY (plan) REFERENCES public.plans (id) MATCH SIMPLE ON UPDATE NO ACTION ON DELETE RESTRICT )
CREATE INDEX "IX_accounts_company" ON public.accounts USING btree (company) TABLESPACE pg_default;
Please update (git pull) the project once I made a small change to it
I created the following database to test:
create schema clayton;
set search_path=clayton;
create sequence plans_id_seq;
create table plans(
id integer NOT NULL DEFAULT nextval('plans_id_seq'::regclass),
name text COLLATE pg_catalog."default" NOT NULL,
company integer NOT NULL,
"default" boolean NOT NULL,
updated_at timestamp without time zone,
updated_by integer,
deleted_at timestamp without time zone,
deleted_by integer,
created_at timestamp without time zone,
created_by integer,
CONSTRAINT "PK_plans" PRIMARY KEY (id)
);
CREATE INDEX "IX_plans_company" ON plans USING btree(company) TABLESPACE pg_default;
create sequence accounts_id_seq;
create table accounts(
id integer NOT NULL DEFAULT nextval('accounts_id_seq'::regclass),
name text COLLATE pg_catalog."default" NOT NULL,
company integer NOT NULL,
cpf_cnpj text COLLATE pg_catalog."default" NOT NULL,
phone text COLLATE pg_catalog."default" NOT NULL,
email text COLLATE pg_catalog."default" NOT NULL,
active boolean NOT NULL,
updated_at timestamp without time zone,
updated_by integer,
deleted_at timestamp without time zone,
deleted_by integer,
created_at timestamp without time zone,
created_by integer,
plan integer,
CONSTRAINT "PK_accounts" PRIMARY KEY (id),
CONSTRAINT "FK_accounts_plans_plan" FOREIGN KEY (plan)
REFERENCES plans (id) MATCH SIMPLE ON UPDATE NO ACTION ON DELETE RESTRICT
);
CREATE INDEX "IX_accounts_company" ON accounts USING btree(company) TABLESPACE pg_default;
insert into plans(name, company, "default") values('test 1', 1, true);
insert into plans(name, company, "default") values('test 2', 2, true);
insert into plans(name, company, "default") values('test 3', 3, true);
insert into accounts(name, company, cpf_cnpj, phone, email, active, plan) values('test 1', 0, '', '', '', true, 1);
insert into accounts(name, company, cpf_cnpj, phone, email, active, plan) values('test 2', 0, '', '', '', true, 1);
insert into accounts(name, company, cpf_cnpj, phone, email, active, plan) values('test 3', 0, '', '', '', true, 2);
insert into accounts(name, company, cpf_cnpj, phone, email, active, plan) values('test 4', 0, '', '', '', true, 3);
insert into accounts(name, company, cpf_cnpj, phone, email, active, plan) values('test 5', 0, '', '', '', true, 3);
insert into accounts(name, company, cpf_cnpj, phone, email, active, plan) values('test 6', 0, '', '', '', true, 3);
I used the following Python code to test the database:
from replace_id import IdReplacer
IdReplacer().execute(
params={
'host': 'localhost',
'port': '5432',
'user': 'postgres',
'password': 'postgres',
'schema': 'clayton',
'db_name': 'postgres',
'serial_name': 'serial_id',
'autocommit': True,
}
)
Hey man! After i made the pull i was still getting the error, what i found is that the FKs are not deleting. I Removed the 'drop if exists' from the sql of the constraints , also made prints for all sql generated in the code. So the real error was that the FK doesnt exists, i fixed puting "" between the FK names.
After that i started to get an error on a column called user, because user is a postgresql reserved name, so in every sql that treated columns i put "{column_name}".
Also, i was creating the database trough my migrations, and every time i have to create the extension, so i put another step in the code to create the pgcrypto extension before runs everything.
And for last i have one table that has a relation with another table but is not by the primary key. In this case i have to delete the FK for the table and then reacriating manually.
Some sugestions. (parameters)
Here is the code if you wanna make some refactoring and made a push, i will not open a MR because i dont think i make a good work because i'm not a python developer.
*PS: Dont forget to add in the readme that we need to provide the 'port', that was the first error that i get i see that you insert this new parameter.
https://pastebin.com/3ZRSv63A - Password: cDMxFBNABc
Thanks for the help man!
Hi, Thanks for your remarks regarding the code. I took your ideas and changed some points in the code:
If it were possible, could you test these changes?
Thanks again for your help.
The part of the code in this function: change_column_to_uuid(conn, *args, **kwargs) doesnt work and generates an error that says : DatatypeMismatch: default for column "id" cannot be cast automatically to type uuid
That is based off of this sql command it generated: alter table public.chats alter column id type uuid using cast(lpad(to_hex(id), 32, '0') as uuid);
I get this on every table I have tried so far though.