IHTSDO / snomed-database-loader

Represent SNOMED CT in a different types of databases
Other
108 stars 74 forks source link

Postgresql loader gives syntax errors #53

Closed rjalexa closed 2 years ago

rjalexa commented 2 years ago

Launched sh load_release-postgresql.sh SnomedCT_InternationalRF2_PRODUCTION_20221031T120000Z.zip snomedct FULL

Tables created as desired but then get errors from tmp_loader.sql as follows:

CREATE TABLE
psql:environment-postgresql.sql:273: NOTICE:  table "owlexpressionrefset_f" does not exist, skipping
DROP TABLE
CREATE TABLE
SET
psql:tmp_loader.sql:7: ERROR:  syntax error at or near "-"
LINE 1: -e COPY concept_f
        ^
psql:tmp_loader.sql:11: ERROR:  syntax error at or near "-"
LINE 1: -e 
        ^
psql:tmp_loader.sql:15: ERROR:  syntax error at or near "-"
LINE 1: -e 
        ^
psql:tmp_loader.sql:19: ERROR:  syntax error at or near "-"
LINE 1: -e 
        ^
psql:tmp_loader.sql:23: ERROR:  syntax error at or near "-"
LINE 1: -e 
        ^
psql:tmp_loader.sql:27: ERROR:  syntax error at or near "-"
LINE 1: -e 
        ^
psql:tmp_loader.sql:31: ERROR:  syntax error at or near "-"
LINE 1: -e 
        ^
psql:tmp_loader.sql:35: ERROR:  syntax error at or near "-"
LINE 1: -e 
        ^
psql:tmp_loader.sql:36: ERROR:  syntax error at or near "-"
LINE 1: -e 
        ^

and the content of tmp_loader.sql is:

/* Generated Loader Script */

set schema 'snomedct';

-e COPY concept_f
-e FROM '/Users/bob/Documents/work/@@@ ontologies and misc sources/SNOMED/SNOMEDCT_INT_202210/snomed-database-loader/PostgreSQL/tmp_extracted/sct2_Concept_Full_INT_20221031.txt'
-e WITH (FORMAT csv, HEADER true, DELIMITER E'  ', QUOTE E');
-e 
-e COPY description_f
-e FROM '/Users/bob/Documents/work/@@@ ontologies and misc sources/SNOMED/SNOMEDCT_INT_202210/snomed-database-loader/PostgreSQL/tmp_extracted/sct2_Description_Full-en_INT_20221031.txt'
-e WITH (FORMAT csv, HEADER true, DELIMITER E'  ', QUOTE E');
-e 
-e COPY stated_relationship_f
-e FROM '/Users/bob/Documents/work/@@@ ontologies and misc sources/SNOMED/SNOMEDCT_INT_202210/snomed-database-loader/PostgreSQL/tmp_extracted/sct2_StatedRelationship_Full_INT_20221031.txt'
-e WITH (FORMAT csv, HEADER true, DELIMITER E'  ', QUOTE E');
-e 
-e COPY relationship_f
-e FROM '/Users/bob/Documents/work/@@@ ontologies and misc sources/SNOMED/SNOMEDCT_INT_202210/snomed-database-loader/PostgreSQL/tmp_extracted/sct2_Relationship_Full_INT_20221031.txt'
-e WITH (FORMAT csv, HEADER true, DELIMITER E'  ', QUOTE E');
-e 
-e COPY textdefinition_f
-e FROM '/Users/bob/Documents/work/@@@ ontologies and misc sources/SNOMED/SNOMEDCT_INT_202210/snomed-database-loader/PostgreSQL/tmp_extracted/sct2_TextDefinition_Full-en_INT_20221031.txt'
-e WITH (FORMAT csv, HEADER true, DELIMITER E'  ', QUOTE E');
-e 
-e COPY attributevaluerefset_f
-e FROM '/Users/bob/Documents/work/@@@ ontologies and misc sources/SNOMED/SNOMEDCT_INT_202210/snomed-database-loader/PostgreSQL/tmp_extracted/der2_cRefset_AttributeValueFull_INT_20221031.txt'
-e WITH (FORMAT csv, HEADER true, DELIMITER E'  ', QUOTE E');
-e 
-e COPY langrefset_f
-e FROM '/Users/bob/Documents/work/@@@ ontologies and misc sources/SNOMED/SNOMEDCT_INT_202210/snomed-database-loader/PostgreSQL/tmp_extracted/der2_cRefset_LanguageFull-en_INT_20221031.txt'
-e WITH (FORMAT csv, HEADER true, DELIMITER E'  ', QUOTE E');
-e 
-e COPY associationrefset_f
-e FROM '/Users/bob/Documents/work/@@@ ontologies and misc sources/SNOMED/SNOMEDCT_INT_202210/snomed-database-loader/PostgreSQL/tmp_extracted/der2_cRefset_AssociationFull_INT_20221031.txt'
-e WITH (FORMAT csv, HEADER true, DELIMITER E'  ', QUOTE E');
-e 
siamchen commented 2 years ago

I have no idea where this tmp_loader.sql come from but mate, you need to remove all those -e from the file. Also, it's not right for QUOTE E'. If the single quote has been used as a QUOTE mark, then try to use it as QUOTE E'\''

pgwilliams commented 2 years ago

The -e was coming from the shell script echo commands, which is a flag so that escaped characters are interpreted, so the difference between: echo -e "foo\tbar" which gives foo bar as output and echo "foo\tbar" which gives you that literal string back foo\tbar

I'd be interested to know what OS this was being run on.

rjalexa commented 2 years ago

tmp_loader.sql is generated by the load_release-postgresql.sh script Environment is MacOS 12.6 with zsh Closed since I switched to MySQL out of urgency (and opened a new issue for it regardin transitive closures) Thank you very much for your support

pgwilliams commented 2 years ago

Hmm, maybe I was wrong about the source of the -e problem then, because echo -e behaves as expected when I try it on my Mac under zsh. I'll try it out, but some other time because you said your MySQL issue was more pressing.