search

IHTSDO / snomed-database-loader

Represent SNOMED CT in a different types of databases
Other
107 stars 74 forks source link

Postgres loader does not change the table suffixes in environment-postgresql.sql for different release types #59

Open boxydog opened 1 year ago

boxydog commented 1 year ago

In load_release-postgresql.sh, it asks for loadType (DELTA, SNAP, FULL, ALL), but then the code to adjust environment-postgresql.sql is commented out, so environment-postgresql.sql always has tables ending in _f, which only matches the FULL type.

siamchen commented 1 year ago

Not sure about load_release-postgresq.sh but my way to load the RF2 files is to use the load-postgresql.sql scripts directly. I'm currently only have the access to SNOMED CT-AU data files so not very sure if the INT release file structure is changed or not. If you can send a list of the INT release file structure then I may spend some time on update those outdated scripts.

boxydog commented 1 year ago

In https://github.com/IHTSDO/snomed-database-loader/blob/master/PostgreSQL/README.md, it says to use load_release-postgresql.sh, so I was just doing whatever is documented.

If I should use load-postgresql.sql directly with hand-editing, perhaps change the README? That is what I ended up doing (hand-editing).

I am not using the INT structure, I'm using the US structure. If it helps, here it is:

$ unzip -l SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z.zip 
Archive:  SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
        0  02-20-2023 19:12   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/
     5641  02-20-2023 19:12   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Readme_en_20230301.txt
        0  02-20-2023 19:12   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Snapshot/
        0  02-20-2023 19:12   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Snapshot/Terminology/
 30527744  02-20-2023 19:12   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Snapshot/Terminology/sct2_Concept_Snapshot_US1000124_20230301.txt
214176051  02-20-2023 19:12   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Snapshot/Terminology/sct2_Description_Snapshot-en_US1000124_20230301.txt
       92  02-20-2023 19:12   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Snapshot/Terminology/sct2_Identifier_Snapshot_US1000124_20230301.txt
372965778  02-20-2023 19:12   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Snapshot/Terminology/sct2_Relationship_Snapshot_US1000124_20230301.txt
  3722541  02-20-2023 19:12   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Snapshot/Terminology/sct2_RelationshipConcreteValues_Snapshot_US1000124_20230301.txt
117383978  02-20-2023 19:12   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Snapshot/Terminology/sct2_StatedRelationship_Snapshot_US1000124_20230301.txt
  4169031  02-20-2023 19:12   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Snapshot/Terminology/sct2_TextDefinition_Snapshot-en_US1000124_20230301.txt
 92139770  02-20-2023 19:12   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Snapshot/Terminology/sct2_sRefset_OWLExpressionSnapshot_US1000124_20230301.txt
        0  02-20-2023 19:13   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Snapshot/Refset/
        0  02-20-2023 19:13   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Snapshot/Refset/Content/
 20812400  02-20-2023 19:13   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Snapshot/Refset/Content/der2_cRefset_AssociationSnapshot_US1000124_20230301.txt
 77667027  02-20-2023 19:13   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Snapshot/Refset/Content/der2_cRefset_AttributeValueSnapshot_US1000124_20230301.txt
  1863943  02-20-2023 19:13   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Snapshot/Refset/Content/der2_Refset_SimpleSnapshot_US1000124_20230301.txt
        0  02-20-2023 19:13   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Snapshot/Refset/Map/
128057044  02-20-2023 19:13   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Snapshot/Refset/Map/der2_iisssccRefset_ExtendedMapSnapshot_US1000124_20230301.txt
 54003519  02-20-2023 19:13   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Snapshot/Refset/Map/der2_sRefset_SimpleMapSnapshot_US1000124_20230301.txt
        0  02-20-2023 19:13   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Snapshot/Refset/Language/
363417770  02-20-2023 19:13   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Snapshot/Refset/Language/der2_cRefset_LanguageSnapshot-en_US1000124_20230301.txt
        0  02-20-2023 19:13   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Snapshot/Refset/Metadata/
    27713  02-20-2023 19:13   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Snapshot/Refset/Metadata/der2_cciRefset_RefsetDescriptorSnapshot_US1000124_20230301.txt
      489  02-20-2023 19:13   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Snapshot/Refset/Metadata/der2_ciRefset_DescriptionTypeSnapshot_US1000124_20230301.txt
     1046  02-20-2023 19:13   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Snapshot/Refset/Metadata/der2_ssRefset_ModuleDependencySnapshot_US1000124_20230301.txt
    91212  02-20-2023 19:13   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Snapshot/Refset/Metadata/der2_sssssssRefset_MRCMDomainSnapshot_US1000124_20230301.txt
    22005  02-20-2023 19:13   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Snapshot/Refset/Metadata/der2_cissccRefset_MRCMAttributeDomainSnapshot_US1000124_20230301.txt
    57413  02-20-2023 19:13   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Snapshot/Refset/Metadata/der2_ssccRefset_MRCMAttributeRangeSnapshot_US1000124_20230301.txt
      403  02-20-2023 19:13   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Snapshot/Refset/Metadata/der2_cRefset_MRCMModuleScopeSnapshot_US1000124_20230301.txt
        0  02-20-2023 19:13   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Full/
        0  02-20-2023 19:13   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Full/Terminology/
 40736607  02-20-2023 19:13   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Full/Terminology/sct2_Concept_Full_US1000124_20230301.txt
371488983  02-20-2023 19:13   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Full/Terminology/sct2_Description_Full-en_US1000124_20230301.txt
       92  02-20-2023 19:13   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Full/Terminology/sct2_Identifier_Full_US1000124_20230301.txt
706529597  02-20-2023 19:13   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Full/Terminology/sct2_Relationship_Full_US1000124_20230301.txt
  3802717  02-20-2023 19:14   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Full/Terminology/sct2_RelationshipConcreteValues_Full_US1000124_20230301.txt
236491109  02-20-2023 19:14   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Full/Terminology/sct2_StatedRelationship_Full_US1000124_20230301.txt
  4407323  02-20-2023 19:14   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Full/Terminology/sct2_TextDefinition_Full-en_US1000124_20230301.txt
128133790  02-20-2023 19:14   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Full/Terminology/sct2_sRefset_OWLExpressionFull_US1000124_20230301.txt
        0  02-20-2023 19:14   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Full/Refset/
        0  02-20-2023 19:14   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Full/Refset/Content/
 23956314  02-20-2023 19:14   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Full/Refset/Content/der2_cRefset_AssociationFull_US1000124_20230301.txt
 98983869  02-20-2023 19:14   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Full/Refset/Content/der2_cRefset_AttributeValueFull_US1000124_20230301.txt
  1938081  02-20-2023 19:14   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Full/Refset/Content/der2_Refset_SimpleFull_US1000124_20230301.txt
        0  02-20-2023 19:14   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Full/Refset/Map/
180176287  02-20-2023 19:14   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Full/Refset/Map/der2_iisssccRefset_ExtendedMapFull_US1000124_20230301.txt
 54271305  02-20-2023 19:14   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Full/Refset/Map/der2_sRefset_SimpleMapFull_US1000124_20230301.txt
        0  02-20-2023 19:14   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Full/Refset/Language/
433560717  02-20-2023 19:14   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Full/Refset/Language/der2_cRefset_LanguageFull-en_US1000124_20230301.txt
        0  02-20-2023 19:14   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Full/Refset/Metadata/
    31270  02-20-2023 19:14   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Full/Refset/Metadata/der2_cciRefset_RefsetDescriptorFull_US1000124_20230301.txt
      619  02-20-2023 19:14   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Full/Refset/Metadata/der2_ciRefset_DescriptionTypeFull_US1000124_20230301.txt
    22695  02-20-2023 19:14   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Full/Refset/Metadata/der2_ssRefset_ModuleDependencyFull_US1000124_20230301.txt
   417071  02-20-2023 19:14   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Full/Refset/Metadata/der2_sssssssRefset_MRCMDomainFull_US1000124_20230301.txt
    29161  02-20-2023 19:14   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Full/Refset/Metadata/der2_cissccRefset_MRCMAttributeDomainFull_US1000124_20230301.txt
   106807  02-20-2023 19:14   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Full/Refset/Metadata/der2_ssccRefset_MRCMAttributeRangeFull_US1000124_20230301.txt
      403  02-20-2023 19:14   SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z/Full/Refset/Metadata/der2_cRefset_MRCMModuleScopeFull_US1000124_20230301.txt
---------                     -------
3766197427                     58 files