callahantiff / PheKnowLator

PheKnowLator: Heterogeneous Biomedical Knowledge Graphs and Benchmarks Constructed Under Alternative Semantic Models
https://github.com/callahantiff/PheKnowLator/wiki
Apache License 2.0
157 stars 29 forks source link

Adding Better File Index to GCS Bucket #111

Closed callahantiff closed 2 years ago

callahantiff commented 2 years ago

Purpose

This PR extends the current build code to create a more comprehensive index of the files used to create all builds. This new information can be previewed below and is written to a document called full_pheknowlator_build_files.json.

{
    "metadata": "For more information on the PheKnowLator Builds, please visit the project GitHub: https://github.com/callahantiff/PheKnowLator. Additional information on the file types can be found on the wiki, here: https://github.com/callahantiff/PheKnowLator/wiki/KG-Construction#table-knowledge-graph-build-output",
    "v2.0.0-2020-5-10": {
        "build_logs": {
            "data": "None",
            "knowledge_graphs": "None"
        },
        "data": {
            "original_data": [
                "https://storage.googleapis.com/pheknowlator/archived_builds/release_v2.0.0/build_10MAY2020/data/original_data/ChEBI2Reactome_All_Levels.txt",
                "https://storage.googleapis.com/pheknowlator/archived_builds/release_v2.0.0/build_10MAY2020/data/original_data/GTEx_Analysis_2017-06-05_v8_RNASeQCv1.1.9_gene_median_tpm.gct",
                "https://storage.googleapis.com/pheknowlator/archived_builds/release_v2.0.0/build_10MAY2020/data/original_data/HPA_tissues.txt",
                "https://storage.googleapis.com/pheknowlator/archived_builds/release_v2.0.0/build_10MAY2020/data/original_data/Homo_sapiens.GRCh38.99.entrez.tsv",
                "https://storage.googleapis.com/pheknowlator/archived_builds/release_v2.0.0/build_10MAY2020/data/original_data/Homo_sapiens.GRCh38.99.gtf", ...
            ],
            "processed_data": [
                "https://storage.googleapis.com/pheknowlator/archived_builds/release_v2.0.0/build_10MAY2020/data/processed_data/INVERSE_RELATIONS.txt",
                "https://storage.googleapis.com/pheknowlator/archived_builds/release_v2.0.0/build_10MAY2020/data/processed_data/OWL_NETS_Property_Types.txt",
                "https://storage.googleapis.com/pheknowlator/archived_builds/release_v2.0.0/build_10MAY2020/data/processed_data/PheKnowLator_MergedOntologiesGeneID_Normalized_Cleaned.owl",
                "https://storage.googleapis.com/pheknowlator/archived_builds/release_v2.0.0/build_10MAY2020/data/processed_data/RELATIONS_LABELS.txt",
                "https://storage.googleapis.com/pheknowlator/archived_builds/release_v2.0.0/build_10MAY2020/data/processed_data/chebi_lite_with_imports.owl", ...
            ]
        },
        "knowledge_graphs": {
            "instance_builds": [
                "https://storage.googleapis.com/pheknowlator/archived_builds/release_v2.0.0/build_10MAY2020/knowledge_graphs/instance_builds/inverse_relations/owl/Master_Edge_List_Dict.json",
                "https://storage.googleapis.com/pheknowlator/archived_builds/release_v2.0.0/build_10MAY2020/knowledge_graphs/instance_builds/inverse_relations/owl/PheKnowLator_v2.0.0_full_Instance_inverseRelations_OWL.owl",
                "https://storage.googleapis.com/pheknowlator/archived_builds/release_v2.0.0/build_10MAY2020/knowledge_graphs/instance_builds/inverse_relations/owl/PheKnowLator_v2.0.0_full_Instance_inverseRelations_OWL_NetworkxMultiDiGraph.gpickle",
                "https://storage.googleapis.com/pheknowlator/archived_builds/release_v2.0.0/build_10MAY2020/knowledge_graphs/instance_builds/inverse_relations/owl/PheKnowLator_v2.0.0_full_Instance_inverseRelations_OWL_NodeLabels.txt",
                "https://storage.googleapis.com/pheknowlator/archived_builds/release_v2.0.0/build_10MAY2020/knowledge_graphs/instance_builds/inverse_relations/owl/PheKnowLator_v2.0.0_full_Instance_inverseRelations_OWL_Stats.txt",
                "https://storage.googleapis.com/pheknowlator/archived_builds/release_v2.0.0/build_10MAY2020/knowledge_graphs/instance_builds/inverse_relations/owl/PheKnowLator_v2.0.0_full_Instance_inverseRelations_OWL_Stats_Terminal_Output.txt", ...
            ],
            "other": [
                "https://storage.googleapis.com/pheknowlator/archived_builds/release_v2.0.0/build_10MAY2020/knowledge_graphs/PheKnowLator_Master_Edge_List_Dict.json",
                "https://storage.googleapis.com/pheknowlator/archived_builds/release_v2.0.0/build_10MAY2020/knowledge_graphs/PheKnowLator_Master_Node_Edge_List_Dict.json",
                "https://storage.googleapis.com/pheknowlator/archived_builds/release_v2.0.0/build_10MAY2020/knowledge_graphs/PheKnowLator_MergedOntologiesGeneID_Normalized_Cleaned.owl", ...
            ],
            "subclass_builds": [
              "https://storage.googleapis.com/pheknowlator/archived_builds/release_v2.0.0/build_10MAY2020/knowledge_graphs/subclass_builds/inverse_relations/owl/Master_Edge_List_Dict.json",
                "https://storage.googleapis.com/pheknowlator/archived_builds/release_v2.0.0/build_10MAY2020/knowledge_graphs/subclass_builds/inverse_relations/owl/PheKnowLator_v2.0.0_full_subclass_inverseRelations_OWL.owl",
                "https://storage.googleapis.com/pheknowlator/archived_builds/release_v2.0.0/build_10MAY2020/knowledge_graphs/subclass_builds/inverse_relations/owl/PheKnowLator_v2.0.0_full_subclass_inverseRelations_OWL_NetworkxMultiDiGraph.gpickle",
                "https://storage.googleapis.com/pheknowlator/archived_builds/release_v2.0.0/build_10MAY2020/knowledge_graphs/subclass_builds/inverse_relations/owl/PheKnowLator_v2.0.0_full_subclass_inverseRelations_OWL_NodeLabels.txt",
             "https://storage.googleapis.com/pheknowlator/archived_builds/release_v2.0.0/build_10MAY2020/knowledge_graphs/subclass_builds/inverse_relations/owl/PheKnowLator_v2.0.0_full_subclass_inverseRelations_OWL_Stats.txt", ...

            ]
        },
sonarcloud[bot] commented 2 years ago

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information