New options for loading from URLs

caufieldjh commented 2 years ago

Fixes for:

62 - can now use a URL in the YAML for embedding_file_name
pre-run checks verify the file extension is one of the expected (for now, that's TSV or tar.gz) for for node/edgelists and embeddings
Decompress compressed single files when specified in a URL (this does not consider local compressed files - yet)
43 - either combined graph.tar.gz or URL to graph.tar.gz specified in new value in YAML, graph_path. The values for node_path and edge_path still need to be present and refer to decompressed graph_path contents.

caufieldjh commented 2 years ago

@caufieldjh one comment - this tar may have other things in it besides the node and edge file. I seem to recall us stuffing in another file in KG-COVID-19

Hrm, I only see node and edge files in the kg-covid-19.tar.gz (at least for https://kg-hub.berkeleybop.io/kg-covid-19/current/kg-covid-19.tar.gz) - what else is likely to be in there?

sonarcloud[bot] commented 2 years ago

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
1 Code Smell

91.5% Coverage
0.0% Duplication

justaddcoffee commented 1 year ago

Sorry, not 100% sure - I think in previous builds we may have put the stats file in there

On Fri, Apr 1, 2022 at 12:08 PM Harry Caufield @.***> wrote:

@caufieldjh https://github.com/caufieldjh one comment - this tar may have other things in it besides the node and edge file. I seem to recall us stuffing in another file in KG-COVID-19

Hrm, I only see node and edge files in the kg-covid-19.tar.gz (at least for https://kg-hub.berkeleybop.io/kg-covid-19/current/kg-covid-19.tar.gz)

what else is likely to be in there?

— Reply to this email directly, view it on GitHub https://github.com/Knowledge-Graph-Hub/NEAT/pull/63#issuecomment-1086238441, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABEWJ6SGVCKR7OYZPGKGL3VC5CS5ANCNFSM5SJI3NVA . You are receiving this because you commented.Message ID: @.***>

-- Justin Reese, Ph.D. Computer Research Scientist Environmental Genomics and Systems Biology Lawrence Berkeley National Laboratory phone (415) 909-9418 | (510) 486-5078

Knowledge-Graph-Hub / neat-ml

New options for loading from URLs #63

62 - can now use a URL in the YAML for `embedding_file_name`

43 - either combined graph.tar.gz or URL to graph.tar.gz specified in new value in YAML, `graph_path`. The values for `node_path` and `edge_path` still need to be present and refer to decompressed `graph_path` contents.

Knowledge-Graph-Hub / neat-ml

New options for loading from URLs #63

62 - can now use a URL in the YAML for embedding_file_name

43 - either combined graph.tar.gz or URL to graph.tar.gz specified in new value in YAML, graph_path. The values for node_path and edge_path still need to be present and refer to decompressed graph_path contents.

62 - can now use a URL in the YAML for `embedding_file_name`

43 - either combined graph.tar.gz or URL to graph.tar.gz specified in new value in YAML, `graph_path`. The values for `node_path` and `edge_path` still need to be present and refer to decompressed `graph_path` contents.