Knowledge-Graph-Hub / neat-ml

Network Embedding All the Things
BSD 3-Clause "New" or "Revised" License
18 stars 1 forks source link

Use URLs as node and edge list locations #36

Closed caufieldjh closed 2 years ago

caufieldjh commented 2 years ago

The current yaml format expects to be provided direct local paths to node and edge lists: https://github.com/Knowledge-Graph-Hub/NEAT/blob/768cdf6d8bb9f069339e1c4d7519d0d73cfef15b/tests/resources/test.yaml#L8-L9 If it could be provided with URLs instead or additionally to local paths, remote runs of NEAT would be easier, since we assume it will have to retrieve node/edgelists from somewhere else anyway.

justaddcoffee commented 2 years ago

Discussion with Harry - at least two ways of doing this:

1) allow users to specify URLs in node_path and edge_path - this would require a regex or something to decide when something is a URL and when it's a file path 2) make a new pair of keys above graph called edge_url and node_url, use these to download the files and proceed as usual. These would have to live outside of the graph key, since the graph key/values are passed directory to Ensmallen

justaddcoffee commented 2 years ago

closed by #40

justaddcoffee commented 2 years ago
  1. allow users to specify URLs in node_path and edge_path - this would require a regex or something to decide when something is a URL and when it's a file path

for the record, we ended up going with 1. above