Knowledge-Graph-Hub / neat-ml

Network Embedding All the Things
BSD 3-Clause "New" or "Revised" License
18 stars 1 forks source link

Loading from URL looks for wrong filename #65

Closed caufieldjh closed 2 years ago

caufieldjh commented 2 years ago

Loading graph objects from a URL isn't quite right - the file is downloaded but yaml_helper looks for a file matching the URL string rather than the 'safe' reformatted filename. Example: The neat.yaml contains this:

    graph_path: https://kg-hub.berkeleybop.io/kg-ontoml/20220304/KG-OntoML.tar.gz 

The file is downloaded:

$ ls -ls https___kg-hub.berkeleybop.io_kg-ontoml_20220304_KG-OntoML.tar.gz 
43200 -rw-r--r-- 1 harry harry 44235554 Apr 13 14:36 https___kg-hub.berkeleybop.io_kg-ontoml_20220304_KG-OntoML.tar.gz

but raises FileNotFoundError upon trying to decompress it:

$ neat run --config neat.yaml
Traceback (most recent call last):
  File "/home/harry/kg-env/bin/neat", line 8, in <module>
    sys.exit(cli())
  File "/home/harry/kg-env/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/harry/kg-env/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/harry/kg-env/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/harry/kg-env/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/harry/kg-env/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/harry/kg-env/lib/python3.8/site-packages/neat/cli.py", line 41, in run
    if not pre_run_checks(yhelp=yhelp):
  File "/home/harry/kg-env/lib/python3.8/site-packages/neat/pre_run_checks/pre_run_checks.py", line 82, in pre_run_checks
    if check_file_extensions and yhelp.main_graph_args():
  File "/home/harry/kg-env/lib/python3.8/site-packages/neat/yaml_helper/yaml_helper.py", line 211, in main_graph_args
    return self.add_indir_to_graph_data(self.yaml['graph_data']['graph'])
  File "/home/harry/kg-env/lib/python3.8/site-packages/neat/yaml_helper/yaml_helper.py", line 164, in add_indir_to_graph_data
    decomp_outfile = tarfile.open(filepath)
  File "/usr/lib/python3.8/tarfile.py", line 1603, in open
    return func(name, "r", fileobj, **kwargs)
  File "/usr/lib/python3.8/tarfile.py", line 1667, in gzopen
    fileobj = GzipFile(name, mode + "b", compresslevel, fileobj)
  File "/usr/lib/python3.8/gzip.py", line 173, in __init__
    fileobj = self.myfileobj = builtins.open(filename, mode or 'rb')
FileNotFoundError: [Errno 2] No such file or directory: 'https://kg-hub.berkeleybop.io/kg-ontoml/20220304/KG-OntoML.tar.gz'

Performing any further actions on the downloaded file should use the updated filename.