Forward vs Backward Slash in Path definition

[metin-akyol]

Dealing with a very simply problem but haven't been able to find a good solution (or any solution):

The local_confic.json file defines folder bath UNIX style with forward slashes: "path": "/allrank/dummy_data",

Now I can't change this to backward slash in there, because the JSON file will interpret those as commands.

Next, the main file passes this as an argument to the load_libsvm_dataset function:

input_path=config.data.path

And in is there, this problem is supposed to be fixed with:

input_path =  Path(input_path)    
path = os.path.join(input_path, "{}.txt".format(role))

But it isn't converted correctly, so I get this error:

FileNotFoundError: [Errno 2] No such file or directory: '/allrank/dummy_data\train.txt'

I have tried fixing this with Path and various other conversions but couldn't get it to work (when I use Path I end up getting double backward slashes). Can anyone suggest a solution?

[kretes]

Hello,

Replying to the origin of the issue so

Now I can't change this to backward slash in there, because the JSON file will interpret those as commands.

What's the error you're getting?

Json file is not interpretting anything as commands, but single backslash \ is an escape character, so if you want a literal backslash you need to put double backslash in JSON, otherwise you'll get

Traceback (most recent call last):
  File "allrank/main.py", line 114, in <module>
    run()
  File "allrank/main.py", line 50, in run
    config = Config.from_json(paths.config_path)
  File "/allrank/allrank/config.py", line 83, in from_json
    config = json.load(config_file)
  File "/usr/local/lib/python3.7/json/__init__.py", line 296, in load
    parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "/usr/local/lib/python3.7/json/__init__.py", line 348, in loads
    return _default_decoder.decode(s)
  File "/usr/local/lib/python3.7/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/lib/python3.7/json/decoder.py", line 353, in raw_decode
    obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Invalid \escape: line 24 column 22 (char 403)

[metin-akyol]

Thank you kretes for your response. Yes, I misspoke in my post, I meant that it is an escape character (I get this error: Invalid escape character in string.json(261). But when I use the double backslash I end up with this error message:

"path": "\\allrank\\dummy_data",

[INFO] 2023-03-07 19:18:19 - exit_code = 0
[INFO] 2023-03-07 19:18:19 - will load train data from \allrank\dummy_data\train.txt
Traceback (most recent call last):
  File "C:\Users\cramk\Documents\allRank\allrank\main.py", line 114, in <module>
    run()
  File "C:\Users\cramk\Documents\allRank\allrank\main.py", line 57, in run
    train_ds, val_ds = load_libsvm_dataset(
  File "C:\Users\cramk\anaconda3\lib\site-packages\allrank\data\dataset_loading.py", line 207, in load_libsvm_dataset
    train_ds = load_libsvm_dataset_role("train", input_path, slate_length)
  File "C:\Users\cramk\anaconda3\lib\site-packages\allrank\data\dataset_loading.py", line 222, in load_libsvm_dataset_role
    ds = load_libsvm_role(input_path, role)
  File "C:\Users\cramk\anaconda3\lib\site-packages\allrank\data\dataset_loading.py", line 179, in load_libsvm_role
    with open_local_or_gs(path, "rb") as input_stream:
  File "C:\Users\cramk\anaconda3\lib\site-packages\allrank\utils\file_utils.py", line 65, in open_local_or_gs
    return open_func(path, mode)
FileNotFoundError: [Errno 2] No such file or directory: '\\allrank\\dummy_data\\train.txt'

Somehow in the function load_libsvm_role, the double backslashes are added, because at the beginning the path seems correct with just one slash. Any changes I make in dataset_loading.py are ignored obviously because the package has been installed or loaded already, and I can't quite figure out how to force it to reimport the file with my changes

[metin-akyol]

I was able to figure it out (took a little longer to get to the bottom of this with all the wrapped functions). With the folder path being relative, I was in the wrong folder.