[Bug]: Applio fails to start due to mistaken encoding of tts_voices.json

[progmars]

Project Version

main branch

Platform and OS Version

Windows 11, 64bit

Affected Devices

Computer

Existing Issues

No response

What happened?

Installed Applio inside pinokio. Attempted to start it. It failed. The error in the logs says the following:

D:\pinokio\api\applio.git\applio>conda_hook && conda deactivate && conda deactivate && conda deactivate && conda activate base && D:\pinokio\api\applio.git\applio\env\Scripts\activate D:\pinokio\api\applio.git\applio\env && python app.py
Traceback (most recent call last):
  File "D:\pinokio\api\applio.git\applio\app.py", line 19, in <module>
    from tabs.inference.inference import inference_tab
  File "D:\pinokio\api\applio.git\applio\tabs\inference\inference.py", line 8, in <module>
    from core import (
  File "D:\pinokio\api\applio.git\applio\core.py", line 33, in <module>
    voices_data = load_voices_data()
  File "D:\pinokio\api\applio.git\applio\core.py", line 30, in load_voices_data
    return json.load(f)
  File "D:\pinokio\bin\miniconda\lib\json\__init__.py", line 293, in load
    return loads(fp.read(),
  File "D:\pinokio\bin\miniconda\lib\encodings\cp1257.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0xa5 in position 125800: character maps to <undefined>

When inspecting the issue, it seems that JSON fails to detect that the file tts_voices.json is in UTF-8 and tries to use cp1257.

Steps to reproduce

1. Install Applio inside pinokio
2. Run Applio in pinokio

Expected behavior

Expected Applio to start.

Attachments

No response

Screenshots or Videos

No response

Additional Information

Running in pinokio

[blaisewf]

you can open a pr with your fix :)

[progmars]

Actually, my initial idea of the fix was insufficient; it failed for the same reason in tts.py. So it needs two fixes:

core.py:

with open(os.path.join("rvc", "lib", "tools", "tts_voices.json"), encoding='utf-8') as f:

tts.py:

with open(json_path, "r", encoding='utf-8') as file:

I will create a PR then.

[AznamirWoW]

if you're using Win11, maybe just easier to enable utf-8 in regional console settings? (in Win10 as well)

[progmars]

@AznamirWoW Wondering if this would affect Python JSON loader as well? Is it using this Windows setting?

@blaisewf I cloned the latest main, just to be sure my PR gets applied to the latest version and works with it... but now I cannot start Applio anymore because of another error:

Traceback (most recent call last):
  File "D:\pinokio\api\applio.git\applio\app.py", line 92, in <module>
    inference_tab()
  File "D:\pinokio\api\applio.git\applio\tabs\inference\inference.py", line 392, in inference_tab
    choices=get_speakers_id(model_file.value),
  File "D:\pinokio\api\applio.git\applio\tabs\inference\inference.py", line 303, in get_speakers_id
    model_data = torch.load(model, map_location="cpu")
  File "D:\pinokio\api\applio.git\applio\env\lib\site-packages\torch\serialization.py", line 997, in load
    with _open_file_like(f, 'rb') as opened_file:
  File "D:\pinokio\api\applio.git\applio\env\lib\site-packages\torch\serialization.py", line 449, in _open_file_like
    return _open_buffer_reader(name_or_buffer)
  File "D:\pinokio\api\applio.git\applio\env\lib\site-packages\torch\serialization.py", line 434, in __init__
    _check_seekable(buffer)
  File "D:\pinokio\api\applio.git\applio\env\lib\site-packages\torch\serialization.py", line 543, in _check_seekable
    raise_err_msg(["seek", "tell"], e)
  File "D:\pinokio\api\applio.git\applio\env\lib\site-packages\torch\serialization.py", line 536, in raise_err_msg
    raise type(e)(msg)
AttributeError: 'NoneType' object has no attribute 'seek'. You can only torch.load from a file that is seekable. Please pre-load the data into a buffer like io.BytesIO and try to load from it instead.

It looks like there have been recent changes in inference.py that break stuff because of issues with model_file.value. This one I don't know how to fix.

[progmars]

Ah, it looks like these were the changes that made it impossible to start Applio: https://github.com/IAHispano/Applio/pull/729 Something might have been missing in that PR. Maybe it expects some kind of model files to be loaded somewhere that were present on the PR developer's machine...

[AznamirWoW]

@AznamirWoW Wondering if this would affect Python JSON loader as well? Is it using this Windows setting?

It looks like there have been recent changes in inference.py that break stuff because of issues with model_file.value. This one I don't know how to fix.

Use a compiled Applio build, not a git clone that may or may not be stable.

As for Python and json loader, as far as I know Python has issues opening files with non-standard console code page, thus enabling utf-8 solves most of the issues.

[progmars]

@AznamirWoW Ok, I'll try that. I used git clone mostly because I wanted to make sure that my PR could be merged above the latest code without conflicts (and also because Applio wrapper for pinokio also uses git clone instead of a stable branch).

Anyway, while browsing the source code, I noticed that some json.load calls already have encoding='utf-8', so I guess someone else also considered it more fool-proof, but it was not applied everywhere consistently.