LoadLocalJSON is converting curly apostrophes to ’ because of encoding issue

[Jonseed]

I am loading JSON from disk with LoadLocalJSON node, but it is converting curly apostrophes ’ to ’. This is probably an encoding error; on my system it defaults to opening the file in cp1252 (Windows-1252) encoding rather than UTF-8.

It looks like you are not explicitly setting the encoding when you open the JSON file, which is probably the source of the issue. Maybe you could detect the encoding of the file first with the chardet library before setting the explicit encoding type when loading the JSON file, like this:

import chardet
....

        # Detect the encoding
        with open(file_path, 'rb') as file:
            raw_data = file.read()
        result = chardet.detect(raw_data)
        encoding = result['encoding']

        # Open and read the file with detected encoding
        with open(file_path, 'r', encoding=encoding) as file:
            data = json.load(file)

Unfortunately, this detection would require another package on install (chardet). If you don't want to require an additional package, you could just set the encoding type explicitly as UTF-8 when you read the JSON file, which I think is the standard encoding format for JSON.

with open(file_path, 'r', encoding='utf-8') as file:
            data = json.load(file)

I tried both of these methods, and my JSON file was properly read in with curly apostrophes intact. Btw, your SaveJSON node already writes JSON to file with explicit UTF-8 encoding. It is just the reading the JSON file back in that is the issue.

[lucafoscili]

It makes absolutely sense to add the utf-8 encoding, I didn't think about it! I added the fix on-the-fly while making the new release. I did quite a big refactoring, if something broke let me know (also if the fix is ok)!

[Jonseed]

The fix works great! I'll let you know if anything else broke.