Hi there, I've been having trouble saving .kiko files correctly with python 3. As far as I can tell, when a kiko file is saved the data files that get added to the underlying archive are truncated at zero bytes. You can reproduce this with the following code snippet:
import os
import tempfile
from kiko.io.kikofile import KikoFile
handle, kiko_path = tempfile.mkstemp(suffix=".kiko")
os.close(handle)
kiko_file = KikoFile(kiko_path)
kiko_file.set_data({"test": "data"})
kiko_file.save()
kiko_file.parse()
When I do this I get an error that looks something like:
Traceback (most recent call last):
File "/home/ben.hawkyard/Documents/scripts/testing/kiko_metadata_testing.py", line 11, in <module>
kiko_file.parse()
File "/opt/rez/rez_package_cache/kiko/1.0.0/a99e/a/python/kiko/io/kikofile.py", line 145, in parse
self._metadata = json.load(v)
^^^^^^^^^^^^
File "/opt/rez/rez_package_cache/python/3.11.8/3e7f/a/lib/python3.11/json/__init__.py", line 293, in load
return loads(fp.read(),
^^^^^^^^^^^^^^^^
File "/opt/rez/rez_package_cache/python/3.11.8/3e7f/a/lib/python3.11/json/__init__.py", line 346, in loads
return _default_decoder.decode(s)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/rez/rez_package_cache/python/3.11.8/3e7f/a/lib/python3.11/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/rez/rez_package_cache/python/3.11.8/3e7f/a/lib/python3.11/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
calling io.seek(0) in _add_to_tar_from_dict means that f_obj.tell() in _add_to_tar returns zero and this causes us to write empty files to our kiko file when adding dicts to it.
I've done some testing and I think this could be fixed by changing the else branch in _add_to_tar:
else:
f_obj = BytesIO(f_obj.read().encode("utf-8")) # TarFile expects file objects to be opened in binary mode.
info.size = f_obj.seek(0, os.SEEK_END)
f_obj.seek(0)
info.mtime = time.time()
I'm happy to put in a pull request for this if there's interest. I can also see that mottosse fixes this in their pull request for kiko here: https://github.com/Toolchefs/kiko/pull/6. Is this likely to be accepted, do you think?
Hi there, I've been having trouble saving .kiko files correctly with python 3. As far as I can tell, when a kiko file is saved the data files that get added to the underlying archive are truncated at zero bytes. You can reproduce this with the following code snippet:
When I do this I get an error that looks something like:
I think this is caused by this bit of code in io/kikofile.py:
calling
io.seek(0)
in_add_to_tar_from_dict
means thatf_obj.tell()
in_add_to_tar
returns zero and this causes us to write empty files to our kiko file when adding dicts to it.I've done some testing and I think this could be fixed by changing the else branch in
_add_to_tar
:I'm happy to put in a pull request for this if there's interest. I can also see that mottosse fixes this in their pull request for kiko here: https://github.com/Toolchefs/kiko/pull/6. Is this likely to be accepted, do you think?