ubc-systopia / Indaleko

Indaleko Project
GNU Affero General Public License v3.0
0 stars 1 forks source link

Windows local ingester issue: "int too big to convert" #14

Open fsgeek opened 7 months ago

fsgeek commented 7 months ago

Here is the error:

Traceback (most recent call last): File "msgpack\_packer.pyx", line 180, in msgpack._cmsgpack.Packer._pack OverflowError: int too big to convert

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Users\TonyMason\source\repos\indaleko-test\IndalekoWindowsLocalIngester.py", line 385, in main() File "C:\Users\TonyMason\source\repos\indaleko-test\IndalekoWindowsLocalIngester.py", line 380, in main ingester.ingest() File "C:\Users\TonyMason\source\repos\indaleko-test\IndalekoWindowsLocalIngester.py", line 191, in ingest obj = self.normalize_index_data(item) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\TonyMason\source\repos\indaleko-test\IndalekoWindowsLocalIngester.py", line 134, in normalize_index_data 'raw_data' : msgpack.packb(data), ^^^^^^^^^^^^^^^^^^^ File "C:\Users\TonyMason\AppData\Local\Programs\Python\Python312\Lib\site-packages\msgpack__init__.py", line 36, in packb return Packer(**kwargs).pack(o) ^^^^^^^^^^^^^^^^^^^^^^^^ File "msgpack\_packer.pyx", line 294, in msgpack._cmsgpack.Packer.pack File "msgpack\_packer.pyx", line 300, in msgpack._cmsgpack.Packer.pack File "msgpack\_packer.pyx", line 297, in msgpack._cmsgpack.Packer.pack File "msgpack\_packer.pyx", line 231, in msgpack._cmsgpack.Packer._pack File "msgpack\_packer.pyx", line 188, in msgpack._cmsgpack.Packer._pack OverflowError: Integer value out of range

We need to handle this more gracefully than bailing out.

fsgeek commented 7 months ago

See 73ed843 for resolution.

What it appears happens here is that the underlying call masks a variety of errors, but in doing so it returns bogus data in the return results. Most errors seem to be related to "access denied" variations but there are at least two others that I have seen, which do not really make sense. More logging was added as a result, which led to more statistics (#12). Further analysis may be justified.