Character maps to <undefined>

zsoi commented 4 years ago

Unity Version: 2019.3.8.f1 (Win64) Analyzer Version: master (49396c2a7a3a63ce5786d48099a4c77655fc76f0) Addressables Version: 1.7.5 SBP Version: 1.5.10 Python: 3.8.2 (Win64)

I've ran the analyzer over our set of asset bundles which are generated by Addressables and at least one bundle could not be parsed properly:

Traceback (most recent call last):
  File ".\analyzer.py", line 1450, in <module>
    main()
  File ".\analyzer.py", line 78, in main
    objs = p.parse(datafile + ".txt")
  File ".\analyzer.py", line 145, in parse
    data = f.read()
  File "[..]\Python38\lib\encodings\cp1252.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 21026952: character maps to <undefined>

Beforehand, some warnings about a unhandled type were printed. Maybe the converted bundle files should be handled as UTF-8?

The analyzer just exits when it reaches this error. Maybe it should continue parsing the rest of the bundles, only printing a warning/error for this bundle?

BeregAl commented 3 years ago

Unity Version: 2019.3.8.f1 (Win64) Analyzer Version: master (49396c2) Addressables Version: 1.7.5 SBP Version: 1.5.10 Python: 3.8.2 (Win64)

I've ran the analyzer over our set of asset bundles which are generated by Addressables and at least one bundle could not be parsed properly:
Traceback (most recent call last):
  File ".\analyzer.py", line 1450, in <module>
    main()
  File ".\analyzer.py", line 78, in main
    objs = p.parse(datafile + ".txt")
  File ".\analyzer.py", line 145, in parse
    data = f.read()
  File "[..]\Python38\lib\encodings\cp1252.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 21026952: character maps to <undefined>
Beforehand, some warnings about a unhandled type were printed. Maybe the converted bundle files should be handled as UTF-8?

The analyzer just exits when it reaches this error. Maybe it should continue parsing the rest of the bundles, only printing a warning/error for this bundle?

Same to me, I have a bunch of Warning: unhandled type Type*! Warning: unhandled type RectOffset! at Traceback (most recent call last): File "C:\Kosmos\MG\analyzer.py", line 1450, in main() File "C:\Kosmos\MG\analyzer.py", line 78, in main objs = p.parse(datafile + ".txt") File "C:\Kosmos\MG\analyzer.py", line 145, in parse data = f.read() and just AppData\Local\Programs\Python\Python39\lib\encodings\cp1251.py", line 23, in decode return codecs.charmap_decode(input,self.errors,decoding_table)[0] UnicodeDecodeError: 'charmap' codec can't decode byte 0x98 in position 67255: character maps to

zsoi commented 3 years ago

You may use my fork at https://github.com/zsoi/asset-bundle-analyzer where I fixed some of these issues

BeregAl commented 3 years ago

You may use my fork at https://github.com/zsoi/asset-bundle-analyzer where I fixed some of these issues Thanks, I tried this one, but still receiving error

Parsing CAB-193f899aa0ebdad4593c934cc1a12fb5
Traceback (most recent call last):
  File "C:\***\MG\analyzer.py", line 1486, in <module>
    main()
  File "C:\***\MG\analyzer.py", line 81, in main
    processor.process_objects(bundle_id, objs, db, f2, args.store_raw)
  File "C:\***\MG\analyzer.py", line 430, in process_objects
    for object_id, obj in objs:
  File "C:\***\MG\analyzer.py", line 128, in parse
    line = f.readline()
  File "C:\Users\****\AppData\Local\Programs\Python\Python39\lib\codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x91 in position 1064: invalid start byte

What can I do?

faelenor commented 3 years ago

I committed a change that should fix this issue.

BeregAl commented 3 years ago

I committed a change that should fix this issue.

Thank you very much, i was able to analyze some of my bundles, but I still getting exception like this. Hope the trace helps.

  ParsedField(level=7, name='size', value='9', type='int')
  ParsedField(level=7, name='data', value='"en"', type='string')
  ParsedField(level=7, name='data', value='"ru"', type='string')
  ParsedField(level=7, name='data', value='"es"', type='string')
  ParsedField(level=7, name='data', value='"mx"', type='string')
  ParsedField(level=7, name='data', value='"de"', type='string')
  ParsedField(level=7, name='data', value='"fr"', type='string')
  ParsedField(level=7, name='data', value='"pt"', type='string')
  ParsedField(level=7, name='data', value='"it"', type='string')
  ParsedField(level=7, name='data', value='"tr"', type='string')
Traceback (most recent call last):
  File "C:\***\MG\analyzer.py", line 1451, in <module>
    main()
  File "C:\***\MG\analyzer.py", line 79, in main
    objs = p.parse(datafile + ".txt")
  File "C:\***\MG\analyzer.py", line 158, in parse
    objects[int(match[0])] = {"ClassID": int(match[1]), "Type": match[2], "Content": self._parse_obj()}
  File "C:\***\MG\analyzer.py", line 297, in _parse_obj
    obj[field_name] = Field(field.type, self._parse_obj(level+1))
  File "C:\***\MG\analyzer.py", line 297, in _parse_obj
    obj[field_name] = Field(field.type, self._parse_obj(level+1))
  File "C:\***\MG\analyzer.py", line 297, in _parse_obj
    obj[field_name] = Field(field.type, self._parse_obj(level+1))
  [Previous line repeated 2 more times]
  File "C:\***\MG\analyzer.py", line 237, in _parse_obj
    raise Exception("Error parsing array!")
Exception: Error parsing array!

shelllee commented 3 years ago

maybe u could have a look at my second commit:

https://github.com/faelenor/asset-bundle-analyzer/pull/16

faelenor / asset-bundle-analyzer

Character maps to <undefined> #8