jhosmer / PySmile

A Python library for encoding/decoding SMILE format data (libsmile)
Apache License 2.0
24 stars 9 forks source link

Problem while decoding similar strings #1

Open nvitucci opened 8 years ago

nvitucci commented 8 years ago

Two SMILE strings encode two similar objects, but good is decoded correctly while bad is not.

>>> good = ':)\n\x05\xfa\x84prova\xfa\x86version\xc4\x84stateCopen\x87settings\xfa\x92index.creation_dateL1460931472628\x97index.number_of_replicas@1\x95index.number_of_shards@5\x89index.uuidU4AyLlTWqQ8aHz4VULvVABA\x94index.version.createdF2030199\xfb\x87mappings\xf8\xfa\x86company\xfa\x89properties\xfa\x88someField\xfa\x83typeEstring\xfb\xfb\xfb\xfb\xf9\x86aliases\xfa\xfb\xfb\xfb'

>>> bad = ':)\n\x05\xfa\x84prova\xfa\x86version\xc4\x84stateCopen\x87settings\xfa\xabindex.analysis.analyzer.startswith.tokenizerFkeyword\x92index.creation_dateL1460931374437\x97index.number_of_replicas@1\x95index.number_of_shards@5\x89index.uuidUaEmQxmlGQcyECp1LmCkCcw\x94index.version.createdF2030199\xfb\x87mappings\xf8\xfa\x86company\xfa\x89properties\xfa\x88someField\xfa\x83typeEstring\xfb\xfb\xfb\xfb\xf9\x86aliases\xfa\xfb\xfb\xfb'

>>> pysmile.decode(good)

{u'prova': {u'aliases': {},
  u'mappings': [{u'company': {u'properties': {u'someField': {u'type': u'string'}}}}],
  u'settings': {u'index.creation_date': u'1460931472628',
   u'index.number_of_replicas': u'1',
   u'index.number_of_shards': u'5',
   u'index.uuid': u'4AyLlTWqQ8aHz4VULvVABA',
   u'index.version.created': u'2030199'},
  u'state': u'open',
  u'version': 2}}

>>> pysmile.decode(bad)

SMILEDecodeError: ('Unable to jsonify string: \'{"prova":{"version":2,"state":"open","settings":{"index.analys":"s.analyzer.startswith.tokenizerFkeyword\\x92in","x.creation_dateL1460931374437\\x97index.nu","er_of_replicas@1\\x95index.number_of_sh","ds@5\\x89index.uuidUaEmQxmlGQcyECp1LmCkCcw\\x94index.versio",,"eatedF2030199\\xfb\\x87mappings\\xf8\\xfa\\x86company\\xfa\\x89properties\\xfa\\x88some","eld\\xfa\\x83typeEstring\\xfb\\xfb\\xfb\\xfb\\xf9\\x86aliases\\xfa\\xfb\\xfb\\xfb"\'', '{"prova":{"version":2,"state":"open","settings":{"index.analys":"s.analyzer.startswith.tokenizerFkeyword\x92in","x.creation_dateL1460931374437\x97index.nu","er_of_replicas@1\x95index.number_of_sh","ds@5\x89index.uuidUaEmQxmlGQcyECp1LmCkCcw\x94index.versio",,"eatedF2030199\xfb\x87mappings\xf8\xfa\x86company\xfa\x89properties\xfa\x88some","eld\xfa\x83typeEstring\xfb\xfb\xfb\xfb\xf9\x86aliases\xfa\xfb\xfb\xfb"')
drax68 commented 6 years ago

Same problem here, SMILEDecodeError with crippled json in stacktrace. @nvitucci have you found solution for that? @spfjr observing same issue with your fork as well.