pmercury UnicodeDecodeError: HTTP user-agent (\x99)

[cglewis]

{
    'src_ip': 'x.x.x.x',
    'dst_ip': 'x.x.x.x',
    'src_port': 10630,
    'dst_port': 80,
    'protocol': 6,
    'event_start': 1295981648.965332,
    'fingerprints': {
        'http': '(474554)(485454502f312e31)(557365722d4167656e74)(486f7374)(43616368652d436f6e74726f6c3a206e6f2d6361636865)'
    },
    'http': {
        'user_agent': b'Skype\x99 5.0',
        'host': b'ui.skype.com'
    }
}

 File "/usr/local/bin/pmercury", line 368, in write_record
    self.out_file_pointer.write('%s\n' % json.dumps(flow_repr))
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x99 in position 290: invalid start byte

[banderson84]

Hi, was this installed via pip or from source? The pip version is a little dated (I will update it soon). I think this is fixed in the current code.

[cglewis]

ah, this was with pip. an updated package would be much appreciated, thanks!

[banderson84]

The cython extensions made pmercury pretty fast (for large pcaps) at the cost of a significantly more complicated process to build pip-ready wheels. I just updated the pip package for linux/windows python 3.6+ (I currently don't have access to an MacOS machine, but I will work on that). Before today, I haven't created a manylinux wheel, so please let me know if you run into any problems.

[cglewis]

Right on! That worked for me on debian, and fixed the issue I was running into. Thanks!