Fix a bug related to unicode. Note we still don't tokenize unicode except to make all unicode characters tokenize to 1 uniformly. But now it doesn't crash when we see a unicode character.
Fix a verbose warning in the numpy code related to casting. The casting still works the same as before.
Fix a bug related to unicode. Note we still don't tokenize unicode except to make all unicode characters tokenize to
1
uniformly. But now it doesn't crash when we see a unicode character.Fix a verbose warning in the numpy code related to casting. The casting still works the same as before.
The tests pass.