When I try to pass a UTF-8 charlist, characters such as ł which equate to <<197, 130>> actually go in as 322 in the charlist.
iex(1)> 'hełło'
[104, 101, 322, 322, 111]
This causes things to break (sometimes I see :erlang.iolist_size([322]) which fails because its > 255, for example), sometimes it just fails to match (depending on the current parsing context I guess)
Am I doing something wrong? (I'm assuming I am!)
I've currently got around this very very crudely by stepping through the bytes and turning it in to a normal list (so I get [197, 130] instead of [322]) then when the result comes back from apply, turn anything in the state that is a string back by stepping through the list and adding to a <<>>.
When I try to pass a UTF-8 charlist, characters such as
łwhich equate to<<197, 130>>actually go in as322in the charlist.This causes things to break (sometimes I see
:erlang.iolist_size([322])which fails because its > 255, for example), sometimes it just fails to match (depending on the current parsing context I guess)Am I doing something wrong? (I'm assuming I am!)
I've currently got around this very very crudely by stepping through the bytes and turning it in to a normal list (so I get
[197, 130]instead of[322]) then when the result comes back fromapply, turn anything in the state that is a string back by stepping through the list and adding to a<<>>.Great work on this BTW!