search

a2800276 / 8583

ruby implementation of iso 8583 financial messages
http://www.kuriositaet.de
MIT License
41 stars 44 forks source link

Update to >= Ruby 1.9 compatibility #7

Closed seandunn closed 9 years ago

seandunn commented 10 years ago

Hi Tim,

It looks like some of the issues I've been seeing with parsing ANS fields are related String encoding in Ruby >= 1.9 (I know the current code base only supports ruby 1.8 but I'm tied to some other libraries which don't support < 1.9).

I've been trying to recreate the issues I've seen with a test file and I've run into other encoding issues with test files. Explicitly setting file encoding to US-ASCII seems to fix most failures in the test suite (except for EBCDIC conversion).

I was wondering what your thoughts how best to properly update things for >= 1.9 I considered updating everything to assume UTF-8 and then converting to US-ASCII, etc where needed. Does this sound sensible?

Ruby's new built in encodings don't seem to cover EBCDIC. iconv is still available as a gem but the code comments in util.rb suggest that it's not reliable on every platform.

Please let me know if I can do anything to help.

Cheers, Sean.

a2800276 commented 10 years ago

I'm afraid the whole situation kindof sucks. One really shouldn't stick to 1.8. But I'm very upset about how 1.9+ handles Strings at the cost of not being able to deal with binary data in a straightforward way.

So concerning your question of how to properly update: I don't have a clue.

Off the top of my head, I'd venture to guess treating strings as UTF-8 per default is not a good idea because I doubt there are any 8583 dialects out there that handle multi-byte chars. Most will either do ascii, ebcdic or local 8859-x encodings, i.e. latin-1.

EBCDIC should be a problem, it's so constrained I would be happy to bang together a lookup table for any necessary conversions. What 8583 dialect are you working with? Do you need EBCDIC?

BTW feel free to send me a mail if it's not appropriate to discuss in public.

seandunn commented 10 years ago

Maybe setting back to US-ASCII would be a better solution then rather than using new string encoding stuff in >=1.9.

I did a quick sed on the source files to insert the encoding comment. The only issue I had was with the EBCDIC encoding in the test suite. I checked our protocol and everything's in ASCII, which I'm guessing, is probably the most common use case.

a2800276 commented 9 years ago

I'm beginning to think the sane thing to do would be a rewrite, preferably in a saner language than ruby. All the 1.9->2.bla-p123 issues, rvm, bundle and gemfiles are beginning to drive me insane...