search

dumbmatter / sas7bdat-js

Read SAS files in JavaScript. Because you always wanted to do that, right?
27 stars 4 forks source link

Big endian not supported #2

Open imathews opened 5 years ago

imathews commented 5 years ago

In the struct_unpack function, an error is thrown if the endianness is BE. I tried swapping the endianness of the buffer as suggested in the corresponding comment (using the native buf.swap64()), but didn't have much success — the script just exits after printing one row as an empty array.

Do you have a sense of what it would take to support big endian encodings? fwiw, the python version of this library seems to work fine, though I'd prefer to leverage the JS implementation here.

dumbmatter commented 5 years ago

IIRC, I skipped big endian support because I didn't have any test data. If you have some (ideally a small file), it'd be helpful if you could share it. I'm not sure why your solution didn't work, but I imagine we could figure it out by comparing what we try in JS with what exactly the Python library does.

imathews commented 5 years ago

Thanks much for the follow-up. I'd love to share a test case, but unfortunately the data I'm working with are fairly sensitive, and I wouldn't know how to appropriately deidentify / create a separate test BE file. I would be happy to test out any future updates against my BE file if it's helpful.

I ended up forking the java-based sas-convert library (which uses parso under the hood), which ended up working quite well and was a good deal faster than the python implementation.

dumbmatter commented 5 years ago

Yeah I understand about sensitive data, I've had the same problem at times. Thanks for the offer for testing help! And Parso is pretty awesome - best of the open source SAS file readers, I think.