search

yesodweb / persistent

Persistence interface for Haskell allowing multiple storage methods.
MIT License
467 stars 296 forks source link

NUL or \0 characters not allowed in middle of SQLite Text or Blobs #310

Closed ryantm closed 9 years ago

ryantm commented 10 years ago

Since bindBlob and bindText use useAsCString, they forbid the binary or text data from containing a NUL or \0 or U+0000 byte in the middle of it. I think this would be fixed by the completion of #92, since direct-sqlite uses unsafeUseAsCStringLen, which doesn't assume byte arrays are terminated by the \0 byte.

snoyberg commented 10 years ago

I agree with you completely on the blobs, that should be fixed. Regarding text, however, I'm not so sure. The docs say:

The result of expressions involving strings with embedded NULs is undefined.

So perhaps this should just be fixed in the case of blobs, not text?

snoyberg commented 10 years ago

Never mind, I found out what the problem was in my previous implementation (accidentally having an empty string treated as a NULL). I can get it to work for both Text and ByteString. I've just pushed that code to Github, and release to Hackage as 2.1.0.1.

MaxGabriel commented 9 years ago

I think this can be closed as fixed by https://github.com/yesodweb/persistent/commit/d7779b0719faa7786983416e9e842a4979d70c6b

nh2 commented 5 years ago

The result of expressions involving strings with embedded NULs is undefined.

So perhaps this should just be fixed in the case of blobs, not text?

Coming via #921. Relevant interpretation (also from @ryantm): https://stackoverflow.com/questions/15230477/is-it-possible-to-save-0-character-in-sqlite-as-a-text#comment41198721_15230944:

I'm not totally sure what that means, but my guess is that it is talking about query expressions sqlite.org/lang_expr.html . When I put \0 into those, it treats it like the end of the query is at that character (usually a syntax error when in the middle of a string). When I use bind (either text or blob), it accepts \0, which is consistent with the sentence previous to the one you quoted:

"If any NUL characters occur at byte offsets less than the value of the fourth parameter then the resulting string value will contain embedded NULs."