fix: don't bind atoms that are prefixed with `nil` as NULL

mhanberg commented 1 year ago

This patch changes the comparison from utf8ncmp to utf8cmp in order to explicitly match on the size of the atom. I noticed a case of inserting an atom of :nil_safe? that was cast to NULL.

I wasn't sure the correct level to add a test, so please let me know the best place and I can get it added.

warmwaffles commented 1 year ago

Oh what an interesting problem. I wanted to cap the check of two raw strings, I completely wiffed on this and should have realized that of course :nil_safe? compared to :nil with 3 would always equal 0.

ruslandoga commented 11 months ago

I think it might make sense to get rid of utf8 functions completely as both SQLite and NIF functions expect just the count of bytes, not utf8 codepoints. For example, getting columns for a table like

sqlite> create table 👋(✍️ text, \U+1FAF1 integer) strict;
sqlite> select * from sqlite_master;
table|👋|👋|2|CREATE TABLE 👋(✍️ text, 🫱 integer) strict
sqlite> .mode box
sqlite> insert into 👋 values ('\U+1FAF1', 10);
sqlite> select * from 👋;
┌────┬────┐
│ ✍️ │ 🫱  │
├────┼────┤
│ 🫱  │ 10 │
└────┴────┘

results in truncated binaries returned in exqlite

iex(1)> {:ok, db} = Exqlite.Sqlite3.open("demo.db")
{:ok, #Reference<0.3292708811.1647181846.21903>}
iex(2)> Exqlite.Sqlite3.prepare(db, "select * from 👋")
{:ok, #Reference<0.3292708811.1647181846.21905>}
iex(3)> {:ok, stmt} = v
{:ok, #Reference<0.3292708811.1647181846.21905>}
iex(4)> Exqlite.Sqlite3.columns(db, stmt)
{:ok, [<<226, 156>>, <<240>>]}
iex(5)> "✍️"
"✍️"
iex(6)> "✍️" <> <<0>>
<<226, 156, 141, 239, 184, 143, 0>>
iex(7)> "🫱" <> <<0>>
<<240, 159, 171, 177, 0>>

It also affects error messages:

iex(8)> Exqlite.Sqlite3.prepare(db, "select * from 🥲")
{:error,
 <<110, 111, 32, 115, 117, 99, 104, 32, 116, 97, 98, 108, 101, 58, 32, 240>>}

It doesn't seem to affect anything else.

elixir-sqlite / exqlite

fix: don't bind atoms that are prefixed with `nil` as NULL #258