Support non-ASCII characters in SQLiteConnection.prepare

[GoogleCodeExporter]

If the SQL query includes some non-ascii characters, the prepare method fails 
to translate them to the correct charset for sqlite.

The following test fails:

SQLiteStatement st = cnx.prepare("select '\uD840\uDC40';");
st.step();
Assert.assertEquals("\uD840\uDC40", st.columnString(0));

There is no problem with SQLiteStatement.bind(int, String). The problem seems 
to come from the JNI code (_SQLiteManualJNI.sqlite3_prepare_v2).

I attached an unit test.

Original issue reported on code.google.com by olivier....@free.fr on 20 Aug 2010 at 1:36

Attachments:

• CharsetBugTest.java

[GoogleCodeExporter]

Thanks for reporting the issue. The prepare() and exec() calls do assume that 
SQL passed is in ASCII characters. This should have been documented better 
(and, arguably, the SQL should have been verified for charset) - right now 
there's a note for the com.almworks.sqlite4java package about this and in 
exec() javadoc.

While I'll think about implications of lifting that restriction, is there any 
real-life requirement for that? Do you really need to pass string literals that 
otherwise would be passed through bind() method? Note that such statements are 
not likely to be effectively cached.

Igor

Original comment by ser...@gmail.com on 20 Aug 2010 at 2:20

[GoogleCodeExporter]

In fact, I'm working on a JDBC layer on-top sqlite4java. This issue breaks a 
unit test. I don't really need this issue to be fixed. I'll check the charset 
in my JDBC layer.

Thanks.

Original comment by olivier....@free.fr on 23 Aug 2010 at 12:29

[GoogleCodeExporter]

I've accepted this issue, but it's low priority for now.

(A JDBC layer? Nice to hear you're using sqlite4java, but you know Xerial 
SQLite JDBC driver, don't you? 
[http://www.xerial.org/trac/Xerial/wiki/SQLiteJDBC])

Original comment by ser...@gmail.com on 23 Aug 2010 at 10:19

• Changed title: Support non-ASCII characters in SQLiteConnection.prepare
• Changed state: Accepted
• Added labels: Priority-Low, Type-Enhancement
• Removed labels: Priority-Medium, Type-Defect

[GoogleCodeExporter]

This is for my work. We already use the Xerial driver but with issues (major is 
no concurrency). So I rewrite one.

As a personal project I also create some kind of persistence layer onto of 
sqlite4java (http://code.google.com/p/tyco/source/list?repo=s4j-persister but 
at a very early stage). So, you're project is useful for me ;). 

Original comment by olivier....@free.fr on 23 Aug 2010 at 10:26

[GoogleCodeExporter]

Original comment by evj...@almworks.com on 12 Jul 2014 at 2:20

• Changed state: Fixed