Switching databases can crash new iOS VM

[GoogleCodeExporter]

After the latest GC fixes my app has started to crash when opening a database. 
This seems to be a regression of some kind as I was able to get a lot further 
in my app before the changes discussed in issue 1339.

My app creates a database, closes it, reopens it and displays the contents in a 
list. My app is now consistently crashing after the load. My app also lets 
users switch between databases, if you select a database, I close the old one 
and open the new one. Again it crashes immediately. This is all new.

I have tried to create a test app to reproduce this. It is very hard to do. 
These GC issues are not related to specific functions, it all seems dependent 
on timing and what else is going on in the app. Trying to extract the key code 
from a much larger app, changes all the timings. I have created a test app that 
crashes on my Ipod 5G, my iPad Mini and when running the generated source code 
in Xcode. I am not 100% certain it is the same bug, but the device crash logs 
are very similar and in both cases are in the same CodenameOne function:

JAVA_VOID java_lang_String_releaseNSString___long(CODENAME_ONE_THREAD_STATE, 
JAVA_LONG ns) {
    if(ns != 0) {
        // this prevents a race condition where the string might get GC'd and the NSString is still pending
        // on a call in the native thread
        dispatch_async(dispatch_get_main_queue(), ^{
            NSString* n = (NSString*)ns;
            [n release];  (EXC_BAD_ACCESS) here
        });
    }
}

Hopefully if you can fix the crash in this test app, it will fix the crash in 
my app too. If not would it be useful to send you the build generated source 
code for the full app? As it crashes in Xcode, you could build it there. Let me 
know if you want me to do that.

In order to run this app you will need to set the property 
ios.fileSharingEnabled to true and you will need to copy the sample data file 
to the app on your device using iTunes.

The code and test file are attached.

Build and run the app. Hit Start, then Load. This is the database load which 
you have run a million times. When it completes the button changes to Open, hit 
Open, the database is queried, this will take very little time, then the 
message Database Ready is displayed.

In my real app, it does not make it this far, it crashes right away. The test 
app does not, but give it time. If you are running on a device it may take up 
to 30 seconds to crash, if running in Xcode it normally crashes faster. After 
Database Ready appears just let it sit there until it crashes.

Original issue reported on code.google.com by mwarn...@readerware.com on 14 Feb 2015 at 12:24

Attachments:

• TWMain.java
• OpenForm.java
• Music.aw3.data

[GoogleCodeExporter]

We've just made an improvement to the GC, can you check if this is still 
happening?

Original comment by shai.almog on 15 Feb 2015 at 10:02

[GoogleCodeExporter]

It's worse.

When I first reported this problem my full app would start and load an existing 
database. The problem I had was with switching databases. I tried to change the 
database by importing a new one, the import would work but the open would 
crash. The other scenario was that I started my app, it would load the 
database. Then I tried to open another existing database and it would crash on 
open.

Now my app won't run at all, it crashes loading the initial database at startup.

The test program I provided continues to crash as described above.

Original comment by mwarn...@readerware.com on 15 Feb 2015 at 10:27

[GoogleCodeExporter]

More on the new crash I am getting at startup.

It is still a EXC_BAD_ACCESS but it is at a different location in your code.

When I say I am having trouble opening the database, it is more than just the 
Database.open() call. I am opening the database, doing a SQL query and reading 
the results. I am doing this on a background thread. Once I have the results I 
go back to the EDT and use them, the background thread dies. In the earlier 
crashes they occurred shortly after a message displayed on the Xcode console 
that the background thread had died. The database was open and loaded before 
the crash occurred.

Running my full app under Xcode, the latest crash happens on the background 
thread at:

#ifdef NEW_CODENAME_ONE_VM
JAVA_OBJECT 
com_codename1_impl_ios_IOSNative_sqlCursorValueAtColumnString___long_int_R_java_
lang_String(CN1_THREAD_STATE_MULTI_ARG JAVA_OBJECT instanceObject, JAVA_LONG 
statement, JAVA_INT col) {
    const char* result = (const char*)sqlite3_column_text((sqlite3_stmt*)statement, col);
    if(result == 0) {
        return JAVA_NULL;
    }
    JAVA_OBJECT str = __NEW_INSTANCE_java_lang_String(threadStateData);
    struct obj__java_lang_String* ss = (struct obj__java_lang_String*)str;
    NSString* ns = [NSString stringWithUTF8String:result];
    ss->java_lang_String_nsString = (JAVA_LONG)ns;

    JAVA_OBJECT destArr = __NEW_ARRAY_JAVA_CHAR(threadStateData, [ns length]);
    ss->java_lang_String_value = destArr;
    ss->java_lang_String_count = [ns length];  (CRASH HERE EXC_BAD_ACCESS)

    __block JAVA_ARRAY_CHAR* dest = (JAVA_ARRAY_CHAR*)((JAVA_ARRAY)destArr)->data;
    __block int length = 0;
    [ns enumerateSubstringsInRange:NSMakeRange(0, [ns length])
                           options:NSStringEnumerationByComposedCharacterSequences
                        usingBlock:^(NSString *substring, NSRange substringRange, NSRange enclosingRange, BOOL *stop) {
                            dest[length] = (JAVA_CHAR)[ns characterAtIndex:length];
                            length++;
                        }];

    return str;
}

Original comment by mwarn...@readerware.com on 15 Feb 2015 at 11:09

[GoogleCodeExporter]

We made another major change to GC behavior today. Can you see if this is still 
happening?

Original comment by shai.almog on 17 Feb 2015 at 7:46

[GoogleCodeExporter]

The new problem is resolved. My full app will now start and open an existing 
database. sqlCursorValueAtColumnString is working again.

However the original problem I reported remains. I cannot open a new database, 
that crashes my full app.

The test app I provided is still crashing but the scenario is a little 
different. I originally reported that my full app would run the import and then 
crash when I tried to open the database. The test app would run the import and 
apparently open the database, but it would crash later if you let it sit idle, 
Now it is behaving just like the full app, run the import and it will complete, 
run the open and it will crash. Hopefully that will make it easier to reproduce 
and get to the bottom of.

Original comment by mwarn...@readerware.com on 17 Feb 2015 at 3:53

[GoogleCodeExporter]

The crash is in the String release code you mentioned. On the string "Boys, The 
Beach" I'm guessing this came from the CSV parsing?
Where could that string have been created?

Original comment by shai.almog on 17 Feb 2015 at 8:44

• Changed state: Accepted

[GoogleCodeExporter]

I am not sure that it came from the CSVParser, it could have but it could also 
have come from the database.

We do call CSVParser. 

If you look in OpenForm.java, there are two threads:

LoadThread - This reads the file, calls CSVParser for each line in the file to 
parse it and then stores the resulting columns in the database. "Boys, The 
Beach" is probably the artist column, the formatting is my problem :-) You 
would see that string as one of the columns in the CSV file and it would have 
been inserted into the database.

Look for:

                // Loop through rows and import
                while ((row=readLine(in, "UTF-8")) != null) {

The parsing and database insert is done in that loop.

OpenThread - This opens the database and issues a SQL query to return the 
results.

Look for:
                while (cursor.next() == true) {
                    Row dbRow = cursor.getRow();
                    String title = dbRow.getString(0);
                    String artist = dbRow.getString(1);
                    long imageKey = dbRow.getLong(0);
                    getImageData(imageKey);
                    rowCount++;
                }

You can see in that loop that we get the artist column from the database, the 
text you saw could have been in that string too.

Original comment by mwarn...@readerware.com on 17 Feb 2015 at 9:03

[GoogleCodeExporter]

This issue was closed by revision r2143.

Original comment by shai.almog on 18 Feb 2015 at 7:31

• Changed state: Fixed

[GoogleCodeExporter]

Turns out this was as trivial as a missing retain on an NSString.

Original comment by shai.almog on 18 Feb 2015 at 8:13