Slowdown in Bayeslite or Crosscat

[tibbetts]

I’m seeing 0.1.4 being notably slower on my laptop than 0.1.3.rc2. Specifically running the following queries:

%time q('''simulate Period_Minutes from satellites_cc GIVEN "Users" = 'Military' limit 1000''');

takes 539ms on 0.1.3rc2 and 21s on 0.1.4 in unscientific testing. The underlying crosscat versions are 0.1.38 and 0.1.49 respectively.

[tibbetts]

Profile run with:

import cProfile
cProfile.run("""q('simulate Period_Minutes from satellites_cc GIVEN "Users" = Military limit 1000')""")

bayeslite-0.1.4-profile.txt bayeslite-0.1.3rc2-profile.txt

[riastradh-probcomp]

All the time seems to be spend inside cursor.execute(), in sqlite3/apsw.

A few wild hypotheses to investigate:

• apsw's statement cache is problematic -- to large or too small. We could try passing statementcachesize=0 or statementcachesize=1000 to apsw.Connection(...).
• apsw is being compiled with -O0 instead of -O2 for some reason.
• apsw is acquiring locks or doing other interprocessor synchronization that the Python sqlite3 module doesn't.