Use of Nil in numeric context in DB::SQLite::Statement line 44

[Tyil]

I'm using DB::SQLite in a project, and one particular statement I'm trying to execute is throwing this error.

Use of Nil in numeric context
  in block  at /home/tyil/.rakudo-star/rakudo-star-2019.03/install/share/perl6/site/sources/A96187CD3259EABBAF49C3BE8984779C46DABD52 (DB::SQLite::Statement) line 44
No such method 'hashes' for invocant of type 'Slip'. Did you mean 'hash'?
  in method get-next-for-project at /home/tyil/projects/personal/gtd-api/lib/App/GTD/Models/Sqlite/Thing.pm6 (App::GTD::Models::Sqlite::Thing) line 152
  in method next at /home/tyil/projects/personal/gtd-api/lib/App/GTD/Models/Sqlite/Project.pm6 (App::GTD::Models::Sqlite::Project) line 84
  in sub MAIN at /home/tyil/projects/personal/gtd-api/lib/App/GTD/Bin/ListProjects.pm6 (App::GTD::Bin::ListProjects) line 30
  in block <unit> at bin/gtd line 11

Looking at the source, it looks like .step is being set to Nil somewhere, though I don't have an idea how that could happen.

[CurtTilmes]

I've played around with this a bit. Can you replicate in a small test case? Barring that, what is the easiest way to reproduce if I download App::GTD?

.step can return Nil if the statement is NULL, but you call .prepare just before calling execute, which should have thrown an exception if it failed, so I'm not sure how that could happen.

Perhaps add a diagnostic assertion that $stmt is set just before calling the execute?

One possibility is a double .finish called somewhere -- you might try removing the :finish to see what effect that might have.

[Tyil]

I've tried the simplest thing first, removing :finish from all the execute calls. This did not affect the issue.

The simplest way to reproduce using App::GTD would be to clone and configure it. However, this may be a bit tedious to request of you, so I'll look for a simpler reproduction path over the course of today. If you want to go ahead and try it out already, these steps should get you there.

cd -- $(mktemp -d)
git clone https://gitlab.com/tyil/perl6-app-gtd .
zef install --deps-only .

If you don't have sqitch, just apply the two SQL files found in sql/sqlite/deploy, otherwise you should be able to use it as follows:

cd sql/sqlite
$EDITOR sqitch.conf
sqitch deploy

Wherever the database ends up needs to be configured in $XDG_CONFIG_HOME/gtd/config.toml. $XDG_CONFIG_HOME defaults to $HOME/.config. Its contents look as follows.

[database]
type = "sqlite"
path = "/home/user/.local/share/gtd/database.sqlite"

Now, the application should be finally usable.

perl6 -Ilib bin/gtd project add Foo
perl6 -Ilib bin/gtd project

[Tyil]

I haven't been able to produce this in a smaller test case just yet, sorry for that. I'll keep trying for a bit more, though.

[Tyil]

I still can't seem to reproduce this in a smaller test case from scratch. I also have little idea on how to debug DB::SQLite myself. If you can give me some pointers on how to debug DB::SQLite issues I can try to get started on that with the App::GTD code I have right now.

[CurtTilmes]

I've installed enough to do the "add Foo" and it inserts a line in the things table. It will be difficult to debug without something to trigger the failure.

Do you have a database.sqlite with enough in it to make it fail repeatably for me?

For debugging, I'd just clone the git repo for perl6-dbsqlite (perhaps perl6-db too if the problem is there) and point PERL6LIB at them and start sticking more diagnostic prints in the code to verify that the args match what is expected. I don't have much tracing built in to the library that you could just turn on/off. (sorry -- patches welcome)

SQLite itself has some tracing capabilities, but they aren't implemented in the Perl 6 library (yet?)

[Tyil]

Most actions indeed work fine, but gtd project (which is the only action right now that uses two queries) is generating the error reported in the issue itself. The following reproduction path should work using an empty database:

• gtd project add foobar
• gtd add foo
• gtd project

I'll clone perl6-dbsqlite and perl6-db tomorrow and see if I can do some troubleshooting myself in order to get further with this issue.

[CurtTilmes]

From an empty database, I executed these:

 perl6 -Ilib bin/gtd project add foobar
 perl6 -Ilib bin/gtd add foo
 perl6 -Ilib bin/gtd project

None had any output.

The database is left like this:

sqlite> select * from projects;
sqlite> select * from things;
1|project add foobar||inbox||2019-09-18T09:00:21.521099-04:00|2019-09-18T09:00:21.524198-04:00
2|add foo||inbox||2019-09-18T09:00:33.400477-04:00|2019-09-18T09:00:33.403515-04:00
3|project||inbox||2019-09-18T09:00:40.127191-04:00|2019-09-18T09:00:40.130354-04:00
sqlite> 

For reference:

SQLite version 3.11.0 2016-02-15 17:29:24
Enter ".help" for usage hints.
sqlite> .version
SQLite 3.11.0 2016-02-15 17:29:24 3d862f207e3adc00f78066799ac5a8c282430a5f

perl6 -v
This is Rakudo version 2019.07.1 built on MoarVM version 2019.07.1
implementing Perl 6.d.

[CurtTilmes]

I'm slightly concerned I may have changed DB::SQLite without doing a re-release -- you might try installing directly from a git clone just in case.

[Tyil]

I still encountered the problem when I was using DB::SQLite included from the cloned repository, but I can re-test it after installing DB::SQLite from the repository, and then running my program if desired.

Can you confirm if you're using the projects branch of App::GTD, and line 34 in lib/App/GTD/Bin/ListProjects.pm6 is using $project.next.elems, instead of a hard-coded 0 that I put there to temporarily work around the issue I was seeing?

[CurtTilmes]

Bingo. I was not -- I was using master. I checked out projects and now I get the error. I'll take closer look.

[Tyil]

Reproducability is a big step, thanks for bearing with me so far!

[CurtTilmes]

A Nil is returned by a Nativecall routine declared to return an int32. I didn't even know that was possible.

[CurtTilmes]

I've gotten it down to this:

use App::GTD::DB;

constant DB = App::GTD::DB;

multi sub MAIN(Bool $first, Bool $second)
{
    if ($first)
    {
        say DB.connection('sqlite')
            .prepare('select * from projects')
            .execute(:finish).hashes;
    }

    if ($second)
    {
        say DB.model('Thing').get-next-for-project(1);
    }
    else
    {
        my $stmt = DB.connection('sqlite').prepare(q:to/STMT/);
        SELECT *
        FROM things
        WHERE state = ?
        AND project_id = ?
        ORDER BY created_at ASC
        STMT
        say $stmt.execute('next',1,:finish).hashes;
    }
}

If I disable the first SQL query (False False or False True) it always works.

If I enable the first SQL query, but run the second query with DB.connection it works fine (True False), but If I run the second query with DB.model('Thing').get-next-for-project (True True) it fails with the error.

It seems to be related to the require loading the model. If you load the model, the first query after loading the model with require works, but if you require it again, the next query fails. Edit: not true, each model (Project/Thing) only calls DB.connection to load SQLite so I don't really get the difference.

On load, DB::SQLite uses NativeLibs to do some magic I don't understand, it may or may not be related to that.

[Tyil]

so I don't really get the difference.

Neither do I, but I also don't have the best grasp of Perl 6 internals, let alone NativeCall structs and behaviours. I wonder if I can get in touch with the author of NativeLibs, and see if he has any ideas. Otherwise I'll have to rewrite my code to work differently, which would feel like defeat.