Reloading data from MDB?

[bensonml]

I cannot seam to be able to 'reload' data from MDB into Bento-meta.

Use case: using Bento-meta for modeling data in web server. End-user Anna goes to "sts.com" and sees the data that is stored in the MDB for the value set "disease". A few minutes later, end-user Bob goes to "sts.com" and makes an update to the disease value set., adding a term to a value set. When Anna revisits the the value set a few minutes later, she should see the now-updated value set.

When it comes to using "Bento-meta", I cannot find a way to reload data. Any attempts to "reload" result in error: $m->load_all_db_models($bolt_url);. I need to be able to have Bento-meta continually polling MDB very quickly to get any changes.

error:

Can't call method "handle" on an undefined value at /Users/bensonml/0_SRC/bento-meta/perl/lib/Bento/Meta/Model/Edge.pm line 30.

The problem is compounded by the fact that it takes about 100 seconds to run $m->load_all_db_models($bolt_url);, which means that each time a person wants to go to a new page, or look at a different value set, or node and I have to reload the model for each query, it will take 1:40 to load and display each page.

Reloading error example snippet:

   $bolt_url //= 'bolt://54.156.191.24:7687';
    print "Using Bento::Meta version=$Bento::Meta::VERSION\n";

    my $m = Bento::Meta->new;

    $m->load_all_db_models($bolt_url);
    my $icdc = $m->model('ICDC');
    print "---A---\n";

    $m = undef;
    $icdc = undef;
    $m = undef;
    $m = Bento::Meta->new;

    print "---B---\n";
    $m->load_all_db_models($bolt_url);
    print "---C---\n";
    $icdc = $m->model('ICDC');
    print "---D---\n";

    my @icdc__nodes = $icdc->nodes();
    #print Dumper (@icdc__nodes);
    foreach my $n (@icdc__nodes) {
        print "---E---\n";
        #my $n_ = $m->modelnode($n);
        #print Dumper ($n_);
        #print "\t\txxxxxxxx\n\n";
        my $n_handle_ = $n->handle();
        print "\t - node: $n_handle_\n";
        my @n_props = $n->props();
        foreach my $n_prop (@n_props){
            print "---F---\n";
            my $n_p_h = $n_prop->handle();
            print "\t\t-node-property: $n_p_h\n";
        }
        #my $name = $n->handle();
        #print "   found $name\n";
    }

    print "---G---\n";

yields the following error:

❯ perl simple.pl
Unescaped left brace in regex is deprecated here (and will be fatal in Perl 5.32), passed through in regex; marked by <-- HERE in m/^([:$?])|({ <-- HERE [^}]+}$)/ at /Users/bensonml/perl5/perlbrew/perls/perl-5.30.2/lib/site_perl/5.30.2/Neo4j/Cypher/Abstract/Peeler.pm line 542.
Unescaped left brace in regex is deprecated here (and will be fatal in Perl 5.32), passed through in regex; marked by <-- HERE in m/^([:$?])|({ <-- HERE [^}]+}$)/ at /Users/bensonml/perl5/perlbrew/perls/perl-5.30.2/lib/site_perl/5.30.2/Neo4j/Cypher/Abstract/Peeler.pm line 120.
---A---
---B---
Can't call method "handle" on an undefined value at /Users/bensonml/0_SRC/bento-meta/perl/lib/Bento/Meta/Model/Edge.pm line 30.

[bensonml]

If the lines in between print "---A---\n"; and print "---D---\n"; are removed, then it works fine. I've attempted various ways to reload the data, but I can't figure out how to get it to work

[majensen]

@bensonml - ok, I'm going to spend some time on this. There is a bug, almost certainly. Also, I think I can improve the response by more lazily loading. The machinery is there for it, but to load a model Meta still goes out an pulls all nodes, edges and properties - can probably do less, but need to think about it.

I think that kind of polling isn't necessary in the first instance if we think of model changes as release-driven, rather than transactional. However, there is probably a clever way with direct queries to the db to check if there have been updates that is a lot faster than pulling the model again and comparing. That another feature to think about.

[majensen]

@bensonml - ok, there is a cache (%Bento::Meta::Model::ObjectMap::Cache) that, if you clear it (set it to () ), the second load will run. This needs to be automatic, clearly.