IKANOW / Aleph2

The IKANOW v2 meta-database and analytics platform
Apache License 2.0
2 stars 1 forks source link

Empty trigger returns false not true in analytic threads? #92

Closed Alex-Ikanow closed 8 years ago

Alex-Ikanow commented 8 years ago

"trigger_config": {"schedule": "1 min"}

Seems to create a trigger but then never activates it? It's supposed to run every minute

Caleb and Yuri have both seen this

Alex-Ikanow commented 8 years ago

OK first off, with the above config it defaults auto_calc to true, which means that it will generate a trigger for all the inputs, and most of them (eg search_index_service) aren't implemented and hence return false every time

Alex-Ikanow commented 8 years ago

If you set auto_calc: false, it generates a "trigger_type" : "time" trigger, as it should, but it never triggers

The times look right:

#(a few minutes later)
mongos> new Date()
ISODate("2016-05-16T16:19:00.478Z")

vs

        "last_checked" : ISODate("2016-05-16T16:14:11.216Z"),
        "next_check" : ISODate("2016-05-16T16:14:21.216Z"),
Alex-Ikanow commented 8 years ago

If I turn the trigger off, and set "poll_frequency" instead, then it immediately activates the jobs but doesn't create a bucket trigger at all

In that case you get the log

Generated 0 trigger(s) (0 group(s)) for bucket /aleph2_testing/524dc2d2e4b0a78f1e37c083/bucket/es/js/test (1 job(s))

Whereas it's

Generated 1 trigger(s) (1 group(s)) for bucket /aleph2_testing/524dc2d2e4b0a78f1e37c083/bucket/es/js/test (1 job(s))

For the case where the trigger is enabled

Alex-Ikanow commented 8 years ago

So based on the above (and the knowledge that the "input file" trigger does work), it seems very much like the time trigger is broken?

next_check and last_checked are incrementing

{
        "_id" : "6f280a8c-ab11-3774-bba2-e0042c13a9f5",
        "is_job_active" : false,
        "is_bucket_active" : false,
        "is_bucket_suspended" : false,
        "is_pending" : false,
        "last_checked" : ISODate("2016-05-16T16:27:31.215Z"),
        "next_check" : ISODate("2016-05-16T16:27:41.215Z"),
        "bucket_id" : "524dc2d2e4b0a78f1e37c083aleph...bucket.Analytics_bucket_test_.time_sliced_elasticsearch_.;",
        "bucket_name" : "/aleph2_testing/524dc2d2e4b0a78f1e37c083/bucket/es/js/test",
        "input_resource_name_or_id" : "/aleph2_testing/524dc2d2e4b0a78f1e37c083/bucket/es/js/test",
        "last_resource_size" : NumberLong(0),
        "trigger_type" : "time"
}
Alex-Ikanow commented 8 years ago

Ah ok found it .. in getManualOrAutomatedTrigger if auto_calculate is false then it returns info.trigger (ie null in my case), which means that the returned optional is empty, which then means it never checks the trigger

Instead if info.trigger is null and auto_calc is false it should just create the default

trigger: { op: AND, dependency_list: [] }