threatgrid / naga

Datalog based rules engine
Eclipse Public License 1.0
379 stars 19 forks source link

Some rules cause other rules to not get run #91

Closed quoll closed 4 years ago

quoll commented 6 years ago

Rules with multiple generation output can cause problems in execution.

The below test data creates an execution path where all dependencies are not being followed:

Data

  [{:db/id :api-server
    ::type :box/app
    :app/min-count 3
    :app/max-count 5
    :map/can-talk-to [:postgres-server]
    :foo [{:a 1} {:b 2}]}
   {:db/id :foo-bar
    ::type :box/app
    :app/max-count 5}
   {:db/id :postgres-server
    ::type :box/db
    :engine :postgres}
   {:db/id :something-else
    ::type :box/db
    :engine :postgres}]

Rules


               [?a ::can-talk-to ?b]
               :-
               [?a :map/can-talk-to ?x]
               [?x :naga/contains ?b])
            ;; Infer ASG from App Box

            (r "ASG gen"
               [?asg ::type :asg]
               [?asg ::created-for ?abox]
               [?asg :tf/id ?id]
               :-
               [?abox ::type :box/app]
               [?abox :db/id ?id])

            (r "MinCount"
               [?asg :min-count ?min-count]
               :-
               [?abox ::type :box/app]
               [?asg ::type :asg]
               [?asg ::created-for ?abox]
               [?abox :app/min-count ?min-count])

            (r "MaxCount"
               [?asg :max-count ?max-count]
               :-
               [?abox ::type :box/app]
               [?asg ::type :asg]
               [?asg ::created-for ?abox]
               [?abox :app/max-count ?max-count])

            ;; Database Box
            (r "DB Box"
               [?dbi ::type :dbi]
               [?dbi :tf/id ?id]
               [?dbi ::created-for ?dbox]
               :-
               [?dbox ::type :box/db]
               [?dbox :db/id ?id])

            (r "Engine"
               [?dbi :engine ?engine]
               :-
               [?dbox ::type :box/db]
               [?dbi ::created-for ?dbox]
               [?dbi ::type :dbi]
               [?dbox :engine ?engine])

            ;; Communication paths
            (r "CommunicationPaths"
               [?sg ::type :sg]
               [?sg :ingress ?ingress]
               [?ingress :ingress/communication ?comm-igs]
               [?comm-igs :naga/first ?comm]
               [?comm-igs :naga/contains ?comm]
               :-
               [?abox ::can-talk-to ?dbox]

               [?abox ::type :box/app]
               [?dbox ::type :box/db]

               [?asg ::created-for ?abox]
               [?asg ::type :asg]

               [?dbi ::created-for ?dbox]
               [?dbi ::type :dbi]

               [?comm :communication/speaker ?dbi]
               [?comm :communication/id "primary"])

            ;; Postgres Database instance communication port
            (r "PostgresComm"
               [?comm ::type ::communication]
               [?comm :communication/id "primary"]
               [?comm :port 5432]
               [?comm :protocol :tcp]
               [?comm :communication/speaker ?dbi]
               :-
               [?dbi ::type :dbi]
               [?dbi :engine :postgres])

            (r "security group ingress/communications to actual ingress"
               [?ingress :ingress ?ig]
               [?igs :naga/first ?ig]
               [?igs :naga/contains ?ig]

               [?ig :from-port ?port]
               [?ig :to-port ?port]
               [?ig :protocol ?protocol]
               [?ig :ingress/security-group ?speaker]
               :-
               [?ingress :ingress/communication ?comms]
               [?comms :naga/contains ?comm]
               [?comm :port ?port]
               [?comm :protocol ?protocol]
               [?comm :communication/speaker ?speaker])])```

In this case, the `CommunicationPaths` rule is run, but there is nothing for it to generate. However, removing the `ASG gen` rule causes a data cascade that correctly creates data from the `CommunicationPaths` rule.
quoll commented 6 years ago

The bug is in the compiler.

Rules with multiple productions (creating an entity), are not detecting downstream rules correctly. The multi-production rules currently have an empty :downstream list.

quoll commented 5 years ago

Now that regeneration is no longer an issue, check if these rules generate downstream rules correctly

SevereOverfl0w commented 5 years ago

I re-attempted to use the program I wrote that exposed this bug, and my comment about not returning anything was still true.

SevereOverfl0w commented 5 years ago

Actually, that's not right. My tests seem to indicate it is consistently returning results now. Perhaps yesterday I had an old jvm running with the old naga.

I'm not confident I've tested this properly :smile: But it seems good from a cursory test.

quoll commented 4 years ago

Ooops. I should have closed this long ago!