typhon-project / typhonql

Typhon Query Language
Eclipse Public License 2.0
4 stars 1 forks source link

[BUG] NLP queries are not scheduled for the NLP engine #115

Closed DavyLandman closed 3 years ago

DavyLandman commented 3 years ago

Describe the bug

Queries on subfield free-text attributes are not scheduled on the NLP engine

Query

from RawTextWarnings u
select u.ew.NamedEntityRecognition.WordToken
where u.ew.NamedEntityRecognition.NamedEntity == "LOCATION"

gets compiled to:

script([
    step(
      "DocumentDB",
      mongo(find("DocumentDB","RawTextWarnings","{\"NLP___\": \"${rawTextWarnings___NLP_0__u_@id_0}\"}","{\"_id\": 1}")),
      ("rawTextWarnings___NLP_0__u_@id_0":field("RawTextWarnings","rawTextWarnings___NLP_0__u","RawTextWarnings___NLP","@id")),
      signature=[]),
    read([<"RawTextWarnings","rawTextWarnings___NLP_0__u","RawTextWarnings___NLP",["ew$NamedEntityRecognition$WordToken"]>])
  ])

Model

entity RawTextWarnings {
    //timeStamp : datetime
    //warning : text
    ew : freetext[NamedEntityRecognition[atb_weather_ner]] 
    pw -> ProcessedWarnings[1]
} 

Expected behavior

both the where clause and the result clause should go to the NLP engine

DavyLandman commented 3 years ago
NORMALIZED: from RawTextWarnings u, RawTextWarnings___NLP rawTextWarnings___NLP_0__u select rawTextWarnings___NLP_0__u.ew$NamedEntityRecognition$WordToken where rawTextWarnings___NLP_0__u.ew$NamedEntityRecognition$NamedEntity == "LOCATION", rawTextWarnings___NLP_0__u.@id == u.NLP___
COMPILING2Mongo: from RawTextWarnings u, #dynamic(RawTextWarnings___NLP rawTextWarnings___NLP_0__u) select #done(rawTextWarnings___NLP_0__u.ew$NamedEntityRecognition$WordToken) where #done(rawTextWarnings___NLP_0__u.ew$NamedEntityRecognition$NamedEntity == "LOCATION"), rawTextWarnings___NLP_0__u.@id == u.NLP___
tvdstorm commented 3 years ago

Should be fixed in 0435cc7e8293c504a58e28e60aa546d6c6863974