jsnoble / tlucene-evaluator

MIT License
0 stars 0 forks source link

Some notes #1

Open kstaken opened 6 years ago

kstaken commented 6 years ago

I did a little experimenting with this. It's a great start! A few notes.

  1. regex
    • The regex syntax in ES is delimitated by slashes which should make it easier to handle. field:/.*/ also I believe it may allow something like field:partial/blah.*/ but I'm not 100% sure on that.
  2. wildcards
    • This needs to handle general wildcards * and ?. Those can appear anywhere in a query and may repeat multiple times.
  3. unquoted dates
    • In ES date field values do not have to be quoted. date:[2018-10-10T17:36:13Z TO 2018-10-10T17:36:13Z] is accepted as well as date:["2018-10-10T17:36:13Z" TO "2018-10-10T17:36:13Z"]
  4. I pulled one query form based on real queries and tried to test it and it wasn't able to parse it even if I quoted the dates. date:["2018-10-10T17:36:13Z" TO "2018-10-10T17:36:13Z"] AND NOT value:(251 OR 252) AND NOT type:example I confirmed ES can evaluate the query.
return ((1539192973000 >= data.date) && (data.date >= 1539192973000)) && (data.<implicit> == "251" || data.<implicit> == "252" && (data.type == "txt")

SyntaxError: Unexpected token <
    at new Function (<anonymous>)
    at DocumentMatcher._buildFilterFn (/Users/kstaken/projects/testing/tlucene-evaluator/lib/document-matcher/index.js:94:35)
    at DocumentMatcher.parse (/Users/kstaken/projects/testing/tlucene-evaluator/lib/document-matcher/index.js:30:14)
    at Object.<anonymous> (/Users/kstaken/projects/testing/tlucene-evaluator/test.js:9:17)
    at Module._compile (module.js:652:30)
    at Object.Module._extensions..js (module.js:663:10)
    at Module.load (module.js:565:32)
    at tryModuleLoad (module.js:505:12)
    at Function.Module._load (module.js:497:3)
    at Function.Module.runMain (module.js:693:10)

Also let's go ahead and get this moved into a project under terascope. Please set the project up with all necessary pieces.

kstaken commented 6 years ago

The project under Terascope should be named xlucene-evaluator in the terascope namespace.