teragrep / pth_10

Data Processing Language (DPL) translator for Apache Spark
GNU Affero General Public License v3.0
0 stars 6 forks source link

problem with eval parsing / processing #37

Open kortemik opened 1 year ago

kortemik commented 1 year ago

ID <384> Reporter <@kortemik>

this is propably a parser issue however adding here too so we get a test case for it

%dpl

| makeresults count=1 | eval _raw="[17/Aug/2023:08:03:55.441546368 +0300] conn=917818 op=5 EXT oid=\"2.16.840.1.113730.3.5.12\" name=\"replication-multimaster-extop\""

gives

java.lang.IllegalStateException: failed to parse at line 3:34 due to token recognition error at: '"[17/Aug/2023:08:03:55.441546368 +0300] conn='

however escaping the equal-signs (=) does help:

%dpl

| makeresults count=1 
| eval _raw="[17/Aug/2023:08:03:55.441546368 +0300] conn\=917818 op\=5 EXT oid\=\"2.16.840.1.113730.3.5.12\" name\=\"replication-multimaster-extop\""
| search "oid=\"2.16.840.1.113730.3.5.12\""

following is returned

_time   _raw
2023-08-21T15:13:54.000+03:00   [17/Aug/2023:08:03:55.441546368 +0300] conn=917818 op=5 EXT oid="2.16.840.1.113730.3.5.12" name="replication-multimaster-extop"

please notice that this is somehow related to #383

kortemik commented 1 year ago

User <@kortemik> Date <[2023-08-21T12:16:47Z]>

default mode is used for search: https://github.com/teragrep/pth_03/blob/main/src/main/antlr4/com/teragrep/pth_03/antlr/DPLLexer.g4#L385

search uses the same mode as initial processing: https://github.com/teragrep/pth_03/blob/main/src/main/antlr4/imports/DPLParserTransform_search.g4#L48-L50

which is at https://github.com/teragrep/pth_03/blob/main/src/main/antlr4/com/teragrep/pth_03/antlr/DPLParser.g4#L60

therefore behaviour experienced #383 should not be possible, however it occurs.

please check if this has something to do with the implementation too.

51-code commented 2 months ago

The query still gives out the same exception. The workaround doesn't give out any exception message but also doesn't find the data row correctly.

51-code commented 2 months ago

Will be looked at in #295

Abigael-JT commented 1 month ago

Has disabled tests in EarliestLatestTest.java : searchIssue384Test and searchIssue384_2Test