ResultsAnalyzer could not parse request

[taylorThunderbolt]

Description

Hi, I start restler in the fuzz-lean mode. The command is as follows:

..\restler_bin\restler\Restler.exe fuzz-lean --grammar_file Compile\grammar.py --dictionary_file Compile\dict.json --settings Compile\engine_settings.json --token_refresh_interval 30 --token_refresh_command "python3 get_token.py" --no_ssl

After a period of time, the ResponseBuckets directory is created, but it is empty.There are a bunch of lines in the ResultsAnalyzerStdErr file like:

Warning: Warning: could not parse request in line 9585: GET /api/alarms/rules?pageNo=1&pageSize=1&sortBy=host&orderBy=host&filter=host&search==B?G|N=g2DVP@n+VmDbJ0aVq}4+kF?v&&\r_^2sqh)r"s5"wR^69Tl HTTP/1.1\r\nAccept: application/json\r\nHost: 192.168.150.137:9060\r\n_OMITTED_AUTH_TOKEN_\r\nContent-Length: 0\r\nUser-Agent: restler/9.0.1\r\n\r\n
Unhandled exception. System.Exception: Unexpected response without prior request at 2022/9/20 15:29:17: { version = "HTTP/1.1"
  statusCode = 400
  statusDescription = "Bad Request"
  headers =
           map
             [("Connection", "close");
              ("Content-Type", "text/plain; charset=utf-8")]
  body = "400 Bad Request"
  str = None }
   at Microsoft.FSharp.Core.PrintfModule.PrintFormatToStringThenFail@1639.Invoke(String message) in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\printf.fs:line 1639
   at Microsoft.FSharp.Collections.Internal.IEnumerator.choose@171.System-Collections-IEnumerator-MoveNext() in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\seq.fs:line 179
   at Microsoft.FSharp.Collections.Internal.IEnumerator.map@75.DoMoveNext(b& curr) in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\seq.fs:line 77
   at Microsoft.FSharp.Collections.Internal.IEnumerator.MapEnumerator`1.System-Collections-IEnumerator-MoveNext() in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\seq.fs:line 64
   at Microsoft.FSharp.Core.CompilerServices.RuntimeHelpers.takeInner@266[T,TResult](ConcatEnumerator`2 x, Unit unitVar0) in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\seqcore.fs:line 268
   at Microsoft.FSharp.Collections.Internal.IEnumerator.map@75.DoMoveNext(b& curr) in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\seq.fs:line 77
   at Microsoft.FSharp.Collections.Internal.IEnumerator.MapEnumerator`1.System-Collections-IEnumerator-MoveNext() in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\seq.fs:line 64
   at Microsoft.FSharp.Collections.Internal.IEnumerator.map@75.DoMoveNext(b& curr) in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\seq.fs:line 77
   at Microsoft.FSharp.Collections.Internal.IEnumerator.MapEnumerator`1.System-Collections-IEnumerator-MoveNext() in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\seq.fs:line 64
   at Microsoft.FSharp.Core.CompilerServices.RuntimeHelpers.takeInner@266[T,TResult](ConcatEnumerator`2 x, Unit unitVar0) in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\seqcore.fs:line 268
   at Microsoft.FSharp.Collections.Internal.IEnumerator.next@193[T](FSharpFunc`2 f, IEnumerator`1 e, FSharpRef`1 started, Unit unitVar0) in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\seq.fs:line 194
   at Microsoft.FSharp.Collections.Internal.IEnumerator.filter@188.System-Collections-IEnumerator-MoveNext() in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\seq.fs:line 196
   at Microsoft.FSharp.Collections.SeqModule.oneStepTo@987[T](IEnumerable`1 source, List`1 prefix, FSharpRef`1 enumeratorR, Int32 i) in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\seq.fs:line 996
   at Microsoft.FSharp.Collections.SeqModule.action@4164-1[T](IEnumerable`1 source, List`1 prefix, FSharpRef`1 enumeratorR, Int32 i, Unit unitVar0) in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\seq.fs:line 1011
   at Microsoft.FSharp.Collections.SeqModule.result@1003.Invoke(Int32 i)
   at Microsoft.FSharp.Collections.Internal.IEnumerator.unfold@205.DoMoveNext(b& curr) in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\seq.fs:line 207
   at Microsoft.FSharp.Collections.Internal.IEnumerator.MapEnumerator`1.System-Collections-IEnumerator-MoveNext() in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\seq.fs:line 64
   at Microsoft.FSharp.Collections.SeqModule.Iterate[T](FSharpFunc`2 action, IEnumerable`1 source) in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\seq.fs:line 496
   at Microsoft.FSharp.Collections.SeqModule.groupByValueType[T,TKey](FSharpFunc`2 keyf, IEnumerable`1 seq) in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\seq.fs:line 1068
   at Microsoft.FSharp.Collections.SeqModule.GroupBy@1076.Invoke(Unit unitVar0) in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\seq.fs:line 1076
   at Microsoft.FSharp.Collections.SeqModule.mkDelayedSeq@471.Invoke(Unit unitVar0) in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\seq.fs:line 471
   at Restler.ResultsAnalyzer.Analyze.Buckets.getBuckets@98.GenerateNext(IEnumerable`1& next) in F:\toys\Code\Python\restler-fuzzer\src\ResultsAnalyzer\Analyze\Buckets.fs:line 98
   at Microsoft.FSharp.Core.CompilerServices.GeneratedSequenceBase`1.MoveNextImpl() in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\seqcore.fs:line 371
   at Microsoft.FSharp.Core.CompilerServices.GeneratedSequenceBase`1.System-Collections-IEnumerator-MoveNext() in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\seqcore.fs:line 403
   at Microsoft.FSharp.Collections.SeqModule.ToList[T](IEnumerable`1 source)
   at Restler.ResultsAnalyzer.Analyze.Main.main(AnalyzeArgs args) in F:\toys\Code\Python\restler-fuzzer\src\ResultsAnalyzer\Analyze\Main.fs:line 86
   at Restler.ResultsAnalyzer.Program.main(String[] argv) in F:\toys\Code\Python\restler-fuzzer\src\ResultsAnalyzer\Program.fs:line 223

What does it mean?

[marina-p]

Hello @taylorThunderbolt,

Results Analyzer is a post-processing step that parses the network logs and bucketizes the errors - its primary use case is to help diagnose configuration issues and improve coverage. Since this step runs after fuzzing finishes, it does not affect the bugs reported by RESTler. To work around this issue, you can disable results analyzer with the --no_results_analyzer flag after the RESTler task (e.g. restler test --no_results_analyzer).

Thanks,

Marina

[marina-p]

This is a known issue that should be fixed soon (also reported in #465).

[taylorThunderbolt]

Your reply helps me a lot, thank u ^_^

[tandrew-test]

I am seeing the warning shown at the top. I've had to write my own parser to ensure that I do not miss anything.

Here are some sample urls that appear to be causing issues when in the log:

• /v2/widget/_Y Ve]\t!_91}\tA}KO\r
• /v2/widget/3>1-Q"Vg!e*Az[~bHG &E
• /v2/widget/B /^(t(O~Nh&g8\n+Fh,*Ji`
• /v2/widget/o 0x<JOyY<O&TS$Kc\nApDc;4
• /v2/widget/XG+7k_o0 7*2x2xIVg;u5M|8}!HM?Cs9
• /v2/widget/cPGnjeR$B55iIeR&qvI))(p?:~JKyA )7f+
• /v2/widget/5{5W}JqtMG:|\x0c52<?kzrT"6ZW\nT(\t95y-WzH)C\tuwf#ki> dY8

I do not have any trouble parsing the logs using Python so it's something to do with the F# implementation.