search

microsoft / restler-fuzzer

RESTler is the first stateful REST API fuzzing tool for automatically testing cloud services through their REST APIs and finding security and reliability bugs in these services.
MIT License
2.52k stars 283 forks source link

Improve logging in RESTler compile step in case of a spec issue instead of just failing without and not giving clear indication of which property is problematic #878

Open SwamyNallamalli opened 3 months ago

SwamyNallamalli commented 3 months ago

Description

When the RESTler compile step is attempted with a spec that has an issue where array element doesn't have a declared type in RESTler. It would be beneficial if the logging is improved to show which specific filed(s) are problematic.

The error is: Unhandled exception. System.ArgumentException: Invalid array schema: found array property without a declared element at Restler.Compiler.SwaggerVisitors.generateGrammarElementForSchema(JsonSchema schema, FSharpOption1 exampleValue, Boolean generateFuzzablePayloadsForExamples, Boolean trackParameters, FSharpOption1 jsonPropertyMaxDepth, Boolean isRequired, Boolean isReadOnly, FSharpList1 parents, SchemaCache schemaCache, FSharpFunc2 cont) in D:\a\_work\_temp\restler-fuzzer\src\compiler\Restler.Compiler\SwaggerVisitors.fs:line 690 at Restler.Compiler.SwaggerVisitors.processProperty(String propertyName, JsonSchemaProperty property, FSharpOption1 propertyPayloadExampleValue, Boolean generateFuzzablePayload, Boolean trackParameters, FSharpOption1 jsonPropertyMaxDepth, FSharpList1 parents, SchemaCache schemaCache, FSharpFunc2 cont) in D:\a\_work\_temp\restler-fuzzer\src\compiler\Restler.Compiler\SwaggerVisitors.fs:line 479 at Restler.Compiler.SwaggerVisitors.declaredPropertyParameters@657.Invoke(KeyValuePair2 item) in D:\a_work_temp\restler-fuzzer\src\compiler\Restler.Compiler\SwaggerVisitors.fs:line 663 at Microsoft.FSharp.Collections.Internal.IEnumerator.choose@171.System-Collections-IEnumerator-MoveNext() in F:\workspace_work\1\s\src\fsharp\FSharp.Core\seq.fs:line 179 at Microsoft.FSharp.Core.CompilerServices.RuntimeHelpers.takeInner@266[T,TResult](ConcatEnumerator2 x, Unit unitVar0) in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\seqcore.fs:line 268 at Microsoft.FSharp.Collections.SeqModule.oneStepTo@987[T](IEnumerable1 source, List1 prefix, FSharpRef1 enumeratorR, Int32 i) in F:\workspace_work\1\s\src\fsharp\FSharp.Core\seq.fs:line 996 at Microsoft.FSharp.Collections.SeqModule.action@4164-1[T](IEnumerable1 source, List1 prefix, FSharpRef1 enumeratorR, Int32 i, Unit unitVar0) in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\seq.fs:line 1011 at Microsoft.FSharp.Collections.SeqModule.result@1003.Invoke(Int32 i) at Microsoft.FSharp.Collections.Internal.IEnumerator.unfold@205.DoMoveNext(b& curr) in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\seq.fs:line 207 at Microsoft.FSharp.Collections.Internal.IEnumerator.MapEnumerator1.System-Collections-IEnumerator-MoveNext() in F:\workspace_work\1\s\src\fsharp\FSharp.Core\seq.fs:line 64 at Microsoft.FSharp.Collections.SeqModule.ToList[T](IEnumerable1 source) in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\seq.fs:line 546 at Restler.Compiler.SwaggerVisitors.generateGrammarElementForSchema(JsonSchema schema, FSharpOption1 exampleValue, Boolean generateFuzzablePayloadsForExamples, Boolean trackParameters, FSharpOption1 jsonPropertyMaxDepth, Boolean isRequired, Boolean isReadOnly, FSharpList1 parents, SchemaCache schemaCache, FSharpFunc2 cont) in D:\a\_work\_temp\restler-fuzzer\src\compiler\Restler.Compiler\SwaggerVisitors.fs:line 885 at Restler.Compiler.SwaggerVisitors.generateGrammarElementForSchema(JsonSchema schema, FSharpOption1 exampleValue, Boolean generateFuzzablePayloadsForExamples, Boolean trackParameters, FSharpOption1 jsonPropertyMaxDepth, Boolean isRequired, Boolean isReadOnly, FSharpList1 parents, SchemaCache schemaCache, FSharpFunc2 cont) in D:\a\_work\_temp\restler-fuzzer\src\compiler\Restler.Compiler\SwaggerVisitors.fs:line 701 at Restler.Compiler.SwaggerVisitors.processProperty(String propertyName, JsonSchemaProperty property, FSharpOption1 propertyPayloadExampleValue, Boolean generateFuzzablePayload, Boolean trackParameters, FSharpOption1 jsonPropertyMaxDepth, FSharpList1 parents, SchemaCache schemaCache, FSharpFunc2 cont) in D:\a\_work\_temp\restler-fuzzer\src\compiler\Restler.Compiler\SwaggerVisitors.fs:line 479 at Restler.Compiler.SwaggerVisitors.declaredPropertyParameters@657.Invoke(KeyValuePair2 item) in D:\a_work_temp\restler-fuzzer\src\compiler\Restler.Compiler\SwaggerVisitors.fs:line 663 at Microsoft.FSharp.Collections.Internal.IEnumerator.choose@171.System-Collections-IEnumerator-MoveNext() in F:\workspace_work\1\s\src\fsharp\FSharp.Core\seq.fs:line 179 at Microsoft.FSharp.Core.CompilerServices.RuntimeHelpers.takeInner@266[T,TResult](ConcatEnumerator2 x, Unit unitVar0) in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\seqcore.fs:line 268 at Microsoft.FSharp.Collections.SeqModule.oneStepTo@987[T](IEnumerable1 source, List1 prefix, FSharpRef1 enumeratorR, Int32 i) in F:\workspace_work\1\s\src\fsharp\FSharp.Core\seq.fs:line 996 at Microsoft.FSharp.Collections.SeqModule.action@4164-1[T](IEnumerable1 source, List1 prefix, FSharpRef1 enumeratorR, Int32 i, Unit unitVar0) in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\seq.fs:line 1011 at Microsoft.FSharp.Collections.SeqModule.result@1003.Invoke(Int32 i) at Microsoft.FSharp.Collections.Internal.IEnumerator.unfold@205.DoMoveNext(b& curr) in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\seq.fs:line 207 at Microsoft.FSharp.Collections.Internal.IEnumerator.MapEnumerator1.System-Collections-IEnumerator-MoveNext() in F:\workspace_work\1\s\src\fsharp\FSharp.Core\seq.fs:line 64 at Microsoft.FSharp.Collections.SeqModule.ToList[T](IEnumerable1 source) in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\seq.fs:line 546 at Restler.Compiler.SwaggerVisitors.generateGrammarElementForSchema(JsonSchema schema, FSharpOption1 exampleValue, Boolean generateFuzzablePayloadsForExamples, Boolean trackParameters, FSharpOption1 jsonPropertyMaxDepth, Boolean isRequired, Boolean isReadOnly, FSharpList1 parents, SchemaCache schemaCache, FSharpFunc2 cont) in D:\a\_work\_temp\restler-fuzzer\src\compiler\Restler.Compiler\SwaggerVisitors.fs:line 885 at Restler.Compiler.Main.allResponses@1225.GenerateNext(IEnumerable1& next) in D:\a_work_temp\restler-fuzzer\src\compiler\Restler.Compiler\Compiler.fs:line 0 at Microsoft.FSharp.Core.CompilerServices.GeneratedSequenceBase1.MoveNextImpl() in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\seqcore.fs:line 371 at Microsoft.FSharp.Core.CompilerServices.GeneratedSequenceBase1.System-Collections-IEnumerator-MoveNext() in F:\workspace_work\1\s\src\fsharp\FSharp.Core\seqcore.fs:line 403 at Microsoft.FSharp.Collections.SeqModule.TryHead[T](IEnumerable1 source) in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\seq.fs:line 1370 at Restler.Compiler.Main.requestDataSeq$cont@1176(Config config, OpenApiDocument swaggerDoc, SchemaCache schemaCache, String ep, KeyValuePair2 m, RequestId requestId, FSharpOption1 exampleConfig, Unit unitVar) in D:\a\_work\_temp\restler-fuzzer\src\compiler\Restler.Compiler\Compiler.fs:line 1268 at Restler.Compiler.Main.getRequestData@1112-1.GenerateNext(IEnumerable1& next) in D:\a_work_temp\restler-fuzzer\src\compiler\Restler.Compiler\Compiler.fs:line 1176 at Microsoft.FSharp.Core.CompilerServices.GeneratedSequenceBase1.MoveNextImpl() in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\seqcore.fs:line 371 at Microsoft.FSharp.Core.CompilerServices.GeneratedSequenceBase1.System-Collections-IEnumerator-MoveNext() in F:\workspace_work\1\s\src\fsharp\FSharp.Core\seqcore.fs:line 403 at Microsoft.FSharp.Core.CompilerServices.RuntimeHelpers.takeInner@266[T,TResult](ConcatEnumerator2 x, Unit unitVar0) in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\seqcore.fs:line 268 at System.Collections.Generic.List1..ctor(IEnumerable1 collection) at Microsoft.FSharp.Collections.SeqModule.ToArray[T](IEnumerable1 source) in F:\workspace_work\1\s\src\fsharp\FSharp.Core\seq.fs:line 825 at Restler.Compiler.Main.generateRequestGrammar(FSharpList1 swaggerDocs, MutationsDictionary dictionary, Config config, FSharpList1 globalExternalAnnotations, FSharpList1 userSpecifiedExamples) in D:\a\_work\_temp\restler-fuzzer\src\compiler\Restler.Compiler\Compiler.fs:line 1338 at Restler.Workflow.generateGrammarFromSwagger(String grammarOutputDirectoryPath, Config config) in D:\a\_work\_temp\restler-fuzzer\src\compiler\Restler.Compiler\Workflow.fs:line 143 at Restler.Workflow.generateRestlerGrammar(Config config) in D:\a\_work\_temp\restler-fuzzer\src\compiler\Restler.Compiler\Workflow.fs:line 250 at Program.main(String[] argv) in D:\a\_work\_temp\restler-fuzzer\src\compiler\Restler.CompilerExe\Program.fs:line 43

Steps to reproduce

Expected results

No response

Actual results

No response

Environment details

No response

yufan412 commented 1 week ago

I also encountered this issue while compiling the specification file for Gitlab. May I ask if you have resolved it