microsoft / restler-fuzzer

RESTler is the first stateful REST API fuzzing tool for automatically testing cloud services through their REST APIs and finding security and reliability bugs in these services.
MIT License
2.56k stars 291 forks source link

Unable to cast object of type 'NJsonSchema.JsonSchema' to type 'NSwag.OpenApiExample' #441

Open LYDE1234 opened 2 years ago

LYDE1234 commented 2 years ago

I'm getting this

Unhandled exception. System.AggregateException: One or more errors occurred. (Unable to cast object of type 'NJsonSchema.JsonSchema' to type 'NSwag.OpenApiExample'.) ---> System.InvalidCastException: Unable to cast object of type 'NJsonSchema.JsonSchema' to type 'NSwag.OpenApiExample'. at NJsonSchema.References.JsonReferenceBase1.NJsonSchema.References.IJsonReferenceBase.set_Reference(IJsonReference value) at NJsonSchema.JsonSchemaReferenceUtilities.JsonReferenceUpdater.VisitJsonReferenceAsync(IJsonReference reference, String path, String typeNameHint) at NJsonSchema.Visitors.AsyncJsonReferenceVisitorBase.VisitAsync(Object obj, String path, String typeNameHint, ISet1 checkedObjects, Action1 replacer) at NJsonSchema.Visitors.AsyncJsonReferenceVisitorBase.VisitAsync(Object obj, String path, String typeNameHint, ISet1 checkedObjects, Action1 replacer) at NJsonSchema.Visitors.AsyncJsonReferenceVisitorBase.VisitAsync(Object obj, String path, String typeNameHint, ISet1 checkedObjects, Action1 replacer) at NJsonSchema.Visitors.AsyncJsonReferenceVisitorBase.VisitAsync(Object obj, String path, String typeNameHint, ISet1 checkedObjects, Action1 replacer) at NJsonSchema.Visitors.AsyncJsonReferenceVisitorBase.VisitAsync(Object obj, String path, String typeNameHint, ISet1 checkedObjects, Action1 replacer) at NJsonSchema.Visitors.AsyncJsonReferenceVisitorBase.VisitAsync(Object obj, String path, String typeNameHint, ISet1 checkedObjects, Action1 replacer) at NJsonSchema.Visitors.AsyncJsonReferenceVisitorBase.VisitAsync(Object obj, String path, String typeNameHint, ISet1 checkedObjects, Action1 replacer) at NJsonSchema.Visitors.AsyncJsonReferenceVisitorBase.VisitAsync(Object obj, String path, String typeNameHint, ISet1 checkedObjects, Action1 replacer) at NJsonSchema.Visitors.AsyncJsonReferenceVisitorBase.VisitAsync(Object obj) at NJsonSchema.JsonSchemaReferenceUtilities.JsonReferenceUpdater.VisitAsync(Object obj) at NJsonSchema.JsonSchemaReferenceUtilities.UpdateSchemaReferencesAsync(Object rootObject, JsonReferenceResolver referenceResolver, IContractResolver contractResolver) at NJsonSchema.Infrastructure.JsonSchemaSerialization.FromJsonAsync[T](String json, SchemaType schemaType, String documentPath, Func2 referenceResolverFactory, IContractResolver contractResolver) at NSwag.OpenApiDocument.FromJsonAsync(String data, String documentPath, SchemaType expectedSchemaType, Func2 referenceResolverFactory) at NSwag.OpenApiDocument.FromFileAsync(String filePath) --- End of inner exception stack trace --- at Microsoft.FSharp.Control.AsyncResult1.Commit() in F:\workspace_work\1\s\src\fsharp\FSharp.Core\async.fs:line 349 at Microsoft.FSharp.Control.AsyncPrimitives.RunSynchronouslyInCurrentThread[a](CancellationToken cancellationToken, FSharpAsync1 computation) in F:\workspace_work\1\s\src\fsharp\FSharp.Core\async.fs:line 882 at Microsoft.FSharp.Control.AsyncPrimitives.RunSynchronously[T](CancellationToken cancellationToken, FSharpAsync1 computation, FSharpOption1 timeout) in F:\workspace_work\1\s\src\fsharp\FSharp.Core\async.fs:line 890 at Microsoft.FSharp.Control.FSharpAsync.RunSynchronously[T](FSharpAsync1 computation, FSharpOption1 timeout, FSharpOption1 cancellationToken) in F:\workspace_work\1\s\src\fsharp\FSharp.Core\async.fs:line 1154 at Restler.Workflow.swaggerDocs@88-1.Invoke(String fp) in D:\a\1\s\src\compiler\Restler.Compiler\Workflow.fs:line 88 at Microsoft.FSharp.Primitives.Basics.List.map[T,TResult](FSharpFunc2 mapping, FSharpList1 x) in F:\workspace_work\1\s\src\fsharp\FSharp.Core\local.fs:line 248 at Microsoft.FSharp.Collections.ListModule.Map[T,TResult](FSharpFunc2 mapping, FSharpList1 list) in F:\workspace_work\1\s\src\fsharp\FSharp.Core\list.fs:line 75 at Restler.Workflow.generateGrammarFromSwagger(String grammarOutputDirectoryPath, FSharpOption1 swaggerDoc, Config config) in D:\a\1\s\src\compiler\Restler.Compiler\Workflow.fs:line 88 at Restler.Workflow.generateRestlerGrammar(FSharpOption1 swaggerDoc, Config config) in D:\a\1\s\src\compiler\Restler.Compiler\Workflow.fs:line 247 at Program.main(String[] argv) in D:\a\1\s\src\compiler\Restler.CompilerExe\Program.fs:line 37

when trying to compile this examples.json

{ "parameters": { "id": "09b11ea3-085c-4641-a84a-ce2f68a09a53" } }

that is referenced from yaml that looks like this:

parameters:

This is a cross-post from:

https://github.com/microsoft/rest-api-fuzz-testing/issues/242

marina-p commented 2 years ago

Hello @LYDE1234,

I cannot reproduce this issue. When I run RESTler standalone, with the repro below, it is compiled successfully. Is there something different in my repro? I put the example.json in the same directory as the repro.yml below.

openapi: 3.0.3
info:
  title: example api
  version: 1.2.3
paths:
  "/test_endpoint_1/{id}":
    post:
      parameters:
        - name: 'id'
          in: 'path'
          description: 'Id of the object'
          required: true
          schema:
            type: 'string'
            format: 'guid'
      examples:
        camera:
          $ref: 'examples.json'
      responses:
        '200':
          description: OK

Thanks,

Marina

LYDE1234 commented 2 years ago

I cant put the example.json in the same dir as the YAML because of how RAFT uses mount points. I mount to this:

{
   "fileShareName": "examples",
      "mountPath": "/examples"
}

and that resolves to

      examples:
        camera:
          $ref: '../examples/examples.json'

in the YAML.

why would this be dependent on the files being in the same dir? the compiler does seem to pick up the YAML and reference the examples. Could it be a casting / schema problem like what's described here:

https://github.com/RicoSuter/NSwag/issues/2921

or is a RESTler update required in RAFT?

marina-p commented 2 years ago

Hi @LYDE1234,

You are correct, the example does not need to be in the same directory (I was trying to simplify the repro, but your reference path above should also work). Can you please confirm, what does the "../examples/examples.json" have in your repro? This is related to my comment/question in the RAFT issue:

I believe the reason for your error is you are referencing the example config file in your specification yaml. The specification should only reference actual examples (like the one you provided which contains "id"). The example config file should only be referenced in config.json, and the example config file contains a list of request types, which point to one or more examples.

Thanks,

Marina

LYDE1234 commented 2 years ago

The examples.json has this:

{ "parameters": { "id": "09b11ea3-085c-4641-a84a-ce2f68a09a53" } }

I've also experimented with a different syntax in the YAML:

  parameters:
    - name: 'id'
      in: 'path'
      description: 'Id of the object'
      required: true
      schema:
        type: 'string'
        format: 'guid'
    - name: 'tasks'
      in: 'query'
      description: 'Get list of all tasks supported by this object'
      required: false
      schema:
        type: 'string'
  x-ms-examples:    
         ex:
          $ref: '../examples/examples.json'

This matches the syntax for the yaml example here:

https://github.com/microsoft/restler-fuzzer/blob/main/src/compiler/Restler.Compiler.Test/swagger/example_demo1.yaml

"parameters:

If I use the above and set "DiscoverExamples" to true, my examples.json does get copied to the Examples folder in the work dir. But, I dont see any traces in the restler logs of the example value being used anywhere.

It would make sense that the "NJsonSchema.JsonSchema' to type 'NSwag.OpenApiExample" problem is caused by the Examples being at the wrong level (and then being parsed as 'schema'), so I'm guessing the above syntax is the better one. But, why doesn't the value in the examples.json get used in the calls that RESTler generates?

marina-p commented 2 years ago

@LYDE1234

If I use the above and set "DiscoverExamples" to true, my examples.json does get copied to the Examples folder in the work dir.

Great! These should work the same in RESTler, I do not know without further investigation why modifying from examples to x-ms-examples resolves your issue, but I will look into this. Were you only seeing this behavior through RAFT, or also running RESTler standalone?

But, I dont see any traces in the restler logs of the example value being used anywhere.

This is most likely because your example parameter is a path parameter, and by default RESTler sets usePathExamples to false. You can override this value by setting it in the config.json. Can you please try this, and confirm you now see the parameter in the trace? It will also be included in the grammar, in case it's quicker to search for it in the grammar than run the Test phase.

(The reason usePathExamples is false by default is that using example values is typically not recommended for paths (we have observed this causes more errors due to outdated values, etc.). In the above case, ideally this parameter should be a restler_custom_payload_uuid4_suffix which RESTler infers automatically, but unfortunately this type does not work with guids yet. I would recommend using a restler_custom_payload for this parameter, with several values in the dictionary, so several objects can be created. )

Thanks,

Marina