schemathesis / schemathesis

Supercharge your API testing, catch bugs, and ensure compliance
https://schemathesis.readthedocs.io
MIT License
2.29k stars 163 forks source link

OAS validation for binary stream payloads of Contract Type application/pdf, image/jpeg, image/png, application/msword #2101

Closed gnanendra-bogireddy closed 7 months ago

gnanendra-bogireddy commented 7 months ago

Is your feature request related to a problem? Please describe

We have a couple of API which expects a binary data (not base64 encoded) from consumers like below. Is there any way to verify the provided OAS file with the provider api?

paths: /anything post: requestBody: content: image/jpeg schema: type: string format: binary application/pdf schema: type: string format: binary

Describe the solution you'd like

We need a way to upload a given pdf or image to provider api using Schemathesis and verify the response from the provider with the OAS file.

Describe alternatives you've considered

We tried to use the Dredd tool for the same but it is encoding files in base64 strings which is not supported by our API.

Stranger6667 commented 7 months ago

Hi @gnanendra-bogireddy

Is there any way to verify the provided OAS file with the provider api?

By default, Schemathesis does not know how to generate valid JPEGs or PDFs, however, in the next minor version it will be possible to define a custom data generator for such cases. The feature is already merged and you can try it in the latest Docker image, for example.

See the docs for examples. The verification step will work as before with any other API response, so you need only to configure data generation.

Let me know if this is helpful for your use case, otherwise I'd be happy to elaborate

gnanendra-bogireddy commented 7 months ago

Thanks for the quick response. Sure I will try and let you know.

gnanendra-bogireddy commented 7 months ago

Hi @Stranger6667

I am using Schemathesis local python package not as a docker image.

When can we expect the next release ?

Is there any way to test custom media types using cli ? It will be very helpful for everyone who's not using python.

What happens if we have multiple custom media types in the same operation that needs data generation ? Can this works for api expects data in multiple content types ?

Can you please provide an example for the open api specification along with python code it will be very helpful.

Open api specification

paths:
  /anything:
    post:
      tags:
        - 'customtag'
      requestBody:
        content:
          image/jpeg:
            schema:
              type: string
              format: binary
            application/pdf:
              schema:
                type: string
                format: binary
            image/png:
              schema:
                type: string
                format: binary
      responses:
        '200':
            description: success
         '400':
            description: bad request
         '401':
            description: unauthorized
         '500':
            description: internal server error
Stranger6667 commented 7 months ago

Hi @gnanendra-bogireddy

I am using Schemathesis local python package not as a docker image.

Probably you can try it with the git installation:

pip install git+https://github.com/schemathesis/schemathesis.git#egg=schemathesis

When can we expect the next release ?

I plan to issue a new release around this weekend.

Is there any way to test custom media types using cli ? It will be very helpful for everyone who's not using python.

You can use extensions in CLI too, however, I don't see a way to provide an ergonomic API to build custom generators via CLI only, the internals are way too complex to be exposed this way.

What happens if we have multiple custom media types in the same operation that needs data generation ? Can this works for api expects data in multiple content types ?

Schemathesis will work per media type, so if you register multiple generators for different media types, they will be used in all places they occur in the spec.

Can you please provide an example for the open api specification along with python code it will be very helpful.

No specific changes to the spec are needed.

Here is a code sample from the docs, you can use your own binary payloads inside the sampled_from input:

from hypothesis import strategies as st
import schemathesis

# Define your own strategy for generating PDFs
# NOTE: This is a simplified example, actual PDF generation is much more complex
pdfs = st.sampled_from([b"%PDF-1.5...", b"%PDF-1.6..."])

# Register the strategy for "application/pdf" media type
schemathesis.openapi.media_type("application/pdf", pdfs)
# You can also specify one or more additional aliases for the media type
schemathesis.openapi.media_type("application/pdf", pdfs, aliases=["application/x-pdf"])

Then this file (lets call it hooks.py) should be enabled via the SCHEMATHESIS_HOOKS env var, see the docs for details.

Let me know if I can answer any further questions

gnanendra-bogireddy commented 7 months ago

Thanks@Stranger6667 for the quick response. Sure I will clone the github repo to my local.

Stranger6667 commented 7 months ago

@gnanendra-bogireddy

FYI, it is released in 3.26.0

gnanendra-bogireddy commented 7 months ago

Thank you @Stranger6667 for your support. Now I am able to serialize the data in application/pdf, image/jpeg.

Stranger6667 commented 7 months ago

Awesome!

Feel free to reopen this issue if you have any further questions!