search

swagger-api / swagger-codegen

swagger-codegen contains a template-driven engine to generate documentation, API clients and server stubs in different languages by parsing your OpenAPI / Swagger definition.
http://swagger.io
Apache License 2.0
16.74k stars 6.02k forks source link

Generate test clients for stress testing #1910

Open wilsonmar opened 8 years ago

wilsonmar commented 8 years ago

Client code currently generated by swagger-codegen is meant for manual invocation.

It would be a revolution in agility if, instead of waiting for test code to be written manually, that immediately after a change in swagger specs, for performance benchmarks be obtained based on generated client test code and test runs.

Provide an example tutorial of how to use swagger-codegen to automatically create and run performance tests. Perhaps use testing frameworks such as JMeter, Gatlin, LoadRunner, etc.

wing328 commented 8 years ago

@wilsonmar thanks for the feedback. There's a client template for JMeter and you can find it in https://editor.swagger.io under "Generate Client". Would that meet your requirement ?

wing328 commented 8 years ago

@wilsonmar did you have a chance to try the JMeter API client? Would that meet your requirement?

wilsonmar commented 8 years ago

@wind328 Sorry, missed your earlier email. The trick with generation of code is the customizations people want to have. In the case of test clients, different people may want different features in the client generated. Also, the difference between a UI-based client and client for testing is that no manual clicking is expected. I'll email you so we can discuss 1:1. I've been designing a set of options that people can select before doing the gen. / Wilson

On Tue, Feb 2, 2016 at 8:32 AM, wing328 notifications@github.com wrote:

@wilsonmar https://github.com/wilsonmar did you have a chance to try the JMeter API client? Would that meet your requirement?

— Reply to this email directly or view it on GitHub https://github.com/swagger-api/swagger-codegen/issues/1910#issuecomment-178674907 .

foragerr commented 8 years ago

@wing328 The JMeter API does not use basePath correctly. Will raise a new issue for this.

foragerr commented 8 years ago

@wilsonmar Generating a .c file for LoadRunner with web_custom_request() is trivial. But do you have any thoughts about generating an entire valid LoadRunner script?

wilsonmar commented 8 years ago

@RaGe. I am so glad you asked. Yes. I have a LoadRunner API load test script at https://github.com/wilsonmar/LoadRunner/tree/master/gapi-lr-starter (It's not for load testing Google, but as a template for a script that calculates JWT tokens based on .p12 and JSON auth certificates). As its README says, the script also makes use of HP's Virtual Table Service and other useful features.

There are TODO steps in the script to note where customization would be useful.

I also have a tutorial in a Word file explaining not just its coding but also how to obtain a Service Account from Google (again an example).

I am currently working on a JavaScript language version of the script.

BTW, this use of Swagger is a key part a whole approach (step-by-step tutorial) for using open-source software in performance testing I'm proposing at http://github.com/wilsonmar/oss-perf

However, we may want to start off with something smaller like the generator that creates a LoadRunner script from an Excel spreadsheet http://www.wilsonmar.com/lrxls2vu.htm

On Thu, Feb 11, 2016 at 11:15 PM, RaGe notifications@github.com wrote:

@wilsonmar https://github.com/wilsonmar Generating a .c file for LoadRunner with web_custom_request() is trivial. But do you have any thoughts about generating an entire valid LoadRunner script?

— Reply to this email directly or view it on GitHub https://github.com/swagger-api/swagger-codegen/issues/1910#issuecomment-183194176 .

foragerr commented 8 years ago

@wilsonmar I looked at your excel macro script generation, you're still only generating only the .c file. I was asking about the remaining LR script files that forms the whole runnable script. Doing only .c files means there will be a manual step in copy pasting code into a LoadRunner script which is less than ideal IMO.

Nonetheless, I'm working on templates for a .c file with web_custom_request wrapped in lr_transaction. I should have something ready soon.

wilsonmar commented 8 years ago

Did I mention that I'm familiar with those extra files? My secret sauce for creating the other files is described in this diagram I created several years ago: http://wilsonmar.com/1loadrun.htm#LRArch

But we need to talk - please email me at wilsonmar@gmail.com. I'm also on skype as wilsonmar4.

On Fri, Feb 12, 2016 at 11:53 AM, RaGe notifications@github.com wrote:

@wilsonmar https://github.com/wilsonmar I looked at your excel macro script generation, you're still only generating only the .c file. I was asking about the remaining LR script files that forms the whole runnable script. Doing only .c files means there will be a manual step in copy pasting code into a LoadRunner script which is less than ideal IMO.

Nonetheless, I'm working on templates for a .c file with web_custom_request wrapped in lr_transaction. I should have something ready soon.

— Reply to this email directly or view it on GitHub https://github.com/swagger-api/swagger-codegen/issues/1910#issuecomment-183448803 .