cybertk / abao

REST API automated testing tool based on RAML
MIT License
354 stars 59 forks source link

Hook file is not working #32

Closed agrover8 closed 9 years ago

agrover8 commented 9 years ago

Hi,

I have created hook coffee script which looks like abao is not reading.

My RAML File look like:

#%RAML 0.8
baseUri: https://test.com/ssvui
title: Origin Api
version: 0.1

resourceTypes:
   - collection:
      description: |
        The <<resourcePathName>> collection.
      get:
        description: Get a list of <<resourcePathName>>.
        headers: &commonHeaders
          content-type:
            displayName: Content Type
            description: MIME type of document being served.
            example: "application/json"

          accept:
            displayName: Accept
            description: MIME types that client is willing to accept.
            example: "*/*"
        responses:
          200:
          500:
            body:
              application/json:
                example: |
                  {"message": "Internal server error" }
      post:
        headers: *commonHeaders
        description: |
          TBD
        body:
          application/json:
            example: <<exampleItemForCreate>>
        responses:
          200:
          302:
          500:
            body:
              application/json:
                example: |
                  {"message": "Internal server error" }

/j_spring_security_check:
    description: | 
      Login
    post:
        headers:
          accept:
            displayName: Accept
            description: MIME types that client is willing to accept.
            example: "*/*"
        queryParameters:
          j_username:
            description: QA_Autotest
            example: QA_Autotest
          j_password:
            description: QA_aut0test
            example: QA_aut0test
          response:
            description: light
            example: light
        body:
          application/json:
        responses:
          302:
            body:
              application/json:
                #schema: security
/auth:
    description: |
      Auth
    get:
        headers:
          accept:
            displayName: Accept
            description: MIME types that client is willing to accept.
            example: "*/*"
        responses:
          200:
            body:
              application/json:

/org/15602/activity:
  /{activityid}:
    description: |
      Activity
    get:
       headers:
         accept:
           displayName: Accept
           description: MIME types that client is willing to accept.
           example: "*/*"
       queryParameters:
         activityid:
           description: Activity
           example: 55493526
       responses:
          200:
            body:
              application/json:

Hook file:

{before, after} = require 'hooks'

before 'GET /j_spring_security_check -> 302', (test, done) ->
 test.request.params =
  j_username: 'username'
  j_pasword: 'test'
  response: 'light'
 done()

before 'GET /org/15602/activity/{activityid} -> 200', (test, done) ->
 test.request.params =
  activityid: '123'
 done()

before 'GET /org/15602/activity/{activityid} -> 200', (test, done) ->
 test.request.params =
  activityid: 'XYZ'
 done()

Can anyone help me on this?

cybertk commented 9 years ago

What is your problem?

agrover8 commented 9 years ago

@cybertk In my hook file, I have provided two different cases for resource 'GET /org/15602/activity/{activityid} -> 200'

Both the cases have incorrect activityid, so I am expecting these test cases should be fail.

But when I am running the following command all the test passed. $ abao origin_new.raml https://test.com/ssvui --hookfiles=hook.coffee

So to debug this I just ran the raml file without hook file: $ abao origin_new.raml https://test.com/ssvui

Result of the above command is also same as the first command, so its look like abao is not reading hook file. Though it says: Found Hookfiles: hook.coffee

Please let me know if you need more information.

cybertk commented 9 years ago

You can define only one test for each case(response code) in Abao. So for your case, you defined two tests/hooks for GET /org/15602/activity/{activityid} -> 200, only one is used and the other one is ignored.

If you want test the error case of your API, you should define it in your RAML. e.g. Your API will response 400 if activityid is not a number with

responses:
          200:
            ...
          400:
           ...

I think Abao should print a warning message for your case, what's your advice?

agrover8 commented 9 years ago

@cybertk

  1. If that is the case then why the first case 'GET /j_spring_security_check -> 302', (test, done) -> is getting pass where in Hook file I have entered incorrect login and RAML has correct login.
  2. How can i test multiple test cases for one case?

For example, I have one resource GET /org/15602/activity/{activityid} -> 200, and I want to test this resource with 3 different activityid where the response code for all 3 cases is 200.

cybertk commented 9 years ago
  1. Each test case is identified by Test Name, such as GET /j_spring_security_check -> 302.
  2. Abao does not support this feature now.
agrover8 commented 9 years ago

@cybertk If we can define only one test for each case(response code) in Abao. So the test mentioned in the Hook file run for that particular resuource, right? Example: RAML file has activityid 12345* for GET /org/15602/activity/{activityid} -> 200 and Hook file has activityid 567* for GET /org/15602/activity/{activityid} -> 200

Abao test will run with activityid 567*\ (Hook file), is that correct?

cybertk commented 9 years ago

Correct.

cybertk commented 9 years ago

@agrover8 I have an idea for your condition, test multiple test cases for one case. You can write multiple hooks, then run Abao multiple times with different hook.

Such as,

$ abao origin_new.raml https://test.com/ssvui --hookfiles=hook1.coffee

$ abao origin_new.raml https://test.com/ssvui --hookfiles=hook2.coffee

$ abao origin_new.raml https://test.com/ssvui --hookfiles=hook3.coffee
cybertk commented 9 years ago

You can open a feature request either, we will implement it in future.

agrover8 commented 9 years ago

@cybertk Can we pass request data from external file to hook file, like in RAML we can !include example.sample? Instead of passing request data in hook file I want to pass reference of the external file.

cybertk commented 9 years ago

Yes, you can. Hook file is just a normal javascript/coffeescript source file, you can do whatever you want.

agrover8 commented 9 years ago

@cybertk There are some instances where we need to pass the response data from one resource to request data, query parameters, uriParameters to other resource, is that possible in ABAO?

cybertk commented 9 years ago

Yes. Do whatever javascript can do

jekhardt commented 9 years ago

Filed #35 for multi test case.

I had some working example going of parameter passing using some global context variable to define things.

Something roughly like...

context = {}

after < test 1 > ... context.example.id = test.response.body.example.id

before < test 2 > ... test.request.query.id = context.example.id

@agrover8 is your hook file working now?

agrover8 commented 9 years ago

@cybertk @jekhardt Yes, hook file is working now. I am closing this issue.