search

svanoort / pyresttest

Python Rest Testing
Apache License 2.0
1.15k stars 325 forks source link

Data input issue #262

Open contactcuong opened 6 years ago

contactcuong commented 6 years ago

Hi,

My REST requires couple of data inputs (json format) in order pass the test but I cannot deal with this inside the yaml file. I firstly tried to seeking similar issue around and no luck.

  1. yaml is as following (sensitive information are being hidden as "xxx"), getting "ValueError: Invalid query: enrollmentState : No JSON object could be decoded"
[root@cuongnguyen ~]# cat payload_test.yaml
---
- config:
    - testset: "Payload_set"
    - timeout: 100
- test:
    - group: "[Payload Group]"
    - name: "Payload_test"
    - headers: {'Content-Type': 'application/json'}
    - url: "/hls/services/1.0/service/discover"
    - method: "POST"
    - body: '{"carrierSubscriberNetworkId": "xxx", "countryCode": xxx, "hardwareId": "xxx"}'
    - expected_status: [200, 201, 301, 302, 307]
    - validators:
        - compare: {jmespath: 'enrollmentState', comparator: "contains", expected: "Provisioned"}
[root@cuongnguyen ~]# 
[root@cuongnguyen ~]# 
[root@cuongnguyen ~]# 
[root@cuongnguyen ~]# 
[root@cuongnguyen ~]# 
[root@cuongnguyen ~]# pyresttest https://<my_url>:xxx payload_test.yaml --verbose --print-bodies true --print-headers true
* About to connect() to xxx port xxx (#0)
*   Trying xxx... * connected
* Connected to xxx (xxx) port xxx (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate:
*       subject: CN=xxx
*       start date: Jan 18 00:00:00 2017 GMT
*       expire date: Apr 17 12:00:00 2019 GMT
*       common name: xxx
*       issuer: CN=xxx
> POST /hls/services/1.0/service/discover HTTP/1.1
User-Agent: PycURL/7.43.0 libcurl/7.19.7 NSS/3.27.1 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
Host: xxx:xxx
Accept: */*
Content-Type:application/json
Connection: close
Content-Length: 96

< HTTP/1.1 307 Temporary Redirect
< Server: nginx
< Date: Fri, 08 Dec 2017 10:52:19 GMT
< Content-Type: application/json
< Content-Length: 0
< Connection: close
< Location: https://<my_url>:xxx/hls/services/1.0/service/discover
< 
* Closing connection #0

[('server', 'nginx'), ('date', 'Fri, 08 Dec 2017 10:52:19 GMT'), ('content-type', 'application/json'), ('content-length', '0'), ('connection', 'close'), ('location', 'https://<my_url>:xxx/hls/services/1.0/service/discover')]
ERROR:Test Failed: Payload_test URL=https://xxx:xxx/hls/services/1.0/service/discover Group=[Payload Group] HTTP Status Code: 307
ERROR:Test Failure, failure type: Extractor Exception, Reason: Extractor threw exception
ERROR:Validator/Error details:Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/pyresttest-1.7.2.dev0-py2.7.egg/pyresttest/validators.py", line 358, in validate
    body=body, headers=headers, context=context)
  File "/usr/local/lib/python2.7/site-packages/pyresttest-1.7.2.dev0-py2.7.egg/pyresttest/validators.py", line 185, in extract
    return self.extract_internal(query=query, body=body, headers=headers, args=self.args)
  File "/usr/local/lib/python2.7/site-packages/pyresttest-1.7.2.dev0-py2.7.egg/pyresttest/ext/extractor_jmespath.py", line 38, in extract_internal
    raise ValueError("Invalid query: " + query + " : " + str(e))
ValueError: Invalid query: enrollmentState : No JSON object could be decoded

Test Group [Payload Group] FAILED: : 0/1 Tests Passed!
[root@cuongnguyen ~]#
  1. the below is what I got from curl invoke
[root@cuongnguyen ~]# curl -vL -X POST -H "content-type: application/json" -d '{"carrierSubscriberNetworkId": "xxx", "countryCode": xxx, "hardwareId": "xxx"}' https://<my_url>:xxx/hls/services/1.0/service/discover | jq .
* About to connect() to <my_url> port xxx (#0)
*   Trying xxx... connected
* Connected to <my_url> (xxx) port xxx (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate:
*       subject: xxx
*       start date: Jan 18 00:00:00 2017 GMT
*       expire date: Apr 17 12:00:00 2019 GMT
*       common name: xxx
*       issuer: xxx
> POST /hls/services/1.0/service/discover HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.27.1 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> Host: <my_url>:xxx
> Accept: */*
> content-type: application/json
> Content-Length: 96
> 
} [data not shown]
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0    96      0    125 --:--:-- --:--:-- --:--:--   125< HTTP/1.1 307 Temporary Redirect
< Server: nginx
< Date: Fri, 08 Dec 2017 10:41:18 GMT
< Content-Type: application/json
< Content-Length: 0
< Connection: keep-alive
< Location: https://<my_url>:xxx/hls/services/1.0/service/discover
< 
  0     0    0     0    0    96      0     85 --:--:--  0:00:01 --:--:--     0* Connection #0 to host <my_url> left intact
* Issue another request to this URL: 'https://<my_url>:xxx/hls/services/1.0/service/discover'
* About to connect() to <my_url> port xxx (#1)
*   Trying xxx... connected
* Connected to <my_url> (xxx) port xxx (#1)
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate:
*       subject: CN=xxx
*       start date: Sep 21 00:00:00 2016 GMT
*       expire date: Sep 26 12:00:00 2018 GMT
*       common name: *.xxx
*       issuer: CN=xxx
> POST /hls/services/1.0/service/discover HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.27.1 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> Host: <my_url>:xxx
> Accept: */*
> content-type: application/json
> Content-Length: 96
> 
} [data not shown]
  0     0    0     0    0    96      0     45 --:--:--  0:00:02 --:--:--    45< HTTP/1.1 200 OK
< Server: nginx
< Date: Fri, 08 Dec 2017 10:41:19 GMT
< Content-Type: application/json
< Content-Length: 942
< Connection: keep-alive
< 
{ [data not shown]
115   942  104   942    0    96    376     38  0:00:02  0:00:02 --:--:--  2125* Connection #1 to host <my_url> left intact

* Closing connection #0
* Closing connection #1
{
  "enrollmentState": "Provisioned",
  "serviceData": {
    "partnerId": "xxx",
    "tenantId": "xxx",
    "subscriberNetworkId": "xxx",
    "deviceId": "xxx",
    "subscriberEndpoint": "xxx",
    "otpEndpoint": "xxx",
    "operatorEndpoint": "xxx",
    "notifyEndpoint": "xxx",
    "scepEndpoint": "xxx",
    "ntpEndpoint": "xxx",
    "snidJoined": false,
    "utcTimestamp": 1507615533293,
    "authenticationMode": "io",
    "homeEndpoint": "xxx",
    "countryCode": xxx,
    "di": xxx
  },
  "retryInterval": 86400
}
[root@cuongnguyen ~]#

Thanks for help!

mvitiuk commented 6 years ago

when you removing - validators: the test passes? Because it says ERROR:Validator/Error details:Traceback (most recent call last): When you using other validator, does it work? Also ERROR:Test Failure, failure type: Extractor Exception, Reason: Extractor threw exception maybe you need to change something in here jmespath: 'enrollmentState',

contactcuong commented 6 years ago

@mvitiuk yes that's correct, the test has passed when removing - validators: Adding another kind of - validators: should work too.

But both are not my initial expected as extra of data inputs would be required, i.e curl -vL -X POST -H "content-type: application/json" -d '{"carrierSubscriberNetworkId": "xxx", "countryCode": xxx, "hardwareId": "xxx"}' https://<my_url>:xxx/hls/services/1.0/service/discover

  1. removing - validators:
[root@cuongnguyen ~]# cat payload_test1.yaml 
---
- config:
    - testset: "Payload_set"
    - timeout: 100
- test:
    - group: "[Payload Group]"
    - name: "Payload_test"
    - headers: {'Content-Type': 'application/json'}
    - url: "/hls/services/1.0/service/discover"
    - method: "POST"
    - body: '{"carrierSubscriberNetworkId": "xxx", "countryCode": xxx, "hardwareId": "xxx"}'
    - expected_status: [200, 201, 301, 302, 307]
[root@cuongnguyen ~]# 
[root@cuongnguyen ~]# 
[root@cuongnguyen ~]# 
[root@cuongnguyen ~]# pyresttest https://xxx:8443 payload_test1.yaml --verbose --print-bodies true --print-headers true
...
<omit part>
...
[('server', 'nginx'), ('date', 'Fri, 08 Dec 2017 11:52:33 GMT'), ('content-type', 'application/json'), ('content-length', '0'), ('connection', 'close'), ('location', 'https://xxx:8443/hls/services/1.0/service/discover')]
Test Group [Payload Group] SUCCEEDED: : 1/1 Tests Passed!
[root@cuongnguyen ~]#
  1. another kind of - validators: 
    
[root@cuongnguyen ~]# cat payload_test2.yaml 
---
- config:
- testset: "Payload_set"
- timeout: 100
- test:
- group: "[Payload Group]"
- name: "Payload_test"
- headers: {'Content-Type': 'application/json'}
- url: "/hls/services/1.0/service/discover"
- method: "POST"
- body: '{"carrierSubscriberNetworkId": "xxx", "countryCode": xxx, "hardwareId": "xxx"}'
- expected_status: [200, 201, 301, 302, 307]
- validators:
    - compare: {header: 'Content-Type', expected: 'application/json'}
[root@cuongnguyen ~]# 
[root@cuongnguyen ~]# 
[root@cuongnguyen ~]# 
[root@cuongnguyen ~]# pyresttest https://xxx:8443 payload_test2.yaml --verbose --print-bodies true --print-headers true
...
<omit part>
...

[('server', 'nginx'), ('date', 'Fri, 08 Dec 2017 11:54:40 GMT'), ('content-type', 'application/json'), ('content-length', '0'), ('connection', 'close'), ('location', 'https://xxx:8443/hls/services/1.0/service/discover')] Test Group [Payload Group] SUCCEEDED: : 1/1 Tests Passed! [root@cuongnguyen ~]#

mvitiuk commented 6 years ago

Don't really follow, sorry. Do you want it as - body: '{"carrierSubscriberNetworkId": "generated1", "countryCode": generated2, "hardwareId": "generated3"} In this case you can use: - body: { template: '{ "carrierSubscriberNetworkId": "generated1",.....}' } and add a generator function in config or so 


 - generators:
      - 'urlvalue': { type: 'env' }```
contactcuong commented 6 years ago

@mvitiuk I am not sure for the declaration inside body:, please forget my body: section, my expectation is what is the best practice of yaml file for following REST, with condition: if stdout of "enrollmentState" contains "Provisioned" string ; then test should PASS!.

Your help would be highest appreciated. Thank you!

$ curl -vL -X POST -H "content-type: application/json" -d '{"carrierSubscriberNetworkId": "xxx", "countryCode": yyy, "hardwareId": "zzz"}' https://<my_url>:xxx/hls/services/1.0/service/discover

{
  "enrollmentState": "Provisioned",
  "serviceData": {
    "partnerId": "xxx",
    "tenantId": "xxx",
    "subscriberNetworkId": "xxx",
    "deviceId": "xxx",
    "subscriberEndpoint": "xxx",
    "otpEndpoint": "xxx",
    "operatorEndpoint": "xxx",
    "notifyEndpoint": "xxx",
    "scepEndpoint": "xxx",
    "ntpEndpoint": "xxx",
    "snidJoined": false,
    "utcTimestamp": 1507615533293,
    "authenticationMode": "io",
    "homeEndpoint": "xxx",
    "countryCode": xxx,
    "di": xxx
  },
  "retryInterval": 86400
}