how to wait for response.parsedBody promise to be resolved?

IvoLeist commented 2 years ago

Dear AnWeber,

when starting to implement tests in httpYac using httpbook which require a little bit more response time from the server it looks to me that the response.parsedBody promise is not resolved in time before my chai test. See error message below:

AssertionError: schema: expected undefined to be truthy at Proxy. ([node_modules/chai-json-schema/index.js:132:16]()) at Proxy.methodWrapper ([node_modules/chai/lib/chai/utils/addMethod.js:57:25]()) at Object. ([test.http:14:42]()) at i2 ([/home/ivo/.vscode-server/extensions/anweber.vscode-httpyac-4.10.2/dist/extension.js:128:90931]()) at Th ([/home/ivo/.vscode-server/extensions/anweber.vscode-httpyac-4.10.2/dist/extension.js:129:30436]()) at Object.action ([/home/ivo/.vscode-server/extensions/anweber.vscode-httpyac-4.10.2/dist/extension.js:129:29800]()) at $B. ([/home/ivo/.vscode-server/extensions/anweber.vscode-httpyac-4.10.2/dist/extension.js:1:4702]()) at Generator.next () at s ([/home/ivo/.vscode-server/extensions/anweber.vscode-httpyac-4.10.2/dist/extension.js:1:3164]()) at runMicrotasks () at processTicksAndRejections (internal[/process/task_queues.js:93:5]())

When doing the same request and test with the command line the chai test passes as expected.

package.json:

{
  "dependencies": {
    "chai": "^4.3.6",
    "chai-json-schema": "^1.5.1"
  }
}

let me know if you need more information :v:

AnWeber commented 2 years ago

You need to export a promise. All promises are resolved, before next Step in Request is executed.

{{
  async function wait(){
    const date = new Date();
    await sleep(10000);
    const afterDate = new Date();
    return afterDate.getTime() - date.getTime();
  }
  exports.wait = wait();
}}
GET https://httpbin.org/anything?delay={{wait}} HTTP/1.1

The difference is probably timing and a different behavior in handling the response. The extension does not terminate after execution. For this reason I have to be careful not to use too much memory when saving the response in the history. For this reason after execution/display the body is stored in a file and removed from the response object (=> undefined). The solution is as mentioned to wait for the execution of the tests. In CLI the test is probably executed just within the runtime of the program and therefore no error occurs here. Presumably this error occurs, however, if one extends the waiting time still further by means of setTimeout/ sleep.

IvoLeist commented 2 years ago

Thank you for the quick reply :) If I have understood it correctly it should be implemented like below, right?

POST {{host}}/test
Authorization: openid password
Content-Type: application/json
{"id":1,"value","hello_world"}
{{
  async function wait(){
    await sleep(10000);
    return chai.expect(response.parsedBody).to.be.jsonSchema(schema);
  }
  test.status(200)
  test.hasResponseBody()
  var chai = require('chai');
  chai.use(require('chai-json-schema'))
  let schema = require('./schemas.json').test;
  exports.wait = wait();
}}

However, unfortunately it runs into the same error (but different traceback)

AssertionError: schema: expected undefined to be truthy at Proxy. ([node_modules/chai-json-schema/index.js:132:16]()) at Proxy.methodWrapper ([node_modules/chai/lib/chai/utils/addMethod.js:57:25]()) at wait ([test.http:12:51]()) at i2 ([/home/ivo/.vscode-server/extensions/anweber.vscode-httpyac-4.10.2/dist/extension.js:128:91092]()) at Th ([/home/ivo/.vscode-server/extensions/anweber.vscode-httpyac-4.10.2/dist/extension.js:129:30430]())

AnWeber commented 2 years ago

Checking for JSON schema does not return a Promise, for this reason the export is not needed. One thought would be to use Console.info to output the object in the output of VSCode. Possibly use @response to change the time of execution (events). I will test it tomorrow.

AnWeber commented 2 years ago

I have now generated my own example and it works fine here. However, I have changed the call to require('schemas.json').test. Is this the error? If not, can you please change my example so that it also generates the error. Thanks.

test.http

GET https://httpbin.org/anything

{{
  const chai = require('chai');
  chai.use(require('chai-json-schema'))
  let schema = require('./schemas.json');  
  test.status(200)
  test.hasResponseBody()
  test('schema is valid', () => {
    chai.expect(response.parsedBody).to.be.jsonSchema(schema);
  });
}}

schemas.json

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "type": "object",
  "properties": {
    "args": {
      "type": "object"
    },
    "data": {
      "type": "string"
    },
    "files": {
      "type": "object"
    },
    "form": {
      "type": "object"
    },
    "headers": {
      "type": "object",
      "properties": {
        "Accept": {
          "type": "string"
        },
        "Accept-Encoding": {
          "type": "string"
        },
        "Host": {
          "type": "string"
        },
        "User-Agent": {
          "type": "string"
        },
        "X-Amzn-Trace-Id": {
          "type": "string"
        }
      },
      "required": [
        "Accept",
        "Accept-Encoding",
        "Host",
        "User-Agent",
        "X-Amzn-Trace-Id"
      ]
    },
    "json": {
      "type": "null"
    },
    "method": {
      "type": "string"
    },
    "origin": {
      "type": "string"
    },
    "url": {
      "type": "string"
    }
  },
  "required": [
    "args",
    "data",
    "files",
    "form",
    "headers",
    "method",
    "origin",
    "url"
  ]
}

IvoLeist commented 2 years ago

Good catch! It has indeed something to do with the dot notation (require('schemas.json').test) So right now I have all possible schemas in one file and select the one needed with ".schemaKey" However, having one schema per one file resolves that issue.

Weird that it works when calling httpyac from the command line, no?

So my workaround (till this is fixed ?) is to split up all our json schemas -> one schema per file

AnWeber commented 2 years ago

Can you change my Schema identical to your original Schema file? Maybe I can track the error. Cli and VSCode are minimally different. Main difference is possibly the NodeJS version. VSCode uses 14.x.

IvoLeist commented 2 years ago

The schema below should throw the error. One weird (interesting) observation: The dot notation works for the first key but fails for any other:

# test.http
GET https://httpbin.org/anything
{{
  const chai = require('chai');
  chai.use(require('chai-json-schema'))
  let schema = require('./jsonschemas/debug.json').testKey;
  let schema2 = require('./jsonschemas/debug.json').testKey2;  
  test.status(200)
  test.hasResponseBody()
  test('schema is valid', () => {
    chai.expect(response.parsedBody).to.be.jsonSchema(schema); //should work
  });
  test('schema is valid', () => {
    chai.expect(response.parsedBody).to.be.jsonSchema(schema2); // should fail
  });
}}

schema.json

{
  "testKey": {
    "$schema": "http://json-schema.org/draft-04/schema#",
    "type": "object",
    "properties": {
      "args": {
        "type": "object"
      },
      "data": {
        "type": "string"
      },
      "files": {
        "type": "object"
      },
      "form": {
        "type": "object"
      },
      "headers": {
        "type": "object",
        "properties": {
          "Accept": {
            "type": "string"
          },
          "Accept-Encoding": {
            "type": "string"
          },
          "Host": {
            "type": "string"
          },
          "User-Agent": {
            "type": "string"
          },
          "X-Amzn-Trace-Id": {
            "type": "string"
          }
        },
        "required": [
          "Accept",
          "Accept-Encoding",
          "Host",
          "User-Agent",
          "X-Amzn-Trace-Id"
        ]
      },
      "json": {
        "type": "null"
      },
      "method": {
        "type": "string"
      },
      "origin": {
        "type": "string"
      },
      "url": {
        "type": "string"
      }
    },
    "required": [
      "args",
      "data",
      "files",
      "form",
      "headers",
      "method",
      "origin",
      "url"
    ]
  },
  "testKey2": {
    "$schema": "http://json-schema.org/draft-04/schema#",
    "type": "object",
    "properties": {
      "args": {
        "type": "object"
      },
      "data": {
        "type": "string"
      },
      "files": {
        "type": "object"
      },
      "form": {
        "type": "object"
      },
      "headers": {
        "type": "object",
        "properties": {
          "Accept": {
            "type": "string"
          },
          "Accept-Encoding": {
            "type": "string"
          },
          "Host": {
            "type": "string"
          },
          "User-Agent": {
            "type": "string"
          },
          "X-Amzn-Trace-Id": {
            "type": "string"
          }
        },
        "required": [
          "Accept",
          "Accept-Encoding",
          "Host",
          "User-Agent",
          "X-Amzn-Trace-Id"
        ]
      },
      "json": {
        "type": "null"
      },
      "method": {
        "type": "string"
      },
      "origin": {
        "type": "string"
      },
      "url": {
        "type": "string"
      }
    },
    "required": [
      "args",
      "data",
      "files",
      "form",
      "headers",
      "method",
      "origin",
      "url"
    ]
  }
}

Good luck with reproducing/fixing it :crossed_fingers: :v:

AnWeber commented 2 years ago

Unfortunately, I can't reproduce the error with the example. It does not occur with me. I have also tested it with HttpBook. using Linux/WSL. schema jsonschema.zip

AnWeber / httpbook

how to wait for response.parsedBody promise to be resolved? #46