AnWeber / vscode-httpyac

Quickly and easily send REST, Soap, GraphQL, GRPC, MQTT and WebSocket requests directly within Visual Studio Code
https://marketplace.visualstudio.com/items?itemName=anweber.vscode-httpyac
MIT License
245 stars 23 forks source link

Dependency check problems in v6.16.3 and v6.16.4 #339

Open alekdavis opened 2 weeks ago

alekdavis commented 2 weeks ago

The following code illustrates nested dependency. Test A completes request, but fails assertion. Test B depends on some condition of test A and test C depends on some condition of test B:

### Test A
# @name a
POST https://httpbin.org/anything

?? status == 201

### Test B
# @name b
# @ref a
{{
  exports.$cancel = (aResponse?.statusCode !== 201);
}}
PATCH https://httpbin.org/anything

?? status == 200

### Test C
# @name c
# @ref b
{{
  exports.$cancel = (bResponse?.statusCode !== 200);
}}
GET https://httpbin.org/anything?b={{bResponse?.statusCode}}

?? status == 200

In , v6.16.2, the result is what you'd expect: both tests B and C get skipped:

---------------------

=== Test A ===

POST https://httpbin.org/anything
=> 200 (2224 ms, 590 B)
✖ status == 201 (AssertionError [ERR_ASSERTION]: status (200) == 201)

---------------------

=== Test B ===

PATCH https://httpbin.org/anything
○ Test skipped

---------------------

=== Test C ===

GET https://httpbin.org/anything?b={{bResponse?.statusCode}}
○ Test skipped

In In , v6.16.3 and 6.16.4, test B gets skipped, but the pre-request script of test C does not get processed due to the ReferenceError: bResponse is not defined error. So, the code to set the $cancel variable does not complete and test C is executed:

---------------------

=== Test A ===

POST https://httpbin.org/anything
=> 200 (1528 ms, 590 B)
✖ status == 201 (AssertionError [ERR_ASSERTION]: status (200) == 201)

---------------------

=== Test B ===

PATCH https://httpbin.org/anything
○ Test skipped

---------------------

=== Test C ===

GET https://httpbin.org/anything?b={{bResponse?.statusCode}}
✖ bResponse is not defined (ReferenceError: bResponse is not defined - Object.userJS (c:\PROJECTS\Test\HttpYacTest\test.http:21:22)

However, test C shows everywhere as successfully executed:

image

I think the test dependency logic in v6.16.2 was mostly correct. The only issue I seem to see with v6.16.2 is that it would try to invoke a failed test multiple times during a test run (for every @ref). But v6.16.3 and v6.16.4 do not work with my logic at all unless I'm doing something wrong.

alekdavis commented 2 weeks ago

The following script could be helpful for testing:

### A
# @name a
POST https://httpbin.org/anything

?? status == 201

### B
# @name b
# @ref a
{{
  exports.$cancel = (aResponse?.statusCode !== 201);
}}
PATCH https://httpbin.org/anything

?? status == 200

### C
# @name c
# @ref b
{{
  exports.$cancel = (bResponse?.statusCode !== 200);
}}
GET https://httpbin.org/anything?b={{bResponse?.statusCode}}

?? status == 200

### D
# @name d
# @ref b
{{
  exports.$cancel = (bResponse?.statusCode !== 200);
}}
GET https://httpbin.org/json?b={{bResponse?.statusCode}}

?? status == 200

It should produce test results:

---------------------

=== A ===

POST https://httpbin.org/anything
=> 200 (545 ms, 590 B)
✖ status == 201 (AssertionError [ERR_ASSERTION]: status (200) == 201)

---------------------

=== B ===

PATCH https://httpbin.org/anything
○ Test skipped

---------------------

=== C ===

GET https://httpbin.org/anything?b={{bResponse?.statusCode}}
○ Test skipped

---------------------

=== D ===

GET https://httpbin.org/json?b={{bResponse?.statusCode}}
○ Test skipped