karatelabs / karate

Test Automation Made Simple
https://karatelabs.github.io/karate
MIT License
8.27k stars 1.95k forks source link

MockSpringMvcServlet not respecting @ControllerAdvice NoHandlerFoundException #710

Closed jesimone57 closed 4 years ago

jesimone57 commented 5 years ago

The code to demonstrate this issue is available as a github project here:

https://github.com/jesimone57/spring_boot_rest_examples

Set up mvn clean compile test

All test pass except for one karate test which tests an endpoint which does not exist for a proper error response. The failing test is ...

Results :

Tests in error: 
  nourl.feature:11 - path: $['content-type'][0], actual: null, expected: 'application/json', reason: actual value is not a string

Tests run: 95, Failures: 0, Errors: 1, Skipped: 1

Description: I just added the Karate mock servlet dependency to my project and followed the instructions for making use of it to test all the karate features without the need to run a spring-boot server. All the feature tests passed successfully (49 karate test pass) except for 1.

What failed is a test that checks for a NoHandlerFoundException when an invalid URL is given. The expected response is

{
   method: "GET",
   status_code: 404,
   uri_path: "/hello",
   error_message: "The URL you have reached is not in service at this time",
   errors: [ ]
}

This test, of necessity, would depend on the following 2 application properties being set in the test environment (application.properties file)

# exception handling (see GlobalExceptionHandler handleNoHandlerFoundException method)
spring.mvc.throw-exception-if-no-handler-found=true
spring.resources.add-mappings=false

This is according to the Spring documentation for handling NoHandlerFoundException.

The junit 4 version of this @SpringBootTest passes successfully and may be found in HelloWorldControllerExceptionTest. We get the expected response as given above.

However, the nourl.feature file given as

Feature:  No URLfound proper error response

  Background:
    * url 'http://localhost:8080'
    * configure lowerCaseResponseHeaders = true

  Scenario: Invalid URL response
    Given path '/hello'
    When method get
    Then status 404
    And match header content-type contains 'application/json'
    And match header content-type contains 'charset=utf-8'
    And match response.status_code == 404
    And match response.method == 'GET'
    And match response.error_message == 'The URL you have reached is not in service at this time'
    And match response.errors == '#[]'

fails because no NoHandlerFoundException is ever thrown by the mock servlet container upon a request for a non-existent URL. The code for this handler is found in @ControllerAdvice annotated class GlobalExceptionHandler.

I added the following

@Configuration
@EnableAutoConfiguration
@PropertySource("classpath:application.properties")

as specified in the MockConfig from the karate mock servlet demo, but still fails.

What am I missing to make this work. Is this a karate bug or limitation.
Thanks.

ptrthomas commented 5 years ago

yes. it may be unrealistic to expect the fake dispatcher servlet to mimic all the edge cases of spring boot. we will take a look at this - but you may be better off ignoring this test in "mock servlet" mode using a tag. marking this ticket as "help wanted"

ptrthomas commented 5 years ago

moving to project roadmap board: https://github.com/intuit/karate/projects/3#card-22529274

Nishant-sehgal commented 5 years ago

@ptrthomas this is also an open issue i guess? shall i pick this up?

ptrthomas commented 5 years ago

@Nishant-sehgal yes, please go ahead

ptrthomas commented 4 years ago

released 0.9.5

jesimone57 commented 4 years ago

verified fix is good in 0.9.5 after changing the MockSpringMvcServlet file to reflect the changes in the Spring MVC Dispatcher Servlet example link in the MockServlet documentation to this:

https://intuit.github.io/karate/karate-mock-servlet/src/test/java/demo/MockSpringMvcServlet.java Thanks @Nishant-sehgal for making this fix.