azagniotov / stubby4j

HTTP/1.1, HTTP/2 and WebSockets stub server for stubbing distributed web services in Docker and non-containerized environments for integration and contract testing
https://www.stubby4j.com
Apache License 2.0
356 stars 103 forks source link

How to match "a token not in the payload" with RegEx pattern? #171

Closed harrysun2006 closed 3 years ago

harrysun2006 commented 3 years ago

Hi Alex,

Also I hit an issue on the RegEx pattern.

I defined a RegEx pattern: .*([^"Declaration"\:]).*"firstName":"(.*?)".*"capSubProductCode":"(\w+)".*"natureOfProduct":"(\w+)".*

hoping it would match to the payload like:

[{"application":{"Address":[{"addressId":8,"customerId":1,"orderItemId":3,"addressType":"STMT","city":"DOCKLANDS","country":"AU","postcode":"3008","state":"VIC","streetName":"Collins","streetNumber":"691","streetType":"ST"},{"addressId":7,"customerId":1,"addressType":"CMAL","city":"DOCKLANDS","country":"AU","postcode":"3008","state":"VIC","streetName":"Collins","streetNumber":"691","streetType":"ST"},{"addressId":6,"customerId":1,"addressType":"CRES","city":"DOCKLANDS","country":"AU","postcode":"3008","state":"VIC","streetName":"Collins","streetNumber":"691","streetType":"ST"}],"AppCustRelationship":[{"customerId":1,"applicationId":2,"relationshipType":"POW"}],"Application":[{"applicationId":2,"applicationSigned":false,"applicationSource":"sola","applicationSourceCode":"QMV","applicationSourceCountry":"AU","applicationVersion":1,"basketId":1,"bsb":"","channel":"I","createdDate":"2021-01-21T09:57:03+11:00","currencyCode":"AUD","modifiedBy":"Apply User","orderId":1000000003}],"CRNRequest":[{"regId":5,"customerId":1,"orderItemId":3,"accessLevel":"Full","createNewCRN":true,"customerClass":"CNE"}],"Contact":[{"contactId":4,"customerId":1,"contactType":"E","email":"test@anz.com"}],"CustOrderItemRelationship":[{"customerId":1,"orderItemId":3,"custAcctRelationship":"SOL","orderItemFlag":"N"}],"Customer":[{"customerId":1,"acceptMarketing":true,"AUTaxResidentOnly":false,"customerType":"IND","depositCustomerType":"STD","existingCustomer":false,"fullName":"Failure Test","ipAddress":"","privacyConsent":true,"verificationStatus":"Verified"}],"Deposit":[{"orderItemId":3,"ATOType":"I","numberOfSignatories":1,"transactionAccountType":"IND"}],"Individual":[{"customerId":1,"dateOfBirth":"1981-01-01T00:00:00Z","firstName":"Failure","gender":"M","lastName":"Test","middleName":"","title":"mr"}],"OrderItem":[{"orderItemId":3,"capProductCode":"DDA","capSubProductCode":"ED","eStatementEmail":"test@anz.com","natureOfProduct":"primary"}]}}]

It passed the online Java Regular Expression Tester. But unfortunately, the stubby4j does not like it. I see following message in the stubby log:

Failed to match on POST BODY [.*([^"Declaration"\:]).*"firstName":"(.*?)".*"capSubProductCode":"(\w+)".*"natureOfProduct":"(\w+)".*] WITH [{"application":{"Address":[{"addressId":8,"customerId":1,"orderItemId":3,"addressType":"STMT","city":"DOCKLANDS","country":"AU","postcode":"3008","state":"VIC","streetName":"Collins","streetNumber":"691","streetType":"ST"},{"addressId":7,"customerId":1,"addressType":"CMAL","city":"DOCKLANDS","country":"AU","postcode":"3008","state":"VIC","streetName":"Collins","streetNumber":"691","streetType":"ST"},{"addressId":6,"customerId":1,"addressType":"CRES","city":"DOCKLANDS","country":"AU","postcode":"3008","state":"VIC","streetName":"Collins","streetNumber":"691","streetType":"ST"}],"AppCustRelationship":[{"customerId":1,"applicationId":2,"relationshipType":"POW"}],"Application":[{"applicationId":2,"applicationSigned":false,"applicationSource":"sola","applicationSourceCode":"QMV","applicationSourceCountry":"AU","applicationVersion":1,"basketId":1,"bsb":"","channel":"I","createdDate":"2021-01-21T09:57:03+11:00","currencyCode":"AUD","modifiedBy":"Apply User","orderId":1000000003}],"CRNRequest":[{"regId":5,"customerId":1,"orderItemId":3,"accessLevel":"Full","createNewCRN":true,"customerClass":"CNE"}],"Contact":[{"contactId":4,"customerId":1,"contactType":"E","email":"test@anz.com"}],"CustOrderItemRelationship":[{"customerId":1,"orderItemId":3,"custAcctRelationship":"SOL","orderItemFlag":"N"}],"Customer":[{"customerId":1,"acceptMarketing":true,"AUTaxResidentOnly":false,"customerType":"IND","depositCustomerType":"STD","existingCustomer":false,"fullName":"Failure Test","ipAddress":"","privacyConsent":true,"verificationStatus":"Verified"}],"Deposit":[{"orderItemId":3,"ATOType":"I","numberOfSignatories":1,"transactionAccountType":"IND"}],"Individual":[{"customerId":1,"dateOfBirth":"1981-01-01T00:00:00Z","firstName":"Failure","gender":"M","lastName":"Test","middleName":"","title":"mr"}],"OrderItem":[{"orderItemId":3,"capProductCode":"DDA","capSubProductCode":"ED","eStatementEmail":"test@anz.com","natureOfProduct":"primary"}]}}]

I know it's tricky to define a bunch of RegEx patterns to match various requests with same URL and method. I'm a bit struggling on this and much appreciated to any suggestions.

Thanks, Harry

azagniotov commented 3 years ago

Hi Harry,

I quickly wrote a unit test (just for the sake of sanity check) and indeed your provided Regex matches your provided payload:

 @Test
    public void quickTest() throws Exception {
        final Pattern originalPattern = Pattern.compile(".*([^\"Declaration\"\\:]).*\"firstName\":\"(.*?)\".*\"capSubProductCode\":\"(\\w+)\".*\"natureOfProduct\":\"(\\w+)\".*");

        final String candidate = "[{\"application\":{\"Address\":[{\"addressId\":8,\"customerId\":1,\"orderItemId\":3,\"addressType\":\"STMT\",\"city\":\"DOCKLANDS\",\"country\":\"AU\",\"postcode\":\"3008\",\"state\":\"VIC\",\"streetName\":\"Collins\",\"streetNumber\":\"691\",\"streetType\":\"ST\"},{\"addressId\":7,\"customerId\":1,\"addressType\":\"CMAL\",\"city\":\"DOCKLANDS\",\"country\":\"AU\",\"postcode\":\"3008\",\"state\":\"VIC\",\"streetName\":\"Collins\",\"streetNumber\":\"691\",\"streetType\":\"ST\"},{\"addressId\":6,\"customerId\":1,\"addressType\":\"CRES\",\"city\":\"DOCKLANDS\",\"country\":\"AU\",\"postcode\":\"3008\",\"state\":\"VIC\",\"streetName\":\"Collins\",\"streetNumber\":\"691\",\"streetType\":\"ST\"}],\"AppCustRelationship\":[{\"customerId\":1,\"applicationId\":2,\"relationshipType\":\"POW\"}],\"Application\":[{\"applicationId\":2,\"applicationSigned\":false,\"applicationSource\":\"sola\",\"applicationSourceCode\":\"QMV\",\"applicationSourceCountry\":\"AU\",\"applicationVersion\":1,\"basketId\":1,\"bsb\":\"\",\"channel\":\"I\",\"createdDate\":\"2021-01-21T09:57:03+11:00\",\"currencyCode\":\"AUD\",\"modifiedBy\":\"Apply User\",\"orderId\":1000000003}],\"CRNRequest\":[{\"regId\":5,\"customerId\":1,\"orderItemId\":3,\"accessLevel\":\"Full\",\"createNewCRN\":true,\"customerClass\":\"CNE\"}],\"Contact\":[{\"contactId\":4,\"customerId\":1,\"contactType\":\"E\",\"email\":\"test@anz.com\"}],\"CustOrderItemRelationship\":[{\"customerId\":1,\"orderItemId\":3,\"custAcctRelationship\":\"SOL\",\"orderItemFlag\":\"N\"}],\"Customer\":[{\"customerId\":1,\"acceptMarketing\":true,\"AUTaxResidentOnly\":false,\"customerType\":\"IND\",\"depositCustomerType\":\"STD\",\"existingCustomer\":false,\"fullName\":\"Failure Test\",\"ipAddress\":\"\",\"privacyConsent\":true,\"verificationStatus\":\"Verified\"}],\"Deposit\":[{\"orderItemId\":3,\"ATOType\":\"I\",\"numberOfSignatories\":1,\"transactionAccountType\":\"IND\"}],\"Individual\":[{\"customerId\":1,\"dateOfBirth\":\"1981-01-01T00:00:00Z\",\"firstName\":\"Failure\",\"gender\":\"M\",\"lastName\":\"Test\",\"middleName\":\"\",\"title\":\"mr\"}],\"OrderItem\":[{\"orderItemId\":3,\"capProductCode\":\"DDA\",\"capSubProductCode\":\"ED\",\"eStatementEmail\":\"test@anz.com\",\"natureOfProduct\":\"primary\"}]}}]";

        final boolean isMatch = originalPattern.matcher(candidate).matches();
        Assert.assertTrue(isMatch); // 'isMatch' is true
    }

I will look deeper to understand what is happening inside stubby4j

azagniotov commented 3 years ago

Harry, I have identified a bug which resulted in the current issue: I was escaping regex pattern characters in stubbed values unnecessarily, which caused that regex matching failures.

azagniotov commented 3 years ago

Tested also by making a POST to the standalone stubby4j jar using ApiKitchen. Works as expected. I will release a new stubby4j version in order to unblock you

azagniotov commented 3 years ago

Version v7.1.2 has been pushed to Maven Central

harrysun2006 commented 3 years ago

Thanks Alex.

harrysun2006 commented 3 years ago

It's tested and working well in v7.1.3.

azagniotov commented 3 years ago

Great to hear, Harry. I will close the current issue