search

apache / jmeter

Apache JMeter open-source load testing tool for analyzing and measuring the performance of a variety of services
https://jmeter.apache.org/
Apache License 2.0
8.26k stars 2.09k forks source link

Parameterized and non parametrized strings handle \ differently, so HTTP request body with \ sends a completely different value when ${..} added #5035

Open asfimport opened 5 years ago

asfimport commented 5 years ago

utostan (Bug 63255): While attempting to create a load test for a client, a post request to a critical endpoint failed due to JMeter converting "\\" to "\" in the post's body data. After the conversion, JMeter sends an invalid JSON to the server and it responds with 400 Bad Request.

The client has a really weird (bad) endpoint where it accepts json in json in json which seems to throw off JMeter parser.

Body Data in HTTP Request (POST):

{"order":"{\"id\":0,\"order_date\":\"3/8/2019\",\"order_data\":{\"items\":[{\"id\":1876318,\"quantity\":1,\"name\":\"Air Filter\",\"price\":8.54,\"retail_price\":8.54,\"msrp\":0.0,\"price_level\":{\"price_level_id\":1,\"price_level\":\"RETAIL\",\"retail_multipler\":1.00,\"description\":\"Regular retail\",\"price_type\":{\"price_type_id\":1,\"price_type\":\"Regular\",\"label\":\"\"}},\"is_hazmat\":false,\"is_accessory\":false,\"oversize_category_id\":null,\"oem_number\":\"W10311524\",\"image_url\":\"https://www.redacted.com/dbImages/i/00004039/Air-Filter-W10311524-01215703.jpg\",\"quantity_by_dc\":[{\"id\":1,\"quantity\":0},{\"id\":2,\"quantity\":0}],\"repair_time\":null,\"repair_savings\":null,\"skill_level\":1,\"labor_hours\":null,\"prodcut_categories\":[{\"id\":1,\"name\":\"Appliances\"}]}],\"email\":\"${email}\",\"billing_address\":{\"first_name\":\"${first_name}\",\"last_name\":\"${last_name}\",\"street1\":\"2600 S River Rd\",\"street2\":\"\",\"city\":\"Des Plaines\",\"state\":\"IL\",\"country\":\"US\",\"zipcode\":\"60018\",\"phone\":\"1231231234\",\"ext\":\"\",\"business_name\":\"\",\"is_business_address\":false,\"is_po_box\":false},\"shipping_address\":{\"first_name\":\"${first_name}\",\"last_name\":\"${last_name}\",\"street1\":\"2600 S River Rd\",\"street2\":\"\",\"city\":\"Des Plaines\",\"state\":\"IL\",\"country\":\"US\",\"zipcode\":\"60018\",\"phone\":\"1231231234\",\"ext\":\"\",\"business_name\":\"\",\"is_business_address\":false,\"is_po_box\":false},\"shipping_choice\":{\"shipping_option_id\":1,\"ship_to_po_box\":false,\"is_saturday_delivery\":false,\"ship_complete\":true,\"is_home_business\":false},\"total\":15.71,\"discount\":0.0,\"subtotal\":8.54,\"shipping_charge\":6.25,\"offshore_surcharge\":0.0,\"saturday_shipping_surcharge\":0.0,\"saturday_shipping_surcharge_for_ship_method\":15.00,\"oversize_surcharge\":0.0,\"weight_surcharge\":0.0,\"international_processing_fee\":0.0,\"sales_tax\":0.92,\"sales_tax_label\":\"Estimated Sales Tax\",\"purchase_order_number\":null,\"promotion_code\":\"\",\"device_data\":\"{\\"device_session_id\\":\\"${sessionId}\\",\\"fraud_merchant_id\\":\\"600000\\"}\",\"is_promo_code_valid\":false},\"sop_number\":\"\",\"security_code\":null,\"rc_order_form\":\"\",\"o_checksum\":\"${o_checksum}\",\"rc_checksum\":\"${rc_checksum}\"}","payment":{"payment_type_id":0,"credit_card":{"nonce":"${creditCardToken}","expiration_month":"03","expiration_year":2019},"paypal":null}}

Request Body in View Results Tree for the above request:

{"order":"{\"id\":0,\"order_date\":\"3/8/2019\",\"order_data\":{\"items\":[{\"id\":1876318,\"quantity\":1,\"name\":\"Air Filter\",\"price\":8.54,\"retail_price\":8.54,\"msrp\":0.0,\"price_level\":{\"price_level_id\":1,\"price_level\":\"RETAIL\",\"retail_multipler\":1.00,\"description\":\"Regular retail\",\"price_type\":{\"price_type_id\":1,\"price_type\":\"Regular\",\"label\":\"\"}},\"is_hazmat\":false,\"is_accessory\":false,\"oversize_category_id\":null,\"oem_number\":\"W10311524\",\"image_url\":\"https://www.redacted.com/dbImages/i/00004039/Air-Filter-W10311524-01215703.jpg\",\"quantity_by_dc\":[{\"id\":1,\"quantity\":0},{\"id\":2,\"quantity\":0}],\"repair_time\":null,\"repair_savings\":null,\"skill_level\":1,\"labor_hours\":null,\"prodcut_categories\":[{\"id\":1,\"name\":\"Appliances\"}]}],\"email\":\"test12@example.com\",\"billing_address\":{\"first_name\":\"John\",\"last_name\":\"Doe\",\"street1\":\"2600 S River Rd\",\"street2\":\"\",\"city\":\"Des Plaines\",\"state\":\"IL\",\"country\":\"US\",\"zipcode\":\"60018\",\"phone\":\"1231231234\",\"ext\":\"\",\"business_name\":\"\",\"is_business_address\":false,\"is_po_box\":false},\"shipping_address\":{\"first_name\":\"John\",\"last_name\":\"Doe\",\"street1\":\"2600 S River Rd\",\"street2\":\"\",\"city\":\"Des Plaines\",\"state\":\"IL\",\"country\":\"US\",\"zipcode\":\"60018\",\"phone\":\"1231231234\",\"ext\":\"\",\"business_name\":\"\",\"is_business_address\":false,\"is_po_box\":false},\"shipping_choice\":{\"shipping_option_id\":1,\"ship_to_po_box\":false,\"is_saturday_delivery\":false,\"ship_complete\":true,\"is_home_business\":false},\"total\":15.71,\"discount\":0.0,\"subtotal\":8.54,\"shipping_charge\":6.25,\"offshore_surcharge\":0.0,\"saturday_shipping_surcharge\":0.0,\"saturday_shipping_surcharge_for_ship_method\":15.00,\"oversize_surcharge\":0.0,\"weight_surcharge\":0.0,\"international_processing_fee\":0.0,\"sales_tax\":0.92,\"sales_tax_label\":\"Estimated Sales Tax\",\"purchase_order_number\":null,\"promotion_code\":\"\",\"device_data\":\"{\"device_session_id\":\"6E0E291D4A295930E103D3F89D9DA58F\",\"fraud_merchant_id\":\"600000\"}\",\"is_promo_code_valid\":false},\"sop_number\":\"\",\"security_code\":null,\"rc_order_form\":\"\",\"o_checksum\":\"7AB4149F2F0E47E9A3F7546FB0E21ABD\",\"rc_checksum\":\"703BE7B71AA016C82132378323E98B9E\"}","payment":{"payment_type_id":0,"credit_card":{"nonce":"tokencc_bd_v3q2pq_nr8xwy_w8k5dt_qcvdvw_yxy","expiration_month":"03","expiration_year":2019},"paypal":null}}

Seems like JMeter converts \\\"device_session_id\\\" to \\"device_session_id\\" (with the latter being invalid JSON).

Please let me know if you need more information. Sorry about the big wall of text.

Thanks, uTosTan

Votes in Bugzilla: 1 Severity: major OS: All

asfimport commented 5 years ago

justin (migrated from Bugzilla): Do you mind providing a simple .jmx file or Results Tree script so I can replicate this myself? I feel I can help fix this if I'm able to replicate what you are experiencing. Having trouble understanding based on your post alone

asfimport commented 4 years ago

avam0nst3r (migrated from Bugzilla): simple JMX, proving that POST request body is being sent "as is", until you add any ${var}. Then it start "fixing" the double-backslashes, and m.b. other chars too.

Created attachment backslash-wrong-escaping.jmx: incorrect multiple-backslash processing in request body

backslash-wrong-escaping.jmx ````xml false true false continue false 1 1 1 false true true false {"field1":"value\\\value","field":"value","fieldWithBackslash":"value\\value\\\Avalue \\\\Avalue",} = localhost POST true false true false true false {"field1":"value\\\value","field":"value${__threadNum}","fieldWithBackslash":"value\\value\\\Avalue \\\\Avalue",} = localhost POST true false true false false saveConfig true true true true true true true false true true false false false true false false false true 0 true true true true true true ````
asfimport commented 4 years ago

avam0nst3r (migrated from Bugzilla): simple jmx attached, just check what it actually tries to send in tree view for both requests

The post request body {"field","value\\value"} is sent exactly as {"field","value\\value"}

While {"field${__threadNum}","value\\value"} is sent as {"field1","value\value"}

asfimport commented 4 years ago

avam0nst3r (migrated from Bugzilla): Jmeter 5.3 still has this problem

asfimport commented 4 years ago

Jerome (migrated from Bugzilla): Same issue here. Apparently, the problematic code is within FunctionParser.compileString(String value) method.

As stated in javadoc: "Removes escapes from '$', ',' and '\'."

This is why it's working perfectly fine when no variable is used.

asfimport commented 4 years ago

Jerome (migrated from Bugzilla): Pull request to fix the issue: https://github.com/apache/jmeter/pull/616

I'm not sure of the overall impact of this modification. It now allows multiple '\' without removing any of them, as long as it's not in front of a $ or a ,.

wxg0103 commented 1 year ago

This problem still exists in jmeter5.5, where can I fix it, or can you mention an alternative method?

vlsi commented 1 year ago

Non parameterized strings do not process \ as a special character, while parameterized strings uses a special character to escape $. For instance, in case the user wants to send $ they use \$.

That is the "proper" way of adding parameterization is to double every \ before adding $. Of course it is not easy, and we could add a button to escape/unescape strings. However, that workaround shound help.

An alternative option would be treating only ${ as a special sequence outside of the parameterization. In that case, there will be no need adding slashes, howewer sending ${ would require something like ${$}{.

Apparently, it would help if we implement syntax highlight. Then the users will see that \\ has a special meaning in case parameterization is used.

The issue is similar to https://youtrack.jetbrains.com/issue/KT-2425/Provide-a-way-for-escaping-the-dollar-sign-symbol-in-multiline-strings-and-string-templates

karhow commented 6 months ago

This problem still exists, anybody can solve it ? @vlsi

karhow commented 6 months ago

This problem still exists, anybody can solve it ? @vlsi

if post body string contains ${param} and\\\"，after ${param} replace by true parameter, the \\\"will be replace by \\"too which is error in json string.