Add __digest function to provide computing of Hashes (SHA-XXX, MDX)

[asfimport]

orimarko (Bug 61724): Add sha1 and sha256 functions to JMeter

Created attachment patch.patch: patch

OS: All

Duplicates:

• https://github.com/apache/jmeter/issues/4590

[asfimport]

orimarko (migrated from Bugzilla): Attached Patch with added two functions and tests

Created attachment patch.patch: Patch with added two functions and tests

[asfimport]

orimarko (migrated from Bugzilla): Add patch with 2 functions and tests

Created attachment patch.patch: Add patch with 2 functions and tests

[asfimport]

@pmouawad (migrated from Bugzilla): Thank you for your contribution, I have few remarks:

• Why salt is not also there for SHA1 ?
• I think 1 more generic function is enough
• It would be better to have something like __digest(algorithm, string to digest, salt (optional),uppercase (optional defaults to false), varName (optional)). This would allow more generic algorithm addition
• synchronized does not seem to be needed for me

[asfimport]

@pmouawad (migrated from Bugzilla): Also:

• patch file should be built from jmeter folder (the one that contains src)
• if possible, documentation functions.xml should be updated, I'll do it if you don't have time
• Apache License 2 Header should be added to java files

[asfimport]

orimarko (migrated from Bugzilla): Add patch with digest functions and tests plus function xdocs

Created attachment patchxdocs.patch: Add patch with digest function xdocs

patchxdocs.patch

````diff Index: usermanual/functions.xml =================================================================== --- usermanual/functions.xml (revision 1813248) +++ usermanual/functions.xml (working copy) @@ -1565,7 +1565,25 @@ The name of the variable to set. - + + +

The digest function returns a encrypted value in the specific SHA algorithm with the optional salt, upper case and variable name.

+ + + + The algorithm to be used to encrypt + For possible algorithms See MessageDigest in https://docs.oracle.com/javase/7/docs/technotes/guides/security/StandardNames.html +

• SHA-1
+
• SHA-256
+
• SHA-384
+
• SHA-512

+ + The String that will be encrypted + Salt to be added to string + Result will be in lower case by default. Choose true to upper case results. + The name of the variable to set. + + ````

[asfimport]

orimarko (migrated from Bugzilla): Created attachment patch.patch: Patch with new Digest function and tests

patch.patch

````diff Index: core/org/apache/jmeter/functions/AbstractFunction.java =================================================================== --- core/org/apache/jmeter/functions/AbstractFunction.java (revision 1813248) +++ core/org/apache/jmeter/functions/AbstractFunction.java (working copy) @@ -20,6 +20,7 @@ import java.util.Collection; +import org.apache.commons.lang3.StringUtils; import org.apache.jmeter.engine.util.CompoundVariable; import org.apache.jmeter.samplers.SampleResult; import org.apache.jmeter.samplers.Sampler; @@ -31,6 +32,8 @@ * Provides common methods for all functions */ public abstract class AbstractFunction implements Function { + + private static final String SHOULD_DO_UPPERCASE = "true"; /** *

@@ -140,4 +143,40 @@ ); } } + + /** + * Utility method to add variable value by variable name + * + * @param variableName variable name to put + * @param value variable value to put + */ + protected void addVariableValue(String value, CompoundVariable[] values, int index) { + if (values.length > index) { + String variableName = values[index].execute(); + if (StringUtils.isNotEmpty(variableName)) { + JMeterVariables vars = getVariables(); + if (vars != null) { + vars.put(variableName, value); + } + } + } + } + + + /** + * Upper case value if optional parameter value is true + * @param encodedString + * @param index + * @return + */ + protected String upperCase(String encodedString, CompoundVariable[] values, int index) { + if (values.length > index) { + String shouldUpperCase = values[index].execute(); + boolean shouldDoUpperCase = SHOULD_DO_UPPERCASE.equalsIgnoreCase(shouldUpperCase); + if (shouldDoUpperCase) { + encodedString = encodedString.toUpperCase(); + } + } + return encodedString; + } } Index: core/org/apache/jmeter/resources/messages.properties =================================================================== --- core/org/apache/jmeter/resources/messages.properties (revision 1813248) +++ core/org/apache/jmeter/resources/messages.properties (working copy) @@ -95,6 +95,7 @@ aggregate_report_stddev=Std. Dev. aggregate_report_total_label=TOTAL ajp_sampler_title=AJP/1.3 Sampler +algorithm_string=Digest algorithm als_message=Note\: The Access Log Parser is generic in design and allows you to plugin als_message2=your own parser. To do so, implement the LogParser, add the jar to the als_message3=/lib directory and enter the class in the sampler. @@ -933,6 +934,7 @@ running_test=Running test runtime_controller_title=Runtime Controller runtime_seconds=Runtime (seconds) +salt_string=Salt to be used for encoding sample_result_save_configuration=Sample Result Save Configuration sample_scope=Apply to: sample_scope_all=Main sample and sub-samples @@ -1023,6 +1025,7 @@ servername=Servername \: session_argument_name=Session Argument Name setup_thread_group_title=setUp Thread Group +sha_string=String to be encoded should_save=You should save your test plan before running it. \nIf you are using supporting data files (ie, for CSV Data Set or __StringFromFile), \nthen it is particularly important to first save your test script. \nDo you want to save your test plan first? shutdown=Shutdown simple_config_element=Simple Config Element @@ -1224,6 +1227,7 @@ update_per_iter=Update Once Per Iteration upload=File Upload upper_bound=Upper Bound +upper_case=Upper case result - default false (optional) url=URL url_config_get=GET url_config_http=HTTP Index: functions/org/apache/jmeter/functions/DigestEncode.java =================================================================== --- functions/org/apache/jmeter/functions/DigestEncode.java (revision 0) +++ functions/org/apache/jmeter/functions/DigestEncode.java (working copy) @@ -0,0 +1,114 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ +package org.apache.jmeter.functions; + +import org.apache.commons.lang3.StringUtils; +import org.apache.jmeter.engine.util.CompoundVariable; +import org.apache.jmeter.functions.AbstractFunction; +import org.apache.jmeter.functions.InvalidVariableException; +import org.apache.jmeter.samplers.SampleResult; +import org.apache.jmeter.samplers.Sampler; +import org.apache.jmeter.util.JMeterUtils; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import java.io.UnsupportedEncodingException; +import java.security.MessageDigest; +import java.security.NoSuchAlgorithmException; + +import java.util.Collection; +import java.util.LinkedList; +import java.util.List; + +/** + * + * Sha512 Encode Function Optional Upper case value and save to variable name + * + * @author orim + * + */ +public class DigestEncode extends AbstractFunction { + + private static final Logger log = LoggerFactory.getLogger(DigestEncode.class); + private static final String UTF_8 = "UTF-8"; + + /* + * The algorithm names in this section can be specified when generating an + * instance of MessageDigest: SHA-1 SHA-256 SHA-384 SHA-512 + */ + private static final List desc = new LinkedList(); + private static final String KEY = "__digest"; + + // Number of parameters expected - used to reject invalid calls + private static final int MIN_PARAMETER_COUNT = 2; + private static final int MAX_PARAMETER_COUNT = 5; + + static { + desc.add(JMeterUtils.getResString("algorithm_string")); + desc.add(JMeterUtils.getResString("sha_string")); + desc.add(JMeterUtils.getResString("salt_string")); + desc.add(JMeterUtils.getResString("upper_case")); + desc.add(JMeterUtils.getResString("function_name_paropt")); + } + + private CompoundVariable[] values; + + @Override + public synchronized String execute(SampleResult previousResult, Sampler currentSampler) throws InvalidVariableException { + String encodedString = null; + String digestAlgorithm = values[0].execute(); + String stringToEncode = values[1].execute(); + String salt = null; + if (values.length > 2) { + salt = values[2].execute(); + } + try { + MessageDigest md = MessageDigest.getInstance(digestAlgorithm); + if (StringUtils.isNotEmpty(salt)) { + md.update(salt.getBytes(UTF_8)); + } + byte[] bytes = md.digest(stringToEncode.getBytes(UTF_8)); + StringBuilder sb = new StringBuilder(); + for (int i = 0; i < bytes.length; i++) { + sb.append(Integer.toString((bytes[i] & 0xff) + 0x100, 16).substring(1)); + } + encodedString = upperCase(sb.toString(), values, 3); + addVariableValue(encodedString, values, 4); + + } catch (NoSuchAlgorithmException | UnsupportedEncodingException e) { + log.warn("Error executing SHA encryption", e); + } + return encodedString; + } + + @Override + public synchronized void setParameters(Collection parameters) throws InvalidVariableException { + checkParameterCount(parameters, MIN_PARAMETER_COUNT, MAX_PARAMETER_COUNT); + values = parameters.toArray(new CompoundVariable[parameters.size()]); + } + + @Override + public String getReferenceKey() { + return KEY; + } + + @Override + public List getArgumentDesc() { + return desc; + } +} \ No newline at end of file ````

[asfimport]

orimarko (migrated from Bugzilla): Patch with new Digest function fixed and tests

Created attachment patch.patch: Patch with new Digest function and tests

patch.patch

````diff Index: core/org/apache/jmeter/functions/AbstractFunction.java =================================================================== --- core/org/apache/jmeter/functions/AbstractFunction.java (revision 1813248) +++ core/org/apache/jmeter/functions/AbstractFunction.java (working copy) @@ -20,6 +20,7 @@ import java.util.Collection; +import org.apache.commons.lang3.StringUtils; import org.apache.jmeter.engine.util.CompoundVariable; import org.apache.jmeter.samplers.SampleResult; import org.apache.jmeter.samplers.Sampler; @@ -31,6 +32,8 @@ * Provides common methods for all functions */ public abstract class AbstractFunction implements Function { + + private static final String SHOULD_DO_UPPERCASE = "true"; /** *

@@ -140,4 +143,40 @@ ); } } + + /** + * Utility method to add variable value by variable name + * + * @param variableName variable name to put + * @param value variable value to put + */ + protected void addVariableValue(String value, CompoundVariable[] values, int index) { + if (values.length > index) { + String variableName = values[index].execute(); + if (StringUtils.isNotEmpty(variableName)) { + JMeterVariables vars = getVariables(); + if (vars != null) { + vars.put(variableName, value); + } + } + } + } + + + /** + * Upper case value if optional parameter value is true + * @param encodedString + * @param index + * @return + */ + protected String upperCase(String encodedString, CompoundVariable[] values, int index) { + if (values.length > index) { + String shouldUpperCase = values[index].execute(); + boolean shouldDoUpperCase = SHOULD_DO_UPPERCASE.equalsIgnoreCase(shouldUpperCase); + if (shouldDoUpperCase) { + encodedString = encodedString.toUpperCase(); + } + } + return encodedString; + } } Index: core/org/apache/jmeter/resources/messages.properties =================================================================== --- core/org/apache/jmeter/resources/messages.properties (revision 1813248) +++ core/org/apache/jmeter/resources/messages.properties (working copy) @@ -95,6 +95,7 @@ aggregate_report_stddev=Std. Dev. aggregate_report_total_label=TOTAL ajp_sampler_title=AJP/1.3 Sampler +algorithm_string=Digest algorithm als_message=Note\: The Access Log Parser is generic in design and allows you to plugin als_message2=your own parser. To do so, implement the LogParser, add the jar to the als_message3=/lib directory and enter the class in the sampler. @@ -933,6 +934,7 @@ running_test=Running test runtime_controller_title=Runtime Controller runtime_seconds=Runtime (seconds) +salt_string=Salt to be used for encoding sample_result_save_configuration=Sample Result Save Configuration sample_scope=Apply to: sample_scope_all=Main sample and sub-samples @@ -1023,6 +1025,7 @@ servername=Servername \: session_argument_name=Session Argument Name setup_thread_group_title=setUp Thread Group +sha_string=String to be encoded should_save=You should save your test plan before running it. \nIf you are using supporting data files (ie, for CSV Data Set or __StringFromFile), \nthen it is particularly important to first save your test script. \nDo you want to save your test plan first? shutdown=Shutdown simple_config_element=Simple Config Element @@ -1224,6 +1227,7 @@ update_per_iter=Update Once Per Iteration upload=File Upload upper_bound=Upper Bound +upper_case=Upper case result - default false (optional) url=URL url_config_get=GET url_config_http=HTTP Index: functions/org/apache/jmeter/functions/DigestEncode.java =================================================================== --- functions/org/apache/jmeter/functions/DigestEncode.java (revision 0) +++ functions/org/apache/jmeter/functions/DigestEncode.java (working copy) @@ -0,0 +1,114 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ +package org.apache.jmeter.functions; + +import org.apache.commons.lang3.StringUtils; +import org.apache.jmeter.engine.util.CompoundVariable; +import org.apache.jmeter.functions.AbstractFunction; +import org.apache.jmeter.functions.InvalidVariableException; +import org.apache.jmeter.samplers.SampleResult; +import org.apache.jmeter.samplers.Sampler; +import org.apache.jmeter.util.JMeterUtils; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import java.io.UnsupportedEncodingException; +import java.security.MessageDigest; +import java.security.NoSuchAlgorithmException; + +import java.util.Collection; +import java.util.LinkedList; +import java.util.List; + +/** + * + * Sha512 Encode Function Optional Upper case value and save to variable name + * + * @author orim + * + */ +public class DigestEncode extends AbstractFunction { + + private static final Logger log = LoggerFactory.getLogger(DigestEncode.class); + private static final String UTF_8 = "UTF-8"; + + /* + * The algorithm names in this section can be specified when generating an + * instance of MessageDigest: SHA-1 SHA-256 SHA-384 SHA-512 + */ + private static final List desc = new LinkedList(); + private static final String KEY = "__digest"; + + // Number of parameters expected - used to reject invalid calls + private static final int MIN_PARAMETER_COUNT = 2; + private static final int MAX_PARAMETER_COUNT = 5; + + static { + desc.add(JMeterUtils.getResString("algorithm_string")); + desc.add(JMeterUtils.getResString("sha_string")); + desc.add(JMeterUtils.getResString("salt_string")); + desc.add(JMeterUtils.getResString("upper_case")); + desc.add(JMeterUtils.getResString("function_name_paropt")); + } + + private CompoundVariable[] values; + + @Override + public String execute(SampleResult previousResult, Sampler currentSampler) throws InvalidVariableException { + String encodedString = null; + String digestAlgorithm = values[0].execute(); + String stringToEncode = values[1].execute(); + String salt = null; + if (values.length > 2) { + salt = values[2].execute(); + } + try { + MessageDigest md = MessageDigest.getInstance(digestAlgorithm); + if (StringUtils.isNotEmpty(salt)) { + md.update(salt.getBytes(UTF_8)); + } + byte[] bytes = md.digest(stringToEncode.getBytes(UTF_8)); + StringBuilder sb = new StringBuilder(); + for (int i = 0; i < bytes.length; i++) { + sb.append(Integer.toString((bytes[i] & 0xff) + 0x100, 16).substring(1)); + } + encodedString = upperCase(sb.toString(), values, 3); + addVariableValue(encodedString, values, 4); + + } catch (NoSuchAlgorithmException | UnsupportedEncodingException e) { + log.warn("Error executing SHA encryption", e); + } + return encodedString; + } + + @Override + public void setParameters(Collection parameters) throws InvalidVariableException { + checkParameterCount(parameters, MIN_PARAMETER_COUNT, MAX_PARAMETER_COUNT); + values = parameters.toArray(new CompoundVariable[parameters.size()]); + } + + @Override + public String getReferenceKey() { + return KEY; + } + + @Override + public List getArgumentDesc() { + return desc; + } +} \ No newline at end of file ````

[asfimport]

@pmouawad (migrated from Bugzilla): Hello, I am currently merging the patch but I don't see tests . Can you attach the patch that only contains them ? Thanks

[asfimport]

orimarko (migrated from Bugzilla): Add patch with tests digest function

Created attachment testpatch.patch: Add patch with tests digest function

testpatch.patch

````diff Index: src/org/apache/jmeter/functions/TestDigestFunction.java =================================================================== --- src/org/apache/jmeter/functions/TestDigestFunction.java (revision 0) +++ src/org/apache/jmeter/functions/TestDigestFunction.java (working copy) @@ -0,0 +1,124 @@ +package org.apache.jmeter.functions; + +import static org.junit.Assert.assertEquals; + +import java.util.Collection; +import java.util.LinkedList; + +import org.apache.jmeter.engine.util.CompoundVariable; +import org.apache.jmeter.junit.JMeterTestCase; +import org.apache.jmeter.samplers.SampleResult; +import org.apache.jmeter.threads.JMeterContext; +import org.apache.jmeter.threads.JMeterContextService; +import org.apache.jmeter.threads.JMeterVariables; +import org.junit.Before; +import org.junit.Test; +/** + * Test Digest function + * + * Add Tests with Sha1 and Sha512 + * + * @author orim + * + */ +public class TestDigestFunction extends JMeterTestCase { + protected AbstractFunction digest; + + private SampleResult result; + + private Collection params; + + private JMeterVariables vars; + + private JMeterContext jmctx; + + @Before + public void setUp() { + digest = new DigestEncode(); + result = new SampleResult(); + jmctx = JMeterContextService.getContext(); + String data = "dummy data"; + result.setResponseData(data, null); + vars = new JMeterVariables(); + jmctx.setVariables(vars); + jmctx.setPreviousResult(result); + params = new LinkedList<>(); + } + + + @Test + public void testParameterCount512() throws Exception { + checkInvalidParameterCounts(digest, 2, 5); + } + + @Test + public void testSha512() throws Exception { + params.add(new CompoundVariable("SHA-512")); + params.add(new CompoundVariable("nofile")); + params.add(new CompoundVariable("nofile")); + digest.setParameters(params); + String returnValue = digest.execute(result, null); + assertEquals("0a76f7cd4f3cd9bec27146c32a8cad3de865a48d07ff6a2a40f18f2b3307fef4d0de695d14c90234bd453b375aa2974ba17743564bc782ad1d6cf8b3f4db72a4", returnValue); + } + + @Test + public void testSha1() throws Exception { + params.add(new CompoundVariable("SHA-1")); + params.add(new CompoundVariable("nofile")); + digest.setParameters(params); + String returnValue = digest.execute(result, null); + assertEquals("4ea2ced10057872be25371cfe638d3b096c58f2f", returnValue); + } + + @Test + public void testSha1Variable() throws Exception { + params.add(new CompoundVariable("SHA-1")); + params.add(new CompoundVariable("nofile")); + params.add(new CompoundVariable("")); + params.add(new CompoundVariable("true")); + params.add(new CompoundVariable("newVar")); + digest.setParameters(params); + String returnValue = digest.execute(result, null); + assertEquals("4EA2CED10057872BE25371CFE638D3B096C58F2F", returnValue); + } + + @Test + public void testSha512Variable() throws Exception { + params.add(new CompoundVariable("SHA-512")); + params.add(new CompoundVariable("nofile")); + params.add(new CompoundVariable("")); + params.add(new CompoundVariable("true")); + params.add(new CompoundVariable("newVar")); + digest.setParameters(params); + String returnValue = digest.execute(result, null); + assertEquals("58DA94D45A97B35B31D7F76D2EBAC184BC4BDA512B966CDBE43FDE1CAE1CFAF89617082CA89928FB5DC1C75D60B93ADB5631F518F970CA6DCC196E1AFC678B8C", returnValue); + } + + @Test + public void testSha512Error() throws Exception { + params.add(new CompoundVariable("nofile")); + boolean isFailed = false; + try { + digest.setParameters(params); + digest.execute(result, null); + } catch (InvalidVariableException e) { + // expect failure + isFailed = true; + } + assertEquals(isFailed, true); + } + + @Test + public void testSha1Error() throws Exception { + boolean isFailed = false; + try { + params.add(new CompoundVariable("SHA-1")); + digest.setParameters(params); + digest.execute(result, null); + } catch (InvalidVariableException e) { + // expect failure + isFailed = true; + } + assertEquals(isFailed, true); + } +} ````

[asfimport]

orimarko (migrated from Bugzilla): Add tests patch

[asfimport]

@pmouawad (migrated from Bugzilla): I merged the function with slight modifications:

• Salt is added at end as it seems to be the standard (https://github.com/mozilla/openbadges-backpack/wiki/How-to-hash-&-salt-in-various-languages.)
• Completed documentation and stated that spaces are taken into account
• Added more tests for spaces

Can you review those changes to see if there are issues and to see how to improve your future contributions that I hope will be exist :-) !

Thanks for this great contribution !

For future patches, can you just ensure that you create patch from root of jmeter project.

Author: pmouawad Date: Tue Nov 7 08:25:42 2017 New Revision: 1814464

URL: http://svn.apache.org/viewvc?rev=1814464&view=rev Log: https://github.com/apache/jmeter/issues/4567 - Add __digest function to provide computing of Hashes (SHA-XXX, MDX) https://github.com/apache/jmeter/issues/4567

Added: jmeter/trunk/src/functions/org/apache/jmeter/functions/DigestEncode.java (with props) Modified: jmeter/trunk/src/core/org/apache/jmeter/functions/AbstractFunction.java jmeter/trunk/src/core/org/apache/jmeter/resources/messages.properties jmeter/trunk/src/core/org/apache/jmeter/resources/messages_fr.properties jmeter/trunk/xdocs/changes.xml jmeter/trunk/xdocs/usermanual/functions.xml

[asfimport]

orimarko (migrated from Bugzilla): There are 2 errors in jenkins build

[checkstyle] /home/jenkins/jenkins-slave/workspace/JMeter-trunk/trunk/src/core/org/apache/jmeter/functions/AbstractFunction.java:152:1: error: File contains tab characters (this is the first instance).

[checkstyle] /home/jenkins/jenkins-slave/workspace/JMeter-trunk/trunk/src/functions/org/apache/jmeter/functions/DigestEncode.java:0: error: File does not end with a newline.

[asfimport]

orimarko (migrated from Bugzilla): Resolved in apache-jmeter-r1814470, build number 6​42​9

Thank you

[asfimport]

@pmouawad (migrated from Bugzilla): Author: pmouawad Date: Wed Nov 8 19:44:02 2017 New Revision: 1814627

URL: http://svn.apache.org/viewvc?rev=1814627&view=rev Log: https://github.com/apache/jmeter/issues/4567 - Add __digest function to provide computing of Hashes (SHA-XXX, MDX) Add tests https://github.com/apache/jmeter/issues/4567

[asfimport]

@pmouawad (migrated from Bugzilla): Author: pmouawad Date: Mon Nov 13 20:58:16 2017 New Revision: 1815133

URL: http://svn.apache.org/viewvc?rev=1815133&view=rev Log: https://github.com/apache/jmeter/issues/4567 - Add __digest function to provide computing of Hashes (SHA-XXX, MDX) Suffix class with Function https://github.com/apache/jmeter/issues/4567

Added: jmeter/trunk/src/functions/org/apache/jmeter/functions/DigestEncodeFunction.java

• copied, changed from r1814986, jmeter/trunk/src/functions/org/apache/jmeter/functions/DigestEncode.java Removed: jmeter/trunk/src/functions/org/apache/jmeter/functions/DigestEncode.java Modified: jmeter/trunk/test/src/org/apache/jmeter/functions/TestDigestFunction.java