search

eclipse-californium / californium.tools

Californium project
Other
59 stars 57 forks source link

CoapSampler cannot be triggered in a docker container #89

Closed dlee-blackline closed 1 year ago

dlee-blackline commented 1 year ago

I created a Java request with the class org.eclipse.californium.tools.jmeter.plugin.CoapSampler in JMeter. 

<?xml version="1.0" encoding="UTF-8"?>
<jmeterTestPlan version="1.2" properties="5.0" jmeter="5.5">
  <hashTree>
    <TestPlan guiclass="TestPlanGui" testclass="TestPlan" testname="Test Plan" enabled="true">
      <stringProp name="TestPlan.comments"></stringProp>
      <boolProp name="TestPlan.functional_mode">false</boolProp>
      <boolProp name="TestPlan.tearDown_on_shutdown">true</boolProp>
      <boolProp name="TestPlan.serialize_threadgroups">false</boolProp>
      <elementProp name="TestPlan.user_defined_variables" elementType="Arguments" guiclass="ArgumentsPanel" testclass="Arguments" testname="User Defined Variables" enabled="true">
        <collectionProp name="Arguments.arguments"/>
      </elementProp>
      <stringProp name="TestPlan.user_define_classpath"></stringProp>
    </TestPlan>
    <hashTree>
      <ThreadGroup guiclass="ThreadGroupGui" testclass="ThreadGroup" testname="Thread Group" enabled="true">
        <stringProp name="ThreadGroup.on_sample_error">continue</stringProp>
        <elementProp name="ThreadGroup.main_controller" elementType="LoopController" guiclass="LoopControlPanel" testclass="LoopController" testname="Loop Controller" enabled="true">
          <boolProp name="LoopController.continue_forever">false</boolProp>
          <stringProp name="LoopController.loops">1</stringProp>
        </elementProp>
        <stringProp name="ThreadGroup.num_threads">1</stringProp>
        <stringProp name="ThreadGroup.ramp_time">1</stringProp>
        <boolProp name="ThreadGroup.scheduler">false</boolProp>
        <stringProp name="ThreadGroup.duration"></stringProp>
        <stringProp name="ThreadGroup.delay"></stringProp>
        <boolProp name="ThreadGroup.same_user_on_next_iteration">true</boolProp>
      </ThreadGroup>
      <hashTree>
        <JavaSampler guiclass="JavaTestSamplerGui" testclass="JavaSampler" testname="Java Request" enabled="true">
          <elementProp name="arguments" elementType="Arguments" guiclass="ArgumentsPanel" testclass="Arguments" enabled="true">
            <collectionProp name="Arguments.arguments">
              <elementProp name="SERVER_ENDPOINT" elementType="Argument">
                <stringProp name="Argument.name">SERVER_ENDPOINT</stringProp>
                <stringProp name="Argument.value">coaps://californium.eclipseprojects.io/telemetry</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="MESSAGE_PAYLOAD" elementType="Argument">
                <stringProp name="Argument.name">MESSAGE_PAYLOAD</stringProp>
                <stringProp name="Argument.value">Enter payload here</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="IDENTITY" elementType="Argument">
                <stringProp name="Argument.name">IDENTITY</stringProp>
                <stringProp name="Argument.value">cali.jm-${__threadNum}</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="SECRET_KEY64" elementType="Argument">
                <stringProp name="Argument.name">SECRET_KEY64</stringProp>
                <stringProp name="Argument.value">LmZvcm5pdW0=</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="CONNECTION_TIME_OUT" elementType="Argument">
                <stringProp name="Argument.name">CONNECTION_TIME_OUT</stringProp>
                <stringProp name="Argument.value">10</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="CONNECTION_IDLE_TIME" elementType="Argument">
                <stringProp name="Argument.name">CONNECTION_IDLE_TIME</stringProp>
                <stringProp name="Argument.value">6</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="AUTO_RESUMPTION_TIME_OUT" elementType="Argument">
                <stringProp name="Argument.name">AUTO_RESUMPTION_TIME_OUT</stringProp>
                <stringProp name="Argument.value">30</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="RECONNECT" elementType="Argument">
                <stringProp name="Argument.name">RECONNECT</stringProp>
                <stringProp name="Argument.value">true</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="METHOD" elementType="Argument">
                <stringProp name="Argument.name">METHOD</stringProp>
                <stringProp name="Argument.value">POST</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="CONTENT_TYPE" elementType="Argument">
                <stringProp name="Argument.name">CONTENT_TYPE</stringProp>
                <stringProp name="Argument.value">text/plain</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="ACCEPT_TYPE" elementType="Argument">
                <stringProp name="Argument.name">ACCEPT_TYPE</stringProp>
                <stringProp name="Argument.value">text/plain</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="MESSAGE_TYPE" elementType="Argument">
                <stringProp name="Argument.name">MESSAGE_TYPE</stringProp>
                <stringProp name="Argument.value">CON</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="EMPTY_AS_ERROR" elementType="Argument">
                <stringProp name="Argument.name">EMPTY_AS_ERROR</stringProp>
                <stringProp name="Argument.value">true</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="RETRY_503" elementType="Argument">
                <stringProp name="Argument.name">RETRY_503</stringProp>
                <stringProp name="Argument.value">3</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="RETRY_EMPTY" elementType="Argument">
                <stringProp name="Argument.name">RETRY_EMPTY</stringProp>
                <stringProp name="Argument.value">3</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="RETRY" elementType="Argument">
                <stringProp name="Argument.name">RETRY</stringProp>
                <stringProp name="Argument.value">3</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="QUIET_MILLIS" elementType="Argument">
                <stringProp name="Argument.name">QUIET_MILLIS</stringProp>
                <stringProp name="Argument.value">0..0</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
            </collectionProp>
          </elementProp>
          <stringProp name="classname">org.eclipse.californium.tools.jmeter.plugin.CoapSampler</stringProp>
        </JavaSampler>
        <hashTree/>
        <ResultCollector guiclass="ViewResultsFullVisualizer" testclass="ResultCollector" testname="View Results Tree" enabled="true">
          <boolProp name="ResultCollector.error_logging">false</boolProp>
          <objProp>
            <name>saveConfig</name>
            <value class="SampleSaveConfiguration">
              <time>true</time>
              <latency>true</latency>
              <timestamp>true</timestamp>
              <success>true</success>
              <label>true</label>
              <code>true</code>
              <message>true</message>
              <threadName>true</threadName>
              <dataType>true</dataType>
              <encoding>false</encoding>
              <assertions>true</assertions>
              <subresults>true</subresults>
              <responseData>false</responseData>
              <samplerData>false</samplerData>
              <xml>false</xml>
              <fieldNames>true</fieldNames>
              <responseHeaders>false</responseHeaders>
              <requestHeaders>false</requestHeaders>
              <responseDataOnError>false</responseDataOnError>
              <saveAssertionResultsFailureMessage>true</saveAssertionResultsFailureMessage>
              <assertionsResultsToSave>0</assertionsResultsToSave>
              <bytes>true</bytes>
              <sentBytes>true</sentBytes>
              <url>true</url>
              <threadCounts>true</threadCounts>
              <idleTime>true</idleTime>
              <connectTime>true</connectTime>
            </value>
          </objProp>
          <stringProp name="filename"></stringProp>
        </ResultCollector>
        <hashTree/>
        <ResultCollector guiclass="SummaryReport" testclass="ResultCollector" testname="Summary Report" enabled="true">
          <boolProp name="ResultCollector.error_logging">false</boolProp>
          <objProp>
            <name>saveConfig</name>
            <value class="SampleSaveConfiguration">
              <time>true</time>
              <latency>true</latency>
              <timestamp>true</timestamp>
              <success>true</success>
              <label>true</label>
              <code>true</code>
              <message>true</message>
              <threadName>true</threadName>
              <dataType>true</dataType>
              <encoding>true</encoding>
              <assertions>true</assertions>
              <subresults>true</subresults>
              <responseData>true</responseData>
              <samplerData>true</samplerData>
              <xml>true</xml>
              <fieldNames>true</fieldNames>
              <responseHeaders>false</responseHeaders>
              <requestHeaders>true</requestHeaders>
              <responseDataOnError>false</responseDataOnError>
              <saveAssertionResultsFailureMessage>true</saveAssertionResultsFailureMessage>
              <assertionsResultsToSave>0</assertionsResultsToSave>
              <bytes>true</bytes>
              <sentBytes>true</sentBytes>
              <url>true</url>
              <fileName>true</fileName>
              <hostname>true</hostname>
              <threadCounts>true</threadCounts>
              <sampleCount>true</sampleCount>
              <idleTime>true</idleTime>
              <connectTime>true</connectTime>
            </value>
          </objProp>
          <stringProp name="filename"></stringProp>
        </ResultCollector>
        <hashTree/>
        <ResultCollector guiclass="StatGraphVisualizer" testclass="ResultCollector" testname="Aggregate Graph" enabled="true">
          <boolProp name="ResultCollector.error_logging">false</boolProp>
          <objProp>
            <name>saveConfig</name>
            <value class="SampleSaveConfiguration">
              <time>true</time>
              <latency>true</latency>
              <timestamp>true</timestamp>
              <success>true</success>
              <label>true</label>
              <code>true</code>
              <message>true</message>
              <threadName>true</threadName>
              <dataType>true</dataType>
              <encoding>false</encoding>
              <assertions>true</assertions>
              <subresults>true</subresults>
              <responseData>false</responseData>
              <samplerData>false</samplerData>
              <xml>false</xml>
              <fieldNames>true</fieldNames>
              <responseHeaders>false</responseHeaders>
              <requestHeaders>false</requestHeaders>
              <responseDataOnError>false</responseDataOnError>
              <saveAssertionResultsFailureMessage>true</saveAssertionResultsFailureMessage>
              <assertionsResultsToSave>0</assertionsResultsToSave>
              <bytes>true</bytes>
              <sentBytes>true</sentBytes>
              <url>true</url>
              <threadCounts>true</threadCounts>
              <idleTime>true</idleTime>
              <connectTime>true</connectTime>
            </value>
          </objProp>
          <stringProp name="filename"></stringProp>
        </ResultCollector>
        <hashTree/>
      </hashTree>
    </hashTree>
  </hashTree>
</jmeterTestPlan>

I can execute it on my local machine (jmeter -n -t my_file.jmx):

summary =      1 in 00:00:01 =    0.8/s Avg:   775 Min:   775 Max:   775 Err:     0 (0.00%)

However, the file cannot be executed properly if I use a docker container. I can run the command docker compose up to start the following service:

version: '3.9'
services:
  load-testing:
    image: openjdk:11.0.16-jre-slim-bullseye
    command: /mnt/apache-jmeter/bin/jmeter -n -t /mnt/test-plans/my_file.jmx
    volumes:
      - ./test-plans:/mnt/test-plans
      - ./apache-jmeter-5.5:/mnt/apache-jmeter

The container did not print any errors, but the request cannot be executed. The response is:

summary =      0 in 00:00:00 = ******/s Avg:     0 Min: 9223372036854775807 Max: -9223372036854775808 Err:     0 (0.00%)

I tested the YML file with HTTP requests and It worked well. Only with the CoapSampler has this issue.

I am using the same JMeter version (https://archive.apache.org/dist/jmeter/binaries/apache-jmeter-5.5.tgz) and JMX file in the both cases. Also, my local machine is using Java 11 as well.

How can I fix the issue happening in containers?

boaks commented 1 year ago

I guess you need a docker expert. You may try to get some more logs from inside the container. Or you check, how to run a UDP client inside a container.

dlee-blackline commented 1 year ago

The container logs didn't provide any information about the problem

software-spikes-load-testing-1  | Mar 14, 2023 7:17:52 PM java.util.prefs.FileSystemPreferences$1 run
software-spikes-load-testing-1  | INFO: Created user preferences directory.
software-spikes-load-testing-1  | Creating summariser <summary>
software-spikes-load-testing-1  | Created the tree successfully using /mnt/test-plans/my_file.jmx
software-spikes-load-testing-1  | Starting standalone test @ 2023 Mar 14 19:17:53 UTC (1678821473035)
software-spikes-load-testing-1  | Waiting for possible Shutdown/StopTestNow/HeapDump/ThreadDump message on port 4445
software-spikes-load-testing-1  | Warning: Nashorn engine is planned to be removed from a future JDK release
software-spikes-load-testing-1  | summary =      0 in 00:00:00 = ******/s Avg:     0 Min: 9223372036854775807 Max: -9223372036854775808 Err:     0 (0.00%)
software-spikes-load-testing-1  | Tidying up ...    @ 2023 Mar 14 19:18:14 UTC (1678821494235)
software-spikes-load-testing-1  | ... end of run

I have created another custom Sampler to send CoAP requests and It's working, so I think it's no problem to run UDP client inside a container

boaks commented 1 year ago

The container logs didn't provide any information about the problem

I have created another custom Sampler to send CoAP requests

I don't understand what you're doing and what's the issue.

Do you run the same JMeter CoAP plugin (cf-jmeter-plugin)? And you have two test plans: Without docker, both works. With docker only one?

Anyway, I will not go to test your jmeterTestPlan with jmeter in docker. I'm not able to help you with the information you provide.

coaps://californium.eclipseprojects.io/telemetry 

<elementProp name="QUIET_MILLIS" elementType="Argument">
                <stringProp name="Argument.name">QUIET_MILLIS</stringProp>
                <stringProp name="Argument.value">0..0</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
</elementProp>

this smells like a "load-test". Sorry, the sandbox is considered for functional tests, not for benchmarks and load-tests.

Please setup your own server!

dlee-blackline commented 1 year ago

Sorry to make you feel confusing.

The container logs I showed to you were about cf-jmeter-plugin (https://github.com/eclipse-californium/californium.tools/blob/main/cf-jmeter-plugin/src/main/java/org/eclipse/californium/tools/jmeter/plugin/CoapSampler.java). The sampler I created doesn't have any problem. I just want to show that it's no problem to run UDP client inside a container.

Although I use load testing as the name, but it only sends one request.

You can reproduce the result by following the steps:

  1. Setup docker compose (https://docs.docker.com/compose/install/)
  2. Download JMeter at https://archive.apache.org/dist/jmeter/binaries/apache-jmeter-5.5.tgz and unzip it
  3. Build cf tools (https://github.com/eclipse-californium/californium.tools), then copy the jar file from cf-jmeter-plugin/target to the directory apache-jmeter-5.5/lib/ext
  4. Create the jmx file named my_file.jmx under the directory test-plans and paste the content I sent to you at the beginning into the file. If you suspect the file I sent to you is invalid, you can also create a new JMX file with a cf-plugin request
  5. Create a file docker-compose.yml with the correct mount paths. (Note: You can run any JMX files in /mnt/test-plans by changing the command /mnt/apache-jmeter/bin/jmeter -n -t /mnt/test-plans/{your_jmx_file}) 
    version: '3.9'
services:
test-cf-plugin:
  image: openjdk:11.0.16-jre-slim-bullseye
  command: /mnt/apache-jmeter/bin/jmeter -n -t /mnt/test-plans/my_file.jmx
  volumes:
    - ./test-plans:/mnt/test-plans
    - ./apache-jmeter-5.5:/mnt/apache-jmeter
  6. execute the command docker compose up to run the docker

You should get the result as bellow:

Mar 14, 2023 7:17:52 PM java.util.prefs.FileSystemPreferences$1 run
INFO: Created user preferences directory.
Creating summariser <summary>
Created the tree successfully using /mnt/test-plans/my_file.jmx
Starting standalone test @ 2023 Mar 14 19:17:53 UTC (1678821473035)
Waiting for possible Shutdown/StopTestNow/HeapDump/ThreadDump message on port 4445
Warning: Nashorn engine is planned to be removed from a future JDK release
summary =      0 in 00:00:00 = ******/s Avg:     0 Min: 9223372036854775807 Max: -9223372036854775808 Err:     0 (0.00%)
Tidying up ...    @ 2023 Mar 14 19:18:14 UTC (1678821494235)
... end of run
dlee-blackline commented 1 year ago

The container logs didn't provide any information about the problem

I have created another custom Sampler to send CoAP requests

I don't understand what you're doing and what's the issue.

Do you run the same JMeter CoAP plugin (cf-jmeter-plugin)? And you have two test plans: Without docker, both works. With docker only one?

Anyway, I will not go to test your jmeterTestPlan with jmeter in docker. I'm not able to help you with the information you provide.

coaps://californium.eclipseprojects.io/telemetry 

<elementProp name="QUIET_MILLIS" elementType="Argument">
                <stringProp name="Argument.name">QUIET_MILLIS</stringProp>
                <stringProp name="Argument.value">0..0</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
</elementProp>

this smells like a "load-test". Sorry, the sandbox is considered for functional tests, not for benchmarks and load-tests.

Please setup your own server!

the elementProp you mentioned was using the default value. I did not change it

boaks commented 1 year ago

Sorry, this still unclear.

cf-jmeter-plugin runs one sampler successful inside the container? But cf-jmeter-plugin fails to runs the other sampler successful inside the container?

What is clear: I don't spend my time to test inside a container.

This project is about Californium. I added the jmeter plugin some time ago, but I'm no jmeter expert. Not do I support to run where ever. Especially, if the one sampler works, but the other not, it's up to you, to narrow down the issue. With the provided information, I don't have an idea, what causes a failure.

dlee-blackline commented 1 year ago

no, the case is the file I sent to you can be executed on local machines, but failed in a container.

Anyway, thanks for your response.