Alibaba Example - Githubissues

hassanSattariNia commented 1 month ago

Thank you for providing examples of Alibaba traces, but using Alibaba traces, I read a document on how to create deployment and applications, but I can't create applications with these files . For instance, when creating /par/app5 with service_mesh.json In this file, I see a structure like this: { "s0": { "external_services": [ { "seq_len": 10000, "services": [] } ] }, "s1": { "external_services": [ { "seq_len": 10000, "services": [] } ] }, ... }

I attempted to create workmode.json using service_mesh.json, but the resulting workmode.json does not define any external services for s0. This is expected, as service_mesh.json also does not define any external services.

I noticed that the traces for /app5 indicate the presence of external services. How can I create a workmode.json for app5 that accurately reflects the conditions found in the Alibaba traces for app5? While I could manually review all the traces and edit service_mesh.json, I don't think this is the best approach. Could you assist me in creating a workmode.json that precisely matches the app5 conditions in the Alibaba traces?

andreadetti commented 1 month ago

For trace driven benchmarks as Alibaba ones, the workmodel file cannot contain any external services, since the external services to be called are defined by the trace sent via HTTP post to s0. See the section of the manual on "trace-driven benchmarks".

hassanSattariNia commented 3 weeks ago

So, if I want to create an application with different branches and varying call depths, I cannot use the Alibaba examples?

I need to develop an application that supports multiple branches and allows me to see different requests in each branch.

Which example can help me achieve this, and how can I create an application with these requirements?

Please provide detailed instructions and descriptions if possible.

I am looking to create an application with multiple requests that simulate a real-world application. Therefore, I reviewed the Alibaba traces.

andreadetti commented 3 weeks ago

You definitively can. Please consider the following instructions.

Alibaba traces

In the examples directory, there is the Alibaba folder with a collection of applications obtained from processing the real Alibaba traces, in the same directory we can find the Matlab scripts used for the processing of the traces. To use these applications, first, we need to unzip the trace-mbench.zip file inside the examples/Alibaba directory. As a result of this operation, we obtain the trace-mbench directory within two folders: par and seq.

muBench/
├─ examples/
│  ├─ Alibaba/
│  │  ├─ Matlab/
│  │  ├─ traces-mbench/
│  │  │  ├─ par/
│  │  │  │  ├─ app2/
│  │  │  │  ├─ app3/
│  │  │  │  ├─ ...
│  │  │  ├─ seq/
│  │  │  │  ├─ app2/
│  │  │  │  ├─ app3/
│  │  │  │  ├─ ...

Each folder contains 29 applications, each one with a set of traces. The difference between the traces in the two folders is that traces in the par directory execute downstream requests in parallel, whereas the traces in the seq directory execute requests in sequence. In each app folder, you will find a service_mesh.json file that represents the app's service graph and its traces. The service_mesh.json does not contain any external services, because the sequence of external services to be called is specified by the trace sent via HTTP POST.

To benchmark an app generated from Alibaba trace, the following steps must be performed:

Create a Config/WorkModelParameters.json file that contains the service_mesh.json file of the application you intend to test. As for the remaining information in the WorkModelParameters.json file, we were unable to derive it from Alibaba's traces, so the user must make his or her own choices. For example, a possible WorkModelParameters.json file for sequential application No. 18 is as follows. In this case, all microservices stress the CPU through the loader function, the average size of responses is 11 kBytes, and Kubernetes' requests and CPU limits are set to 250m.

{
  "WorkModelParameters":{
     "f2": {
        "type":"function",
        "value": {
           "name": "loader",
           "recipient": "service",
           "probability":1,
           "parameters": {
              "cpu_stress": {"run":true,"range_complexity": [100, 100], "thread_pool_size": 1, "trials": 1},
              "memory_stress":{"run":false, "memory_size": 10000, "memory_io": 1000},
              "disk_stress":{"run":false,"tmp_file_name":  "mubtestfile.txt", "disk_write_block_count": 1000, "disk_write_block_size": 1024},
              "mean_response_size": 11
           },
           "workers":8,
           "threads":128,
           "cpu-requests": "250m",
           "cpu-limits": "250m"
        }
     },
     "request_method":{
        "type": "metadata",
        "value":"rest"
     },
     "databases_prefix": {
        "type":"metadata",
        "value": "sdb"
     },
     "override": {},
     "ServicemeshFilePath": {
        "type": "metadata", 
        "value":"Examples/Alibaba/traces-mbench/seq/app18/service_mesh.json"
     },
     "OutputPath": {
        "type":"metadata",
        "value":"SimulationWorkspace"
     }
  }
}

Generate the workmodel.json file with the WorkModelGenerator by running the following command:
```
python3 WorkModelGenerator/RunWorkModelGen.py -c Configs/WorkModelParameters.json
```
Deploy the application with K8sDeployer by providing as input the workmodel.json file created in the previous step ad by using the following command. If necessary update properly the Configs/K8sParameters.json
```
python3 Deployers/K8sDeployer/RunK8sDeployer.py -c Configs/K8sParameters.json
```
To send a trace-driven request, we can send an HTTP POST to the NGINX access gateway with the JSON file of the trace as the body. For example, we can use the bash command curl to send the first trace of sequential application n. 18.
```
curl -X POST -H "Content-Type: application/json" http://<access-gateway-ip>:31113/s0 -d @examples/Alibaba/traces-mbench/seq/app18/trace00001.json
```

hassanSattariNia commented 3 weeks ago

I followed the steps to create a working model based on the service_mesh.json file. Then, I sent requests based on individual trace files located in the provided folder. I observed that the requests worked well. Here is a picture of sending requests for app5/trace0001.json, trace0002.json, trace0003.json, and trace0004.json. request_trace_base

However, when I try to create a workload on the application using Runner.py provided in this application, I notice that all the requests are the same and do not exit from s0, as shown in the picture below.

request_when_runner

To simulate a real application, how can I create a realistic workload? Should I write a custom script that sends requests to different traces like this:

send random of these requests: curl -X POST -H "Content-Type: application/json" http://:31113/s0 -d @examples/Alibaba/traces-mbench/seq/app18/trace00001.json

Or should I use JMeter or other benchmarking tools?

Please provide instructions on how to create a more realistic workload and effectively simulate a real application workload.

andreadetti commented 3 weeks ago

Hi, Runner is not designed for trace-based benchmarks. It simply sends HTTP GET to S0, rather than HTTP POST with the trace. For trace benchmarking, you need to use another tool. For example, we used Apache Jmeter. I don't remember how :-(. Anyway, the idea was to randomly choose a trace from the possible ones for each request and use HTTP post.

mSvcBench / muBench

Alibaba Example #20

Alibaba traces