How to start the load generator when deploying in a Kubernetes cluster?

[riksanyal]

These are the steps I have followed:

• started a minikube cluster (checked it's running)
• applied the deploy.yaml file by running

  kubectl apply -f https://raw.githubusercontent.com/aws-containers/retail-store-sample-app/main/dist/kubernetes/deploy.yaml
  kubectl wait --for=condition=available deployments --all

• started the load generator pod by following the steps here

  apiVersion: v1
  kind: Pod
  metadata:
  name: load-generator
  spec:
  containers:
  - name: artillery
  image: artilleryio/artillery:2.0.0-31
  args:
  - "run"
  - "-t"
  - "http://ui.ui.svc"
  - "/scripts/scenario.yml"
  volumeMounts:
  - name: scripts
    mountPath: /scripts
  initContainers:
  - name: setup
  image: public.ecr.aws/aws-containers/retail-store-sample-utils:load-gen.0.3.0
  command:
  - bash
  args:
  - -c
  - "cp /artillery/* /scripts"
  volumeMounts:
  - name: scripts
    mountPath: "/scripts"
  volumes:
  - name: scripts
  emptyDir: {}
  EOF

• ran the bash script bash src/load-generator/scripts/run.sh

It seems that the load is not being generated as I have run the micro-service (also the load-generator pod) for more than 5 hours now and the ui deployment log file has only ~70 lines with timestamps of the lines corresponding to the deployment start time.

Could someone tell me what files I have to edit to generate the load using the load generator pod that I created? @niallthomson

[niallthomson]

Hey @riksanyal do the logs of the load generator pod show anything that looks like it could be a problem?

[riksanyal]

Hi @niallthomson The log of the load generator pod has many blocks like the one below with different period values and timestamps:

Warning: multiple batches of metrics for period 1720766920000 2024-07-12T06:48:40.000Z
--------------------------------------
Metrics for period to: 06:49:00(+0000) (width: 9.002s)
--------------------------------------

errors.ENOTFOUND: .............................................................. 10
http.request_rate: ............................................................. 1/sec
http.requests: ................................................................. 10
vusers.created: ................................................................ 10
vusers.created_by_name.Test: ................................................... 10
vusers.failed: ................................................................. 10

[niallthomson]

The Kubernetes manifest thats generated for the sample application puts all the deployments in the same namespace, whereas that load generator manifest assumes that UI is in its own namespace called ui. Determine what namespace you deployed the manifest to (its probably default) and in the load generator manifest change http://ui.ui.svc to http://ui.<namespace>.svc where namespace is where you deployed the application.

[riksanyal]

Hi @niallthomson , I have fixed that but the microservices still doesn't seem to pick any load. I have run it for ~10m and don't see anything in the ui logs after the initial setup lines (see the screenshot below):

[niallthomson]

I'm not sure it logs anything per request. Can you add log output from the load generator pod again?

[riksanyal]

Hi @niallthomson, sorry for the delay. Seems like the load is being generated but nothing is being logged in the other microservices. How can I log things like the timestamp, etc. per request?

Here is the log output of the load generator:

--------------------------------------
Metrics for period to: 01:44:20(+0000) (width: 5.462s)
--------------------------------------

http.codes.200: ................................................................ 48
http.codes.303: ................................................................ 6
http.request_rate: ............................................................. 10/sec
http.requests: ................................................................. 54
http.response_time:
  min: ......................................................................... 20
  max: ......................................................................... 544
  median: ...................................................................... 39.3
  p95: ......................................................................... 247.2
  p99: ......................................................................... 518.1
http.responses: ................................................................ 54
vusers.completed: .............................................................. 6
vusers.created: ................................................................ 6
vusers.created_by_name.Test: ................................................... 6
vusers.failed: ................................................................. 0
vusers.session_length:
  min: ......................................................................... 301.5
  max: ......................................................................... 2028.7
  median: ...................................................................... 368.8
  p95: ......................................................................... 1022.7
  p99: ......................................................................... 1022.7

[niallthomson]

Hi @riksanyal adding logging per request is something we'd need to add. It would not be difficult but theres a couple of reasons why I put it off this long, for example whether logging volume is something desirable by default or should be turned on explicitly.

Are you looking for log messages in all components? Or just in the UI? It might be possible to get some by enabling access logs which is something I can check if it will help you're scenario.

[riksanyal]

Hi @niallthomson, yes, it would be great if we could get log messages from all components for each request. Any help in this regard would be greatly appreciated

[riksanyal]

Hi @niallthomson , are there any updates on this?

[niallthomson]

its something we can add to the list of issues but unsure when it would get done.

Right now the only thing you could do is add environment variables specific to each language/framework to enable access logging. For example the component based on Spring Boot could use something like SERVER_TOMCAT_ACCESS_LOG_ENABLED=true. I think the checkout service might actually be enabled by default. The catalog service I'm unsure about.