[BUG] throughput degradation query test

[timofeevmd]

OS and Environment

linux, k8s

GIT commit hash

a62640d8a

Minimum working example / Steps to reproduce

java-sdk version: e23b548 iroha2-perf: 94b7855

test scenario:

queryPostFindAssetById 
queryPostFindAccountById 
queryPostFindAssetQuantityById

The maximum throughput was 1633 RPS without the PARCA plugin and 3041 RPS with the PARCA plugin connected. After reaching peak values, performance decreases.

1. opensearch logs&_a=(columns:!(_source),filters:!(('$state':(store:appState),meta:(alias:!n,disabled:!f,index:'7c942030-5184-11ef-8ed8-b7c4a5d2f81f',key:kubernetes.pod.name,negate:!f,params:(query:iroha2),type:phrase),query:(match_phrase:(kubernetes.pod.name:iroha2))),('$state':(store:appState),meta:(alias:!n,disabled:!f,index:'7c942030-5184-11ef-8ed8-b7c4a5d2f81f',key:kubernetes.namespace,negate:!f,params:(query:iroha2-test),type:phrase),query:(match_phrase:(kubernetes.namespace:iroha2-test))),('$state':(store:appState),meta:(alias:!n,disabled:!f,index:'7c942030-5184-11ef-8ed8-b7c4a5d2f81f',key:logdata,negate:!f,params:!(uri%3D%2Fquery),type:phrases,value:uri%3D%2Fquery),query:(bool:(minimum_should_match:1,should:!((match_phrase:(logdata:uri%3D%2Fquery))))))),index:'7c942030-5184-11ef-8ed8-b7c4a5d2f81f',interval:s,query:(language:kuery,query:''),sort:!(!('@timestamp',asc))))

2. parca profile:

   • diff 2_minuts_before_decreases/2_min_after_throughput_peak profile_moreThen_3000_tps.pb.gz
   • full profile profile_full_queries_more_then_3000_tps.pb.gz

3. full performance report (test 9, test 10)

config for 9 test: OUTSIDE test with PARCA PROFILER, 5 peer, iroha2_loglevel: DEBUG

config for 10 test: OUTSIDE test, 5 peer, iroha2_loglevel: DEBUG

./mvnw gatling:test -DtargetURL=https://iroha2.test.tachi.soramitsu.co.jp/ -DremoteLogin=i -DremotePassword= -DconcurrentUsers=50 -Dtimes=100 -DstageDuration=18 -DstartingFrom=0 -DrampDuration=18 -DmaxDuration=1800

Actual result

performance decreases

Expected result

he load continues to increase

Logs

Log contents

```json look at "step for reproduce" ```

Who can help to reproduce?

@timofeevmd

Notes

No response