search

parksunwoo / memo-archive

memo of dev issues
0 stars 0 forks source link

locust 설정 #11

Open parksunwoo opened 3 years ago

parksunwoo commented 3 years ago

공식문서 : https://docs.locust.io/en/stable/installation.html

테스트 시나리오를 locustfile.py 에 작성함 간단히 django 에서 jwt token 을 사용하는 예를 아래 코드로 작성

import json
from locust import HttpUser, TaskSet, task, between

class QuickstartUser(HttpUser):
    wait_time = between(1, 2.5)

    def __init__(self, parent):
        super(QuickstartUser, self).__init__(parent)
        self.token = ""
        self.headers = {}

    @task(1)
    def today1(self):
        self.client.get("/api-nodes/v1/today/", headers=self.headers)

    def on_start(self):
        self.token = self.login()
        self.headers = {'Authorization': 'Bearer ' + self.token}

    def login(self):
        response = self.client.post("/token/", data={'username': 'test', 'password': 'password'})
        return json.loads(response._content)['token'][0]

확인해보고자 하는 host를 아래와 같이 입력후 실행

locust --host=https://example.com --locustfile locustfile.py

Python 부하테스트기 ECS+Locust로 부하 테스트 진행하기 성능, 부하, 스트레스 테스트에 대하여 locust 스크립트 참고예제

parksunwoo commented 3 years ago

statistics 에서 각 항목의 숫자가 의미하는 건 무엇인가?

90%ile (ms)

Average size (bytes)

Current RPS(Request Per Second)

--

parksunwoo commented 3 years ago
Type Name Request Count Failure Count Median Response Time Average Response Time Min Response Time Max Response Time Average Content Size Requests/s Failures/s 50% 66% 75% 80% 90% 95% 98% 99% 99.9% 99.99% 100%
GET /api-nodes/v2/today/ 4453 0 1900.0 2909.3759682840800 61.47821199999730 21806.545243 354.0 5.186655048994790 0.0 1900 2900 3800 4600 7300 9500 12000 14000 20000 22000 22000
GET /api-nodes/waitnp/ 4574 0 1800.0 2759.618527971570 41.53982000002540 21241.605963000000 84.0 5.327590432091210 0.0 1800 2800 3600 4300 6500 9000 12000 14000 19000 21000 21000
GET /api-stages/compnp/ 4613 0 2300.0 3274.0944297095200 258.07007699995700 22276.42045000000 2316.0 5.3730158861470800 0.0 2300 3300 4200 4800 7100 9800 13000 14000 21000 22000 22000
GET /api-stages/v2/ 4387 0 1900.0 2864.119820623880 65.29831600005310 20899.839187000000 1302.0 5.109781203669470 0.0 1900 2900 3800 4500 6800 9200 12000 14000 20000 21000 21000
POST /token/ 100 0 4300.0 4976.974921080000 345.0469000000000 13483.054183 413.0 0.11647552322018400 0.0 4600 5900 7100 8100 10000 12000 13000 13000 13000 13000 13000
  Aggregated 18127 0 2000.0 2964.855468811500 41.53982000002540 22276.42045000000 1014.9202846582400 21.113518094122700 0.0 2000 3000 3900 4600 7000 9400 12000 14000 20000 22000 22000

editor1 300 명 / 10명 15 RPS / 0% / (Target.Timeout) / Healthy Host Count 0

Type Name Request Count Failure Count Median Response Time Average Response Time Min Response Time Max Response Time Average Content Size Requests/s Failures/s 50% 66% 75% 80% 90% 95% 98% 99% 99.9% 99.99% 100%
GET /api-nodes/v2/today/ 3154 0 7800.0 8777.681317629980 83.33169300000170 30746.432530000000 354.0 6.571118246542400 0.0 7800 11000 13000 14000 17000 20000 23000 24000 28000 31000 31000
GET /api-nodes/waitnp/ 3153 0 7600.0 8792.393547054550 56.08469700000290 30725.529127 84.0 6.569034822875140 0.0 7600 11000 13000 14000 18000 21000 23000 25000 30000 31000 31000
GET /api-stages/compnp/ 3175 0 8600.0 9540.712495514670 385.5007960000020 31586.433076000000 2316.0 6.614870143554890 0.0 8600 12000 13000 15000 18000 21000 24000 25000 30000 32000 32000
GET /api-stages/v2/ 3055 0 7900.0 8866.660217546310 84.81679699999850 29837.893403000000 1302.0 6.3648593034835300 0.0 7900 11000 13000 14000 18000 20000 23000 25000 28000 30000 30000
POST /token/ 300 0 4900.0 6852.395257823340 211.76022199999900 22968.891739000000 413.0 0.625027100178415 0.0 4900 8100 10000 12000 18000 20000 22000 22000 23000 23000 23000
  Aggregated 12837 0 7900.0 8946.198601299200 56.08469700000290 31586.433076000000 999.9359663472770 26.744909616634400 0.0 7900 11000 13000 14000 18000 20000 23000 25000 30000 31000 32000
parksunwoo commented 3 years ago

변경되는 today, stages api 호출 테스트 STG editor1 100 명 / 10명 55 RPS / 10%

Type Name Request Count Failure Count Median Response Time Average Response Time Min Response Time Max Response Time Average Content Size Requests/s Failures/s 50% 66% 75% 80% 90% 95% 98% 99% 99.9% 99.99% 100%
GET /api-nodes/v2/today/ 10235 942 56 134.03324505334600 5.783588999975110 7133.133826999990 332.64738641915 13.062555315170900 1.2022400690660500 56 70 97 130 270 440 710 990 6400 7100 7100
GET /api-nodes/waitnp/ 10425 1009 45 120.48122729448400 5.8372720000079400 7109.065755000020 87.6778896882494 13.305045350332900 1.287749713044210 45 57 83 110 260 420 660 940 6700 7100 7100
GET /api-stages/compnp/ 10040 1019 130.0 200.2945130643420 5.638586000031860 7268.720405000010 2093.322111553790 12.8136839632942 1.3005123464737800 130 150 180 220 370 530 760 1000 6900 7300 7300
GET /api-stages/v2/ 10314 977 58 134.63088824277700 5.86830400004601 7238.486234999980 1147.6758774481300 13.163380119264600 1.246909286069570 58 73 98 130 300 460 700 990 6700 7200 7200
POST /token/ 100 0 1200.0 1533.3041962300000 291.5261090000010 5389.763017 414.0 0.12762633429575900 0.0 1200 1700 2200 2600 3100 4100 5100 5400 5400 5400 5400
  Aggregated 41114 3947 64 150.33121004502100 5.638586000031860 7268.720405000010 905.1459600136210 52.47229108235830 5.037411414653610 64 110 140 160 310 480 740 1000 6700 7200 7300

editor1 300 명 / 10명 90 RPS / 3%

Type Name Request Count Failure Count Median Response Time Average Response Time Min Response Time Max Response Time Average Content Size Requests/s Failures/s 50% 66% 75% 80% 90% 95% 98% 99% 99.9% 99.99% 100%
GET /api-nodes/v2/today/ 99567 2782 820.0 1476.4556932643800 5.610466000007360 15981.479055999900 347.5176916046480 22.986789530690500 0.6422735291249200 820 1400 2000 2400 3800 5200 6700 7800 11000 14000 16000
GET /api-nodes/waitnp/ 99897 2665 790.0 1451.160664603490 5.725932999894210 15525.64324299970 85.01374415648120 23.06297582278650 0.6152620255635910 790 1400 2000 2400 3800 5200 6700 7700 11000 13000 16000
GET /api-stages/compnp/ 99708 2659 920.0 1573.1315019551600 5.817969999952770 15864.575748000300 2257.4906928230400 23.019341855495100 0.6138768202527540 920 1500 2100 2500 3900 5200 6800 8000 11000 14000 16000
GET /api-stages/v2/ 99726 2661 820.0 1472.5697108483300 5.959252999900860 15564.513134000200 1224.7680344143000 23.023497471427700 0.6143385553563660 820 1400 2000 2400 3800 5100 6700 7700 11000 14000 16000
POST /token/ 300 0 360.0 740.7718425400010 164.27179300000000 6750.350256000000 414.0 0.06926026554186770 0.0 370 640 950 1200 1700 2300 4000 6100 6800 6800 6800
  Aggregated 399198 10767 840.0 1492.748899686140 5.610466000007360 15981.479055999900 978.0840860926160 92.16186494594170 2.4857509302976300 840 1500 2000 2400 3800 5200 6700 7800 11000 14000 16000

db.t3.medium -> db.t3.large editor1 100 명 / 10명 55 RPS / 13% -> 3% (AWS) 10 RPS / 0% (AZURE)

Type Name Request Count Failure Count Median Response Time Average Response Time Min Response Time Max Response Time Average Content Size Requests/s Failures/s 50% 66% 75% 80% 90% 95% 98% 99% 99.9% 99.99% 100%
GET /api-nodes/v2/today/ 13313 399 50 61.8762947756323 6.100723000002970 4310.306250999990 347.0467963644560 13.594002937684000 0.407421856240961 50 56 61 65 83 110 190 250 490 4000 4300
GET /api-nodes/waitnp/ 13343 402 40 51.54059074743330 5.7595849999927400 4019.4395899999900 85.14486996927230 13.62463615995780 0.41048517846833700 40 44 48 52 69 96 160 230 540 3900 4000
GET /api-stages/compnp/ 13308 418 110.0 126.43542137225700 5.829233000000050 4259.023847000010 2247.087165614670 13.588897400638400 0.4268228970143400 110 120 130 140 160 200 260 320 3700 4200 4300
GET /api-stages/v2/ 13390 414 51 63.76277299350280 5.804060000002660 4284.644386999990 1219.9692307692300 13.672628208186600 0.4227384673778390 51 57 62 66 84 110 180 240 750 4100 4300
POST /token/ 100 0 200.0 222.62413218000000 168.48655199999900 458.9761820000010 414.0 0.10211074091252200 0.0 200 230 240 250 310 380 440 460 460 460 460
  Aggregated 53454 1633 53 76.14236064492870 5.7595849999927400 4310.306250999990 973.4976989561120 54.582275447379300 1.667468399101480 53 72 97 110 130 160 210 270 730 4100 4300

editor1 300 명 / 10명 90 RPS / 10%

Type Name Request Count Failure Count Median Response Time Average Response Time Min Response Time Max Response Time Average Content Size Requests/s Failures/s 50% 66% 75% 80% 90% 95% 98% 99% 99.9% 99.99% 100%
GET /api-nodes/v2/today/ 16004 0 1200.0 1478.4707978766000 38.21566699999930 10190.954697000000 354.0 22.34920379488810 0.0 1200 1700 2100 2300 3100 3700 4500 5200 8000 10000 10000
GET /api-nodes/waitnp/ 16150 0 1200.0 1451.3970437445800 30.919324999999300 10245.885569000000 84.0 22.553089308138100 0.0 1200 1700 2100 2300 3000 3600 4400 5200 8100 9400 10000
GET /api-stages/compnp/ 16331 0 1300.0 1564.5342780853000 84.09717199999990 9587.107821999980 2316.0 22.80585148552340 0.0 1300 1800 2200 2400 3100 3800 4500 5200 8300 9100 9600
GET /api-stages/v2/ 16199 0 1200.0 1491.914219536270 39.78927900000070 9217.575831000010 1255.0 22.621516637927500 0.0 1200 1700 2100 2300 3100 3700 4500 5200 7900 8900 9200
POST /token/ 300 0 340.0 743.2888387833330 161.67678499999900 4161.897434000000 414.0 0.41894283544529000 0.0 350 710 1100 1300 1900 2300 3200 3600 4200 4200 4200
  Aggregated 64984 0 1200.0 1493.3279468023800 30.919324999999300 10245.885569000000 1004.8405299766100 90.74860406192240 0.0 1200 1700 2100 2300 3100 3700 4500 5200 8100 9400 10000
Type Name Request Count Failure Count Median Response Time Average Response Time Min Response Time Max Response Time Average Content Size Requests/s Failures/s 50% 66% 75% 80% 90% 95% 98% 99% 99.9% 99.99% 100%
GET /api-nodes/v2/today/ 34059 3066 900.0 1230.116555238980 5.6064089999381400 10190.954697000000 333.1152999207260 24.542951783955400 2.2093628752930800 900 1400 1800 2100 2900 3600 4600 5400 8000 9900 10000
GET /api-nodes/waitnp/ 34305 3181 880.0 1202.480119558760 5.418104999989740 10245.885569000000 87.52362629354320 24.72021964674800 2.292231998143280 880 1400 1800 2000 2800 3600 4500 5400 8300 10000 10000
GET /api-stages/compnp/ 34557 3174 1000.0 1318.1111824687100 5.618084000047930 9844.482753000190 2114.4849379286400 24.901811115950100 2.287187790665440 1000 1500 1900 2200 3000 3700 4700 5500 8200 9500 9800
GET /api-stages/v2/ 34302 3170 900.0 1233.558750713230 5.6556629999704500 9593.330642999950 1150.2944434726800 24.718057843543200 2.2843053863923900 900 1400 1800 2100 2900 3600 4600 5400 7700 8900 9600
POST /token/ 300 0 340.0 743.2888387833330 161.67678499999900 4161.897434000000 414.0 0.21618032047877500 0.0 350 710 1100 1300 1900 2300 3200 3600 4200 4200 4200
  Aggregated 137523 12591 920.0 1245.13068583087 5.418104999989740 10245.885569000000 923.4813667531980 99.09922071067540 9.073088050494210 920 1400 1800 2100 2900 3600 4600 5400 8000 9600 10000

editor1 500 명 / 10명 90 RPS / 0%

Type Name Request Count Failure Count Median Response Time Average Response Time Min Response Time Max Response Time Average Content Size Requests/s Failures/s 50% 66% 75% 80% 90% 95% 98% 99% 99.9% 99.99% 100%
GET /api-nodes/v2/today/ 20480 0 2700.0 3584.9273383480200 37.132899000001300 18517.86980700000 354.0 22.95974382615370 0.0 2700 4200 5400 6100 8200 9900 12000 13000 16000 17000 19000
GET /api-nodes/waitnp/ 20262 0 2700.0 3535.076601283410 31.95941900000320 18437.848146000000 84.0 22.715348115504200 0.0 2700 4200 5300 6000 8100 9800 12000 13000 15000 17000 18000
GET /api-stages/compnp/ 19852 0 2800.0 3679.803336032730 84.3128550000003 18240.865302000000 2316.0 22.255704806484500 0.0 2800 4300 5400 6200 8300 9900 12000 13000 16000 18000 18000
GET /api-stages/v2/ 20320 0 2700.0 3540.5730417507400 40.54314200000060 17511.644031000000 1255.0 22.780370827511800 0.0 2700 4100 5200 6000 8100 9800 12000 13000 16000 17000 18000
POST /token/ 500 0 860.0 1487.875621936000 162.0679850000000 9930.004547000000 414.0 0.5605406207557040 0.0 870 1600 2200 2500 3800 5200 6500 7200 9900 9900 9900
  Aggregated 81414 0 2800.0 3571.70599443389 31.95941900000320 18517.86980700000 990.4655219004100 91.27170819640980 0.0 2800 4200 5300 6100 8200 9800 12000 13000 16000 17000 19000

Instance type: t3.xlarge -> t3.2xlarge

editor1 100 명 / 10명 55 RPS / 0%

editor1 300 명 / 10명 95 RPS / 15%

editor1 500 명 / 10명 95 RPS / 0%

Instance type: t3.2xlarge -> t3.xlarge

editor1 100 명 / 10명 55 RPS / 10%

editor1 300 명 / 10명 95 RPS / 0%

editor1 500 명 / 10명 95 RPS / 0%

parksunwoo commented 3 years ago

STAGE

Instance type: t3.xlarge db type : db.t3.large

변경된 테스트 시나리오

300 명 / 10명 20 ~ 25 RPS / 0%

400 명 / 10명 20 ~ 25 RPS / 0%

500 명 / 10명 20 RPS / 7%