Error: No such file or directory

casys-kaist / LLMServingSim

LLMServingSim: A HW/SW Co-Simulation Infrastructure for LLM Inference Serving at Scale

MIT License

58 stars 8 forks source link

Error: No such file or directory #3

Open lhpp1314 opened 2 months ago

lhpp1314 commented 2 months ago

Hello! I use this simulator for LLM serving, but when I run the following cmd:

python3 -u main.py --model_name 'gpt3-6.7b' --npu_num 1 --npu_group 1 --npu_mem 24 --dataset 'dataset/share-gpt-req100-rate10.tsv' --fast_run

An error occurs:

ERROR [09/17/2024 09:29:18 PM] [Errno 2] No such file or directory: '../../../inputs/custom_workload/gpt3-6.7b_b1_s94_orca_n1.txt'

Do you have any idea how this happen?

JaehongCS20 commented 1 month ago

This issue is the same as the one mentioned in https://github.com/casys-kaist/LLMServingSim/issues/2. The problem occurs because there is no NPU simulator, and thus no pre-generated simulator_result for that command. If you run the commands in the evaluation folder instead of those in the test run, it should work correctly.

We are receiving many questions about the NPU simulator, and we plan to provide a detailed explanation on this matter soon.