search

casys-kaist / LLMServingSim

LLMServingSim: A HW/SW Co-Simulation Infrastructure for LLM Inference Serving at Scale
MIT License
61 stars 8 forks source link

ERROR 'Layer' object has no attribute 'input_memory_loc' #4

Open GyeonggeunJung opened 2 months ago

GyeonggeunJung commented 2 months ago

I am currently using the LLM Serving Sim paper to see how this simulator works, and I am using it to figure out the results of the simulator according to the settings.

However, the following error occurred while using the simulator. The message is as follows : ERROR 'Layer' object has no attribute 'input_memory_loc'

In the case of Test Run on Github, the next request is created even if an error occurs, but in the case of other simulations, if an error occurs, the simulation stops and it does not stop with Throughput 0. How can I resolve this error?

JaehongCS20 commented 2 months ago

The error message occurs in the following file: https://github.com/JaehongCS20/chakra/blob/687651ab0cfffc83aa6073c200b4fef791ca9e89/et_converter/llm_chakra_converter.py.

The error arises during the process of converting the simulation trace (.txt) file into a chakra graph, as the layer does not contain the necessary attribute.

Please check why the layer lacks the corresponding attribute by reviewing both the llm_chakra_converter.py file and the corresponding simulation trace (.txt) file together.

If the issue is related to the absence of the NPU simulator, meaning the trace for the given situation does not exist, please refer to https://github.com/casys-kaist/LLMServingSim/issues/2 for further information.