Open littletomatodonkey opened 2 months ago
Hi, I tried with latest main and it seems OK, could you please try with that? Thanks!
BTW, with latest main, C++ runtime can also be used by removing --use_py_session
.
Hi @littletomatodonkey , do u still encouter such issue w/ @dongxuy04 's suggestion? If not, I'll close this ticket.
Hi, when i use medusa decoding on trtllm-090 which profiling, error occrued as follows. Could you please help to have a look? Thanks!
If i do not use
--run_profiling
, the inference process is normal.