bug: could not load library ~\cortex/engines/cortex.tensorrt-llm/engine.dll when chatting with model

[0xSage]

Describe the bug A clear and concise description of what the bug is.

To Reproduce

1. cortex run llama3:tensorrt-llm-windows-ada (valid model branch this time)

2. check that model dlls , binaries all exist (model downloaded successfully)

   √ Model loaded
   In order to exit, type 'exit()'.
   >> hi
   Something went wrong! Please check model status.
   500 Request failed with status code 500

3. When checking logs

   {"method":"POST","path":"/v1/threads","statusCode":201,"ip":"127.0.0.1","content_length":"483","user_agent":"CortexClient/JS 0.1.7","x_correlation_id":""} HTTP
   {"method":"GET","path":"/v1/threads/1f597d11-5e1c-41c8-b1ec-23d1ae8339be/messages?limit=10&order=desc","statusCode":200,"ip":"127.0.0.1","content_length":"44","user_agent":"CortexClient/JS 0.1.7","x_correlation_id":""} HTTP
   Validating app logs. Next attempt in  120000
   Validating app logs. Next attempt in  120000
   Validating app logs. Next attempt in  120000
   {"method":"GET","path":"/v1/models/llama3:tensorrt-llm-windows-ada","statusCode":200,"ip":"127.0.0.1","content_length":"690","user_agent":"CortexClient/JS 0.1.7","x_correlation_id":""} HTTP
   - Loading model...
   20240815 15:53:40.063000 UTC 1424 WARN  Engine is not loaded yet - server.cc:166
   20240815 15:53:40.080000 UTC 1424 ERROR Could not load engine: Could not load library "C:\Users\n\cortex/engines/cortex.tensorrt-llm/engine.dll"
   The specified module could not be found.
   
   - server.cc:299
   × Model loading failed
   {"method":"POST","path":"/v1/threads/1f597d11-5e1c-41c8-b1ec-23d1ae8339be/messages","statusCode":201,"ip":"127.0.0.1","content_length":"376","user_agent":"CortexClient/JS 0.1.7","x_correlation_id":""} HTTP
   {"method":"POST","path":"/v1/chat/completions","statusCode":500,"ip":"127.0.0.1","content_length":"35","user_agent":"CortexClient/JS 0.1.7","x_correlation_id":""} HTTP
   - Loading model...
   20240815 15:53:43.167000 UTC 1424 WARN  Engine is not loaded yet - server.cc:166
   20240815 15:53:43.169000 UTC 1424 ERROR Could not load engine: Could not load library "C:\Users\n\cortex/engines/cortex.tensorrt-llm/engine.dll"
   The specified module could not be found.
   
   - server.cc:299
   × Model loading failed
   {"method":"POST","path":"/v1/chat/completions","statusCode":500,"ip":"127.0.0.1","content_length":"35","user_agent":"CortexClient/JS 0.1.7","x_correlation_id":""} HTTP
   - Loading model...
   20240815 15:53:44.080000 UTC 1424 WARN  Engine is not loaded yet - server.cc:166
   20240815 15:53:44.082000 UTC 1424 ERROR Could not load engine: Could not load library "C:\Users\n\cortex/engines/cortex.tensorrt-llm/engine.dll"
   The specified module could not be found.
   
   - server.cc:299
   × Model loading failed
   {"method":"POST","path":"/v1/chat/completions","statusCode":500,"ip":"127.0.0.1","content_length":"35","user_agent":"CortexClient/JS 0.1.7","x_correlation_id":""} HTTP
   Validating app logs. Next attempt in  120000
   Validating app logs. Next attempt in  120000

Expected behavior A clear and concise description of what you expected to happen.

Screenshots If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: [e.g. iOS]
• Browser [e.g. chrome, safari]
• Version [e.g. 22]

Smartphone (please complete the following information):

• Device: [e.g. iPhone6]
• OS: [e.g. iOS8.1]
• Browser [e.g. stock browser, safari]
• Version [e.g. 22]

Additional context Add any other context about the problem here.

[louis-jan]

The same root cause with Failed to init tensorrt-llm. Fixed - Wrong binary lookup.

Could not load library "C:\Users\n\cortex/engines/cortex.tensorrt-llm/engine.dll" The specified module could not be found.

[imtuyethan]

I saw the PR is merged, it's fixed right? @louis-jan

[louis-jan]

Yes @imtuyethan

[louis-jan]

From now on, cortex-cpp will handle engine stuff. I will pass it over to @vansangpfiev. (Not to take action, just to follow up when release)