-
I calculate conv2d in both pytorch and trt-llm, the result is quiet different.
```
env:
ubuntu 20.04
Tesla V100
Driver Version: 470.103.01
TensorRT-LLM: a75618df24e97ecf92b8…
-
Hi do you have support for scanning text / prompts in batches?
I am thinking about something like presidios `BatchAnalyzerEngine`.
Right now llm-guard can efficiently only be used for single prompts…
-
bigdl-core-xe 2.4.0b20231101
bigdl-core-xe-esimd 2.4.0b20231101
bigdl-llm 2.4.0b20231101
-----------------------------------------------------
the second in…
-
I recently encountered documentation that was clearly written by a Large Language Model (LLM) in a JOSS submission. I wondered what the policy of JOSS is on the use of LLMs, on authorship etc.
Pers…
-
API: https://github.com/NVIDIA/NeMo-Guardrails?tab=readme-ov-file#guardrails-server
Related issue: https://github.com/openhackathons-org/End-to-End-LLM/issues/16
-
### Feature Description
Feature Description
Now the LLM application answer questions on anything without restricting to the vectorised topics.
It will be a great add on if Nemo Guard rails like fea…
-
Is it possible to create a memgpt feature and make it available to all the agents rather than having a separate agent like it's discussed in #530?
-
bigdl-core-xe 2.4.0b20231101
bigdl-core-xe-esimd 2.4.0b20231101
bigdl-llm 2.4.0b20231101
-------------------------------------------------------…
-
### System Info
hi,
i generated the tensorrt llm engine for a llama based model and see that the performance is much worse than vllm.
i did the following:
- compile model with tensorrt llm c…
-
**Title:** Extend Chirps to Scan LLM APIs for Security Issues
**Description:**
Chirps currently provides functionality to scan next-generation AI systems and checks for issues with the vectorDB.…