NVIDIA / TensorRT-LLM

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.
https://nvidia.github.io/TensorRT-LLM
Apache License 2.0
8.42k stars 950 forks source link

need a copy code widget to be able to copy code snippets #2288

Open stas00 opened 1 week ago

stas00 commented 1 week ago

Is it possible to add to https://nvidia.github.io/TensorRT-LLM/ the code copy widget that you already have on https://nvidia.github.io/TensorRT-Model-Optimizer/? For example if you go to https://nvidia.github.io/TensorRT-Model-Optimizer/guides/_pytorch_quantization.html#apply-post-training-quantization-ptq you can copy the code

Image

But if you go to https://nvidia.github.io/TensorRT-LLM/llm-api-examples/index.html#llm-examples-introduction you can't - manual copy-n-paste results in double new lines for me.

Image

Thank you!

I think it's the way the HTML is encoded which breaks each line, because you have 2 spans per line

<span class="linenos"> 2</span>
<span class="linenos"> 3</span><span class="n">prompts</span> <span class="o">=</span> <span class="p">[</span>
<span class="linenos"> 4</span>    <span class="s2">&quot;Hello, my name is&quot;</span><span class="p">,</span>
<span class="linenos"> 5</span>    <span class="s2">&quot;The president of the United States is&quot;</span><span class="p">,</span>
<span class="linenos"> 6</span>    <span class="s2">&quot;The capital of France is&quot;</span><span class="p">,</span>
<span class="linenos"> 7</span>    <span class="s2">&quot;The future of AI is&quot;</span><span class="p">,</span>
<span class="linenos"> 8</span><span class="p">]</span>
nv-guomingz commented 1 day ago

Hi @stas00 , thanks for remining us on this missing feature. We'll enable the copy button on next release and Tensor Tuesday(weekly release)

stas00 commented 16 hours ago

Thank you, @nv-guomingz!