-
Error when running sample **python3 examples/offline_inference_neuron.py**, after installing v0.3.3 (from cloned source or from pip install git+...).
### Cause:
directory **vllm/model_executor/m…
-
I want to know how we can run the speculative decoding(Assisted Generation) to increase the token/sec for llama2 based model for optimum.neuron to run on inf2. Similar to what transformers have done f…
-
Hello Spikeinterface team,
We’ve been working on extracting single-neuron activity from Neuropixels data, which often requires extensive manual evaluation of spike clusters.
To streamline this …
-
I started a `inf2.48xlarge` ec2, pull and get into [TGI-Neuron DLC with optimum-neuron 0.0.17 installed](https://github.com/aws/deep-learning-containers/releases/tag/v1.0-hf-tgi-0.0.17-pt-1.13.1-inf-n…
-
Trying to run t5 model using optimum neuron and run_summarization script leads to failure.
4 experiments were run:
1. ON 0.0.18 using the script on ON github
2. ON 0.0.14 using the script on ON gi…
-
When compiling models using optimum-cli, it supports many input parameters that are not supported by the Python Wrappers, for instance:
When using optimum-cli, you can use parameters like --disable…
-
### System Info
```shell
Hugging Face Neuron Deep Learning AMI (Ubuntu 22.04)[ami=ami-073e0687022c65b38 ]
Instance type: Inf2.48xlarge
pre-installed package:
aws_neuron_venv_pytorch) ubun…
-
As there is more demand for LoRA based fine-tuning of models, we would like to have support for it in Optimum Neuron to optimize the user experience. We need to make sure that it can effectively use t…
-
Hi, I'm following the sample [here](https://github.com/aws-neuron/aws-neuron-sagemaker-samples/blob/master/inference/inf2-bert-on-sagemaker/inf2_bert_sagemaker.ipynb) to try to compile a model to Neur…
-
An issue was posted to the NEURON forum
https://www.neuron.yale.edu/phpBB/viewtopic.php?t=4694
The author writes:
> I am attempting to compile the mod files with the command nrnivmodl . for a singl…