poganesh
commented
3 months ago @mattchai Could you please check the versions of the following libraries in your conda environment:
- transformers==4.37.2
- torch==2.1.2
- numpy==1.26.4
- tokenizers==0.15.2
You can check the versions by running pip show transformers torch numpy tokenizers. Additionally, please refer to the env.yaml and requirements.txt which lists the specific versions of the libraries used in the environment. If the issue persists, please share your pip list output.
mattchai
shivani-athavale
uday610
model_name: llama-2-7b-chat [load_smoothquant_model] model loaded ... modules.json: 100%|███████████████████████████████████████████████████████████████████████████| 349/349 [00:00<?, ?B/s] config_sentence_transformers.json: 100%|██████████████████████████████████████████████████████| 124/124 [00:00<?, ?B/s] README.md: 100%|███████████████████████████████████████████████████████████████████| 94.8k/94.8k [00:00<00:00, 373kB/s] sentence_bert_config.json: 100%|████████████████████████████████████████████████████████████| 52.0/52.0 [00:00<?, ?B/s] config.json: 100%|████████████████████████████████████████████████████████████████████████████| 743/743 [00:00<?, ?B/s] model.safetensors: 100%|████████████████████████████████████████████████████████████| 133M/133M [00:08<00:00, 16.3MB/s] tokenizer_config.json: 100%|██████████████████████████████████████████████████████████████████| 366/366 [00:00<?, ?B/s] vocab.txt: 100%|█████████████████████████████████████████████████████████████████████| 232k/232k [00:00<00:00, 356kB/s] tokenizer.json: 100%|████████████████████████████████████████████████████████████████| 711k/711k [00:00<00:00, 857kB/s] special_tokens_map.json: 100%|████████████████████████████████████████████████████████████████| 125/125 [00:00<?, ?B/s] 1_Pooling/config.json: 100%|██████████████████████████████████████████████████████████████████| 190/190 [00:00<?, ?B/s] Creating new index. Running on local URL: http://localhost:7860
To create a public link, set
share=Trueinlaunch().Context information is below.file_path: C:\RyzenAI-SW\example\transformers\models\rag\dataset\relnotes.rst
NPU and Compiler
Extended the support range of some operators
Improvement in device partition
Demos
New Demos link: https://account.amd.com/en/forms/downloads/ryzen-ai-software-platform-xef.html?filename=transformers_2308.zip
Known issues
Version 0.7
Quantizer
Docker Containers
Pytorch Quantizer
ONNX Quantizer
Provided Python wheel file for installation
Supports quantizing ONNX models for NPU as a plugin for the ONNX Runtime native quantizer
Supports power-of-two quantization with both QDQ and QOP format
Supports Non-overflow and Min-MSE quantization methods
Supports various quantization configurations in power-of-two quantization in both QDQ and QOP format.
Supports signed and unsigned configurations.
Supports symmetry and asymmetry configurations.
Supports per-tensor and per-channel configurations.
Supports bias quantization using int8 datatype for NPU.
Supports quantization parameters (scale) refinement for NPU.
Supports excluding certain operations from quantization for NPU.
Supports ONNX models larger than 2GB.
Supports using CUDAExecutionProvider for calibration in quantization
Open source and upstreamed to Microsoft Olive Github repo
TensorFlow 2.x Quantizer
TensorFlow 1.x Quantizer
file_path: C:\RyzenAI-SW\example\transformers\models\rag\dataset\getstartex.rst
I20231129 13:19:57.389281 14796 PartitionPass.cpp:6142] xir::Op{name = output_, type = fix2float} is not supported by current target. Target name: AMD_AIE2_Nx4_Overlay, target type: IPU_PHX. Assign it to CPU. I20231129 13:19:58.546655 14796 compile_pass_manager.cpp:565] Total device subgraph number 3, CPU subgraph number 1 I20231129 13:19:58.546655 14796 compile_pass_manager.cpp:574] Total device subgraph number 3, DPU subgraph number 1 I20231129 13:19:58.546655 14796 compile_pass_manager.cpp:583] Total device subgraph number 3, USER subgraph number 1 I20231129 13:19:58.547658 14796 compile_pass_manager.cpp:639] Compile done. I20231129 13:19:58.583139 14796 anchor_point.cpp:444] before optimization: ... [Vitis AI EP] No. of Operators : CPU 2 IPU 398 99.50% [Vitis AI EP] No. of Subgraphs : CPU 1 IPU 1 Actually running on IPU 1 ... Final results: Predicted label is cat and actual label is cat Predicted label is ship and actual label is ship Predicted label is ship and actual label is ship Predicted label is airplane and actual label is airplane Predicted label is frog and actual label is frog Predicted label is frog and actual label is frog Predicted label is truck and actual label is automobile Predicted label is frog and actual label is frog Predicted label is cat and actual label is cat Predicted label is automobile and actual label is automobile
..
##################################### License #####################################
Ryzen AI is licensed under
MIT License <https://github.com/amd/ryzen-ai-documentation/blob/main/License>. Refer to theLICENSE File <https://github.com/amd/ryzen-ai-documentation/blob/main/License>for the full license text and copyright notice.Given the context information and not prior knowledge, answer the query. Query: who are you Answer: Traceback (most recent call last): File "C:\ProgramData\anaconda3\envs\ryzenai-transformers\Lib\site-packages\gradio\queueing.py", line 521, in process_events response = await route_utils.call_process_api( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ProgramData\anaconda3\envs\ryzenai-transformers\Lib\site-packages\gradio\route_utils.py", line 276, in call_process_api output = await app.get_blocks().process_api( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ProgramData\anaconda3\envs\ryzenai-transformers\Lib\site-packages\gradio\blocks.py", line 1945, in process_api result = await self.call_function( ^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ProgramData\anaconda3\envs\ryzenai-transformers\Lib\site-packages\gradio\blocks.py", line 1511, in call_function prediction = await fn(processed_input) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ProgramData\anaconda3\envs\ryzenai-transformers\Lib\site-packages\gradio\utils.py", line 798, in async_wrapper response = await f(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ProgramData\anaconda3\envs\ryzenai-transformers\Lib\site-packages\gradio\chat_interface.py", line 516, in _submit_fn response = await anyio.to_thread.run_sync( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ProgramData\anaconda3\envs\ryzenai-transformers\Lib\site-packages\anyio\to_thread.py", line 56, in run_sync return await get_async_backend().run_sync_in_worker_thread( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ProgramData\anaconda3\envs\ryzenai-transformers\Lib\site-packages\anyio_backends_asyncio.py", line 2177, in run_sync_in_worker_thread return await future ^^^^^^^^^^^^ File "C:\ProgramData\anaconda3\envs\ryzenai-transformers\Lib\site-packages\anyio_backends_asyncio.py", line 859, in run result = context.run(func, args) ^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\RyzenAI-SW\example\transformers\models\rag\run.py", line 81, in prompt response_str = query_engine.query(query_text) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ProgramData\anaconda3\envs\ryzenai-transformers\Lib\site-packages\llama_index\core\instrumentation\dispatcher.py", line 223, in wrapper result = func(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "C:\ProgramData\anaconda3\envs\ryzenai-transformers\Lib\site-packages\llama_index\core\base\base_query_engine.py", line 51, in query query_result = self._query(str_or_query_bundle) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ProgramData\anaconda3\envs\ryzenai-transformers\Lib\site-packages\llama_index\core\instrumentation\dispatcher.py", line 223, in wrapper result = func(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "C:\ProgramData\anaconda3\envs\ryzenai-transformers\Lib\site-packages\llama_index\core\query_engine\retriever_query_engine.py", line 190, in _query response = self._response_synthesizer.synthesize( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ProgramData\anaconda3\envs\ryzenai-transformers\Lib\site-packages\llama_index\core\instrumentation\dispatcher.py", line 223, in wrapper result = func(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "C:\ProgramData\anaconda3\envs\ryzenai-transformers\Lib\site-packages\llama_index\core\response_synthesizers\base.py", line 240, in synthesize response_str = self.get_response( ^^^^^^^^^^^^^^^^^^ File "C:\ProgramData\anaconda3\envs\ryzenai-transformers\Lib\site-packages\llama_index\core\instrumentation\dispatcher.py", line 223, in wrapper result = func(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "C:\ProgramData\anaconda3\envs\ryzenai-transformers\Lib\site-packages\llama_index\core\response_synthesizers\compact_and_refine.py", line 43, in get_response return super().get_response( ^^^^^^^^^^^^^^^^^^^^^ File "C:\ProgramData\anaconda3\envs\ryzenai-transformers\Lib\site-packages\llama_index\core\instrumentation\dispatcher.py", line 223, in wrapper result = func(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "C:\ProgramData\anaconda3\envs\ryzenai-transformers\Lib\site-packages\llama_index\core\response_synthesizers\refine.py", line 183, in get_response response = self._give_response_single( ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ProgramData\anaconda3\envs\ryzenai-transformers\Lib\site-packages\llama_index\core\response_synthesizers\refine.py", line 238, in _give_response_single program( File "C:\ProgramData\anaconda3\envs\ryzenai-transformers\Lib\site-packages\llama_index\core\response_synthesizers\refine.py", line 84, in call answer = self._llm.predict( ^^^^^^^^^^^^^^^^^^ File "C:\ProgramData\anaconda3\envs\ryzenai-transformers\Lib\site-packages\llama_index\core\instrumentation\dispatcher.py", line 223, in wrapper result = func(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "C:\ProgramData\anaconda3\envs\ryzenai-transformers\Lib\site-packages\llama_index\core\llms\llm.py", line 438, in predict response = self.complete(formatted_prompt, formatted=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ProgramData\anaconda3\envs\ryzenai-transformers\Lib\site-packages\llama_index\core\instrumentation\dispatcher.py", line 223, in wrapper result = func(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "C:\ProgramData\anaconda3\envs\ryzenai-transformers\Lib\site-packages\llama_index\core\llms\callbacks.py", line 389, in wrapped_llm_predict f_return_val = f(_self, *args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\RyzenAI-SW\example\transformers\models\rag\custom_llm.py", line 266, in complete response = self.generate_response(prompt) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\RyzenAI-SW\example\transformers\models\rag\custom_llm.py", line 240, in generate_response resp = self.decode_prompt1(prompt, max_new_tokens=m, do_sample=do_sample, temperature=temperature) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\RyzenAI-SW\example\transformers\models\rag\custom_llm.py", line 167, in decode_prompt1 generate_ids = self.model.generate( ^^^^^^^^^^^^^^^^^^^^ File "C:\ProgramData\anaconda3\envs\ryzenai-transformers\Lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\test\AppData\Roaming\Python\Python311\site-packages\transformers\generation\utils.py", line 1989, in generate result = self._sample( ^^^^^^^^^^^^^ File "C:\Users\test\AppData\Roaming\Python\Python311\site-packages\transformers\generation\utils.py", line 2932, in _sample outputs = self(model_inputs, return_dict=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ProgramData\anaconda3\envs\ryzenai-transformers\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl return self._call_impl(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ProgramData\anaconda3\envs\ryzenai-transformers\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl return forward_call(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\RyzenAI-SW\example\transformers\tools\llm_eval.py", line 79, in forward outputs = super().forward(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\test\AppData\Roaming\Python\Python311\site-packages\transformers\models\llama\modeling_llama.py", line 1141, in forward outputs = self.model( ^^^^^^^^^^^ File "C:\ProgramData\anaconda3\envs\ryzenai-transformers\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl return self._call_impl(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ProgramData\anaconda3\envs\ryzenai-transformers\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl return forward_call(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\test\AppData\Roaming\Python\Python311\site-packages\transformers\models\llama\modeling_llama.py", line 944, in forward layer_outputs = decoder_layer( ^^^^^^^^^^^^^^ File "C:\ProgramData\anaconda3\envs\ryzenai-transformers\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ProgramData\anaconda3\envs\ryzenai-transformers\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl return forward_call(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\test\AppData\Roaming\Python\Python311\site-packages\transformers\models\llama\modeling_llama.py", line 677, in forward hidden_states, self_attn_weights, present_key_value = self.self_attn( ^^^^^^^^^^^^^^^ File "C:\ProgramData\anaconda3\envs\ryzenai-transformers\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ProgramData\anaconda3\envs\ryzenai-transformers\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl return forward_call(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\RyzenAI-SW\example\transformers\ops\python\llama_flash_attention.py", line 198, in forward cos, sin = self.rotary_emb(value_states, seq_len=kv_seq_len) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ProgramData\anaconda3\envs\ryzenai-transformers\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl return self._call_impl(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ProgramData\anaconda3\envs\ryzenai-transformers\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl return forward_call(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ProgramData\anaconda3\envs\ryzenai-transformers\Lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ TypeError: LlamaRotaryEmbedding.forward() got an unexpected keyword argument 'seq_len'