search

run-llama / llama-hub

A library of data loaders for LLMs made by the community -- to be used with LlamaIndex and/or LangChain
https://llamahub.ai/
MIT License
3.42k stars 727 forks source link

Pandas Query Engine with Mistral[Bug]: #958

Open jaideep11061982 opened 4 months ago

jaideep11061982 commented 4 months ago

Bug Description

I get Cuda device assert error while trying to run locally hosted mistral model using Query engine like chain of table and native Pandas query engine

Version

0.10.5

Steps to Reproduce

from llama_index.llms.huggingface import (
    HuggingFaceInferenceAPI,
    HuggingFaceLLM,
)

locally_run = HuggingFaceLLM(model_name="/kaggle/input/mistral/pytorch/7b-v0.1-hf/1")
from llama_index.core.query_engine import PandasQueryEngine
query_engine = PandasQueryEngine(df, llm=locally_run, verbose=True)
response = query_engine.query("How many males survived in ship ?")

Relevant Logs/Tracbacks

etting `pad_token_id` to `eos_token_id`:2 for open-end generation.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [96,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [97,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [98,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [99,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [100,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [101,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [102,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [103,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [104,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [105,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [106,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [107,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [108,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [109,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [110,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [111,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [112,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [113,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [114,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [115,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [116,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [117,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [118,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [119,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [120,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [121,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [122,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [123,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [124,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [125,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [126,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [127,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [0,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [1,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [2,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [3,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [4,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [5,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [6,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [7,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [8,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [9,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [10,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [11,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [12,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [13,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [14,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [15,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [16,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [17,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [18,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [19,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [20,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [21,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [22,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [23,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [24,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [25,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [26,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [27,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [28,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [29,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [30,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [31,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [32,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [33,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [34,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [35,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [36,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [37,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [38,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [39,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [40,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [41,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [42,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [43,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [44,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [45,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [46,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [47,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [48,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [49,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [50,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [51,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [52,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [53,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [54,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [55,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [56,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [57,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [58,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [59,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [60,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [61,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [62,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [188,0,0], thread: [63,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [182,0,0], thread: [64,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [182,0,0], thread: [65,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [182,0,0], thread: [66,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [182,0,0], thread: [67,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [182,0,0], thread: [68,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [182,0,0], thread: [69,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [182,0,0], thread: [70,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [182,0,0], thread: [71,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [182,0,0], thread: [72,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [182,0,0], thread: [73,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [182,0,0], thread: [74,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [182,0,0], thread: [75,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [182,0,0], thread: [76,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [182,0,0], thread: [77,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [182,0,0], thread: [78,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [182,0,0], thread: [79,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [182,0,0], thread: [80,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [182,0,0], thread: [81,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [182,0,0], thread: [82,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [182,0,0], thread: [83,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [182,0,0], thread: [84,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [182,0,0], thread: [85,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [182,0,0], thread: [86,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [182,0,0], thread: [87,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [182,0,0], thread: [88,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [182,0,0], thread: [89,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [182,0,0], thread: [90,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [182,0,0], thread: [91,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [182,0,0], thread: [92,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [182,0,0], thread: [93,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [182,0,0], thread: [94,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [182,0,0], thread: [95,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
Cell In[5], line 3
      1 #query_engine = ChainOfTableQueryEngine(df, llm=locally_run, verbose=True)
      2 query_engine = PandasQueryEngine(df, llm=locally_run, verbose=True)
----> 3 response = query_engine.query("How many males survived in ship ?")

File /opt/conda/lib/python3.10/site-packages/llama_index/core/base/base_query_engine.py:40, in BaseQueryEngine.query(self, str_or_query_bundle)
     38 if isinstance(str_or_query_bundle, str):
     39     str_or_query_bundle = QueryBundle(str_or_query_bundle)
---> 40 return self._query(str_or_query_bundle)

File /opt/conda/lib/python3.10/site-packages/llama_index/core/query_engine/pandas/pandas_query_engine.py:156, in PandasQueryEngine._query(self, query_bundle)
    153 """Answer a query."""
    154 context = self._get_table_context()
--> 156 pandas_response_str = self._llm.predict(
    157     self._pandas_prompt,
    158     df_str=context,
    159     query_str=query_bundle.query_str,
    160     instruction_str=self._instruction_str,
    161 )
    163 if self._verbose:
    164     print_text(f"> Pandas Instructions:\n" f"\n{pandas_response_str}\n```\n")

File /opt/conda/lib/python3.10/site-packages/llama_index/core/llms/llm.py:257, in LLM.predict(self, prompt, **prompt_args)
    255 else:
    256     formatted_prompt = self._get_prompt(prompt, **prompt_args)
--> 257     response = self.complete(formatted_prompt, formatted=True)
    258     output = response.text
    260 return self._parse_output(output)

File /opt/conda/lib/python3.10/site-packages/llama_index/core/llms/callbacks.py:219, in llm_completion_callback.<locals>.wrap.<locals>.wrapped_llm_predict(_self, *args, **kwargs)
    209 with wrapper_logic(_self) as callback_manager:
    210     event_id = callback_manager.on_event_start(
    211         CBEventType.LLM,
    212         payload={
   (...)
    216         },
    217     )
--> 219     f_return_val = f(_self, *args, **kwargs)
    220     if isinstance(f_return_val, Generator):
    221         # intercept the generator and add a callback to the end
    222         def wrapped_gen() -> CompletionResponseGen:

File /opt/conda/lib/python3.10/site-packages/llama_index/llms/huggingface/base.py:281, in HuggingFaceLLM.complete(self, prompt, formatted, **kwargs)
    278     if key in inputs:
    279         inputs.pop(key, None)
--> 281 tokens = self._model.generate(
    282     **inputs,
    283     max_new_tokens=self.max_new_tokens,
    284     stopping_criteria=self._stopping_criteria,
    285     **self.generate_kwargs,
    286 )
    287 completion_tokens = tokens[0][inputs["input_ids"].size(1) :]
    288 completion = self._tokenizer.decode(completion_tokens, skip_special_tokens=True)

File /opt/conda/lib/python3.10/site-packages/torch/utils/_contextlib.py:115, in context_decorator.<locals>.decorate_context(*args, **kwargs)
    112 @functools.wraps(func)
    113 def decorate_context(*args, **kwargs):
    114     with ctx_factory():
--> 115         return func(*args, **kwargs)

File /opt/conda/lib/python3.10/site-packages/transformers/generation/utils.py:1474, in GenerationMixin.generate(self, inputs, generation_config, logits_processor, stopping_criteria, prefix_allowed_tokens_fn, synced_gpus, assistant_model, streamer, negative_prompt_ids, negative_prompt_attention_mask, **kwargs)
   1457     return self.assisted_decoding(
   1458         input_ids,
   1459         candidate_generator=candidate_generator,
   (...)
   1470         **model_kwargs,
   1471     )
   1472 if generation_mode == GenerationMode.GREEDY_SEARCH:
   1473     # 11. run greedy search
-> 1474     return self.greedy_search(
   1475         input_ids,
   1476         logits_processor=prepared_logits_processor,
   1477         stopping_criteria=prepared_stopping_criteria,
   1478         pad_token_id=generation_config.pad_token_id,
   1479         eos_token_id=generation_config.eos_token_id,
   1480         output_scores=generation_config.output_scores,
   1481         return_dict_in_generate=generation_config.return_dict_in_generate,
   1482         synced_gpus=synced_gpus,
   1483         streamer=streamer,
   1484         **model_kwargs,
   1485     )
   1487 elif generation_mode == GenerationMode.CONTRASTIVE_SEARCH:
   1488     if not model_kwargs["use_cache"]:

File /opt/conda/lib/python3.10/site-packages/transformers/generation/utils.py:2335, in GenerationMixin.greedy_search(self, input_ids, logits_processor, stopping_criteria, max_length, pad_token_id, eos_token_id, output_attentions, output_hidden_states, output_scores, return_dict_in_generate, synced_gpus, streamer, **model_kwargs)
   2332 model_inputs = self.prepare_inputs_for_generation(input_ids, **model_kwargs)
   2334 # forward pass to get next token
-> 2335 outputs = self(
   2336     **model_inputs,
   2337     return_dict=True,
   2338     output_attentions=output_attentions,
   2339     output_hidden_states=output_hidden_states,
   2340 )
   2342 if synced_gpus and this_peer_finished:
   2343     continue  # don't waste resources running the code we don't need

File /opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py:1518, in Module._wrapped_call_impl(self, *args, **kwargs)
   1516     return self._compiled_call_impl(*args, **kwargs)  # type: ignore[misc]
   1517 else:
-> 1518     return self._call_impl(*args, **kwargs)

File /opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py:1527, in Module._call_impl(self, *args, **kwargs)
   1522 # If we don't have any hooks, we want to skip the rest of the logic in
   1523 # this function, and just call forward.
   1524 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
   1525         or _global_backward_pre_hooks or _global_backward_hooks
   1526         or _global_forward_hooks or _global_forward_pre_hooks):
-> 1527     return forward_call(*args, **kwargs)
   1529 try:
   1530     result = None

File /opt/conda/lib/python3.10/site-packages/accelerate/hooks.py:165, in add_hook_to_module.<locals>.new_forward(module, *args, **kwargs)
    163         output = module._old_forward(*args, **kwargs)
    164 else:
--> 165     output = module._old_forward(*args, **kwargs)
    166 return module._hf_hook.post_forward(module, output)

File /opt/conda/lib/python3.10/site-packages/transformers/models/mistral/modeling_mistral.py:1154, in MistralForCausalLM.forward(self, input_ids, attention_mask, position_ids, past_key_values, inputs_embeds, labels, use_cache, output_attentions, output_hidden_states, return_dict)
   1151 return_dict = return_dict if return_dict is not None else self.config.use_return_dict
   1153 # decoder outputs consists of (dec_features, layer_state, dec_hidden, dec_attn)
-> 1154 outputs = self.model(
   1155     input_ids=input_ids,
   1156     attention_mask=attention_mask,
   1157     position_ids=position_ids,
   1158     past_key_values=past_key_values,
   1159     inputs_embeds=inputs_embeds,
   1160     use_cache=use_cache,
   1161     output_attentions=output_attentions,
   1162     output_hidden_states=output_hidden_states,
   1163     return_dict=return_dict,
   1164 )
   1166 hidden_states = outputs[0]
   1167 logits = self.lm_head(hidden_states)

File /opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py:1518, in Module._wrapped_call_impl(self, *args, **kwargs)
   1516     return self._compiled_call_impl(*args, **kwargs)  # type: ignore[misc]
   1517 else:
-> 1518     return self._call_impl(*args, **kwargs)

File /opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py:1527, in Module._call_impl(self, *args, **kwargs)
   1522 # If we don't have any hooks, we want to skip the rest of the logic in
   1523 # this function, and just call forward.
   1524 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
   1525         or _global_backward_pre_hooks or _global_backward_hooks
   1526         or _global_forward_hooks or _global_forward_pre_hooks):
-> 1527     return forward_call(*args, **kwargs)
   1529 try:
   1530     result = None

File /opt/conda/lib/python3.10/site-packages/transformers/models/mistral/modeling_mistral.py:1001, in MistralModel.forward(self, input_ids, attention_mask, position_ids, past_key_values, inputs_embeds, use_cache, output_attentions, output_hidden_states, return_dict)
    997     attention_mask = attention_mask if (attention_mask is not None and 0 in attention_mask) else None
    998 elif self._attn_implementation == "sdpa" and not output_attentions:
    999     # output_attentions=True can not be supported when using SDPA, and we fall back on
   1000     # the manual implementation that requires a 4D causal mask in all cases.
-> 1001     attention_mask = _prepare_4d_causal_attention_mask_for_sdpa(
   1002         attention_mask,
   1003         (batch_size, seq_length),
   1004         inputs_embeds,
   1005         past_key_values_length,
   1006     )
   1007 else:
   1008     # 4d mask is passed through the layers
   1009     attention_mask = _prepare_4d_causal_attention_mask(
   1010         attention_mask,
   1011         (batch_size, seq_length),
   (...)
   1014         sliding_window=self.config.sliding_window,
   1015     )

File /opt/conda/lib/python3.10/site-packages/transformers/modeling_attn_mask_utils.py:371, in _prepare_4d_causal_attention_mask_for_sdpa(attention_mask, input_shape, inputs_embeds, past_key_values_length, sliding_window)
    366         attention_mask = inverted_mask.masked_fill(
    367             inverted_mask.to(torch.bool), torch.finfo(inputs_embeds.dtype).min
    368         )
    369         return attention_mask
--> 371 elif not is_tracing and torch.all(attention_mask == 1):
    372     if query_length == 1:
    373         # For query_length == 1, causal attention and bi-directional attention are the same.
    374         attention_mask = None

RuntimeError: CUDA error: device-side assert triggered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.