search

Giskard-AI / giskard

🐢 Open-Source Evaluation & Testing for ML & LLM systems
https://docs.giskard.ai
Apache License 2.0
4.1k stars 270 forks source link

AttributeError: module 'giskard.llm' has no attribute 'embeddings' #2047

Closed MachoPanko closed 1 month ago

MachoPanko commented 1 month ago

Issue Type

Bug

Source

source

Giskard Library Version

2.15.2

OS Platform and Distribution

Windows

Python version

3.12

Installed python packages

aiohappyeyeballs                         2.4.2
aiohttp                                  3.10.7
aiosignal                                1.3.1
altair                                   5.4.1
annotated-types                          0.7.0
anyio                                    4.6.0
appdirs                                  1.4.4
asgiref                                  3.8.1
asttokens                                2.4.1
attrs                                    24.2.0
backoff                                  2.2.1
bcrypt                                   4.2.0
bert-score                               0.3.13
blinker                                  1.8.2
bokeh                                    3.4.3
build                                    1.2.2.post1
cachetools                               5.5.0
certifi                                  2024.7.4
chardet                                  5.2.0
charset-normalizer                       3.3.2
chroma-hnswlib                           0.7.6
chromadb                                 0.5.15
click                                    8.1.7
cloudpickle                              3.1.0
colorama                                 0.4.6
coloredlogs                              15.0.1
comm                                     0.2.2
contourpy                                1.3.0
cycler                                   0.12.1
databricks-sdk                           0.34.0
dataclasses-json                         0.6.7
datasets                                 3.0.1
debugpy                                  1.8.6
decorator                                5.1.1
deepeval                                 1.4.1
Deprecated                               1.2.14
dill                                     0.3.8
distlib                                  0.3.8
distro                                   1.9.0
docopt                                   0.6.2
docx2txt                                 0.8
durationpy                               0.9
evaluate                                 0.4.3
execnet                                  2.1.1
executing                                2.1.0
faiss-cpu                                1.8.0
fastapi                                  0.115.2
filelock                                 3.15.4
flatbuffers                              24.3.25
fonttools                                4.54.1
frozenlist                               1.4.1
fsspec                                   2024.6.1
giskard                                  2.15.2
gitdb                                    4.0.11
GitPython                                3.1.43
google-auth                              2.35.0
googleapis-common-protos                 1.65.0
greenlet                                 3.1.1
griffe                                   0.48.0
grpcio                                   1.63.2
h11                                      0.14.0
httpcore                                 1.0.5
httptools                                0.6.4
httpx                                    0.27.2
huggingface-hub                          0.25.1
humanfriendly                            10.0
idna                                     3.10
importlib-metadata                       7.0.0
importlib_resources                      6.4.5
iniconfig                                2.0.0
ipykernel                                6.29.5
ipython                                  8.27.0
jedi                                     0.19.1
Jinja2                                   3.1.4
jiter                                    0.5.0
joblib                                   1.4.2
jsonpatch                                1.33
jsonpointer                              3.0.0
jsonschema                               4.23.0
jsonschema-specifications                2024.10.1
jupyter_client                           8.6.3
jupyter_core                             5.7.2
kiwisolver                               1.4.7
kubernetes                               31.0.0
langchain                                0.3.1
langchain-community                      0.3.1
langchain-core                           0.3.6
langchain-openai                         0.2.1
langchain-text-splitters                 0.3.0
langdetect                               1.0.9
langsmith                                0.1.129
llvmlite                                 0.43.0
Markdown                                 3.7
markdown-it-py                           3.0.0
MarkupSafe                               3.0.1
marshmallow                              3.22.0
matplotlib                               3.9.2
matplotlib-inline                        0.1.7
mdurl                                    0.1.2
mixpanel                                 4.10.1
mlflow-skinny                            2.17.0
mmh3                                     5.0.1
monotonic                                1.6
mpmath                                   1.3.0
multidict                                6.1.0
multiprocess                             0.70.16
mypy-extensions                          1.0.0
narwhals                                 1.9.3
nest-asyncio                             1.6.0
networkx                                 3.4.1
num2words                                0.5.13
numba                                    0.60.0
numpy                                    1.26.4
oauthlib                                 3.2.2
onnxruntime                              1.19.2
openai                                   1.52.0
opentelemetry-api                        1.24.0
opentelemetry-exporter-otlp-proto-common 1.24.0
opentelemetry-exporter-otlp-proto-grpc   1.24.0
opentelemetry-instrumentation            0.45b0
opentelemetry-instrumentation-asgi       0.45b0
opentelemetry-instrumentation-fastapi    0.45b0
opentelemetry-proto                      1.24.0
opentelemetry-sdk                        1.24.0
opentelemetry-semantic-conventions       0.45b0
opentelemetry-util-http                  0.45b0
orjson                                   3.10.7
overrides                                7.7.0
packaging                                24.1
pandas                                   2.2.3
parso                                    0.8.4
pillow                                   10.4.0
pip                                      24.2
pipenv                                   2024.0.1
platformdirs                             4.3.6
pluggy                                   1.5.0
portalocker                              2.10.1
posthog                                  3.7.0
prompt_toolkit                           3.0.48
protobuf                                 4.25.5
psutil                                   6.0.0
pure_eval                                0.2.3
pyarrow                                  17.0.0
pyasn1                                   0.6.1
pyasn1_modules                           0.4.1
pydantic                                 2.9.2
pydantic_core                            2.23.4
pydantic-settings                        2.5.2
pydeck                                   0.9.1
Pygments                                 2.18.0
pynndescent                              0.5.13
pyparsing                                3.2.0
pypdf                                    3.17.0
PyPika                                   0.48.9
pyproject_hooks                          1.2.0
pyreadline3                              3.5.4
pysbd                                    0.3.4
pytest                                   8.3.3
pytest-repeat                            0.9.3
pytest-xdist                             3.6.1
python-dateutil                          2.9.0.post0
python-dotenv                            1.0.1
pytz                                     2024.2
pywin32                                  306
PyYAML                                   6.0.2
pyzmq                                    26.2.0
ragas                                    0.2.1
referencing                              0.35.1
regex                                    2024.9.11
requests                                 2.32.3
requests-oauthlib                        2.0.0
requests-toolbelt                        1.0.0
rich                                     13.9.2
rpds-py                                  0.20.0
rsa                                      4.9
safetensors                              0.4.5
scikit-learn                             1.5.2
scipy                                    1.11.4
sentry-sdk                               2.17.0
setuptools                               75.1.0
shellingham                              1.5.4
six                                      1.16.0
smmap                                    5.0.1
sniffio                                  1.3.1
SQLAlchemy                               2.0.35
sqlparse                                 0.5.1
stack-data                               0.6.3
starlette                                0.40.0
streamlit                                1.39.0
sympy                                    1.13.3
tabulate                                 0.9.0
tenacity                                 8.4.2
threadpoolctl                            3.5.0
tiktoken                                 0.7.0
tokenizers                               0.20.1
toml                                     0.10.2
torch                                    2.4.1
tornado                                  6.4.1
tqdm                                     4.66.5
traitlets                                5.14.3
transformers                             4.45.2
typer                                    0.12.5
typing_extensions                        4.12.2
typing-inspect                           0.9.0
tzdata                                   2024.2
umap-learn                               0.5.6
urllib3                                  2.2.3
uvicorn                                  0.32.0
virtualenv                               20.26.3
watchdog                                 5.0.3
watchfiles                               0.24.0
wcwidth                                  0.2.13
websocket-client                         1.8.0
websockets                               13.1
wrapt                                    1.16.0
xxhash                                   3.5.0
xyzservices                              2024.9.0
yarl                                     1.13.1
zipp                                     3.20.2
zstandard                                0.23.0

Current Behaviour?

i followed the following link to set up a RAGET : https://docs.giskard.ai/en/latest/open_source/testset_generation/testset_generation/index.html

Is there something wrong with the package? cant seem to get it to work while others on the forum seem to have no problem. I have tried pip install --upgrade giskard[llm] but still no luck

Standalone code OR list down the steps to reproduce the issue

import os
import giskard

# Set the Azure OpenAI API key and endpoint
os.environ['AZURE_OPENAI_API_KEY'] = secret
os.environ['AZURE_OPENAI_ENDPOINT'] = secret
os.environ['AZURE_OPENAI_API_VERSION'] ="2024-02-15-preview"

# You'll need to provide the name of the model that you've deployed
# Beware, the model provided must be capable of using function calls
giskard.llm.set_llm_model('gpt-35-turbo')
giskard.llm.embeddings.set_embedding_model('text-embedding-ada-002')

Relevant log output

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[3], line 12
      9 # You'll need to provide the name of the model that you've deployed
     10 # Beware, the model provided must be capable of using function calls
     11 giskard.llm.set_llm_model('gpt-35-turbo')
---> 12 giskard.llm.embeddings.set_embedding_model('text-embedding-ada-002')

AttributeError: module 'giskard.llm' has no attribute 'embeddings'
MachoPanko commented 1 month ago

At the bottom right, change latest to stable . and u should get correct docs.