wandb.proto.wandb_internal_pb2 error when running dspy MIPRO

[NasonZ]

I'm trying to run this notebook https://github.com/weaviate/recipes/blob/main/integrations/llm-frameworks/dspy/llms/Llama3.ipynb

It works fine but when I get to trying to run MIPRO

from dspy.teleprompt import MIPRO

import openai
gpt4 = dspy.OpenAI(model="gpt-4o", max_tokens=4000, model_type="chat")

teleprompter = MIPRO(prompt_model=gpt4, 
                     task_model=llama3_ollama, 
                     metric=MetricWrapper, 
                     num_candidates=3, 
                     init_temperature=0.5)
kwargs = dict(num_threads=1, 
              display_progress=True, 
              display_table=0)
MIPRO_compiled_RAG = teleprompter.compile(RAG(), trainset=trainset[:5], num_trials=3, max_bootstrapped_demos=1, max_labeled_demos=0, eval_kwargs=kwargs)

I get this error:

AttributeError: module 'wandb.proto.wandb_internal_pb2' has no attribute 'Result'

I know its a dspy issue but I was wondering if: a) The issue is with the notebook and not on my end. (Does anyone else encounter the same error) b) If anyone had any ideas how to get MIPRO working?

[NasonZ]

Full trace for reference:

---

AttributeError Traceback (most recent call last) Cell In[8], line 1 ----> 1 from dspy.teleprompt import MIPRO 3 import openai 4 gpt4 = dspy.OpenAI(model="gpt-4o", max_tokens=4000, model_type="chat")

File \lib\site-packages\dspy\teleprompt__init__.py:7 5 from .knn_fewshot import 6 from .mipro_optimizer import MIPRO ----> 7 from .mipro_optimizer_v2 import MIPROv2 8 from .random_search import 9 from .signature_opt import SignatureOptimizer

File \lib\site-packages\dspy\teleprompt\mipro_optimizer_v2.py:30 15 from dspy.teleprompt.utils import ( 16 create_n_fewshot_demo_sets, 17 eval_candidate_program, (...) 26 setup_logging, 27 ) 29 try: ---> 30 import wandb 31 except ImportError: 32 wandb = None

File \lib\site-packages\wandb__init__.py:27 24 # This needs to be early as other modules call it. 25 from wandb.errors.term import termsetup, termlog, termerror, termwarn ---> 27 from wandb import sdk as wandb_sdk 29 import wandb 31 wandb.wandb_lib = wandb_sdk.lib # type: ignore

File \lib\site-packages\wandb\sdk__init.py:25 3 all__ = ( 4 "Config", 5 "Settings", (...) 21 "helper", 22 ) 24 from . import wandb_helper as helper ---> 25 from .artifacts.artifact import Artifact 26 from .wandb_alerts import AlertLevel 27 from .wandb_config import Config

File \lib\site-packages\wandb\sdk\artifacts\artifact.py:47 45 from wandb import data_types, env, util 46 from wandb.apis.normalize import normalize_exceptions ---> 47 from wandb.apis.public import ArtifactCollection, ArtifactFiles, RetryingClient, Run 48 from wandb.data_types import WBValue 49 from wandb.errors.term import termerror, termlog, termwarn

File \lib\site-packages\wandb\apis__init__.py:43 38 _disable_ssl() 41 reset_path = util.vendor_setup() ---> 43 from .internal import Api as InternalApi # noqa 44 from .public import Api as PublicApi # noqa 46 reset_path()

File \lib\site-packages\wandb\apis\internal.py:3 1 from typing import Any ----> 3 from wandb.sdk.internal.internal_api import Api as InternalApi 6 class Api: 7 """Internal proxy to the official internal API."""

File \lib\site-packages\wandb\sdk\internal\internal_api.py:48 45 from wandb.sdk.lib.gql_request import GraphQLSession 46 from wandb.sdk.lib.hashutil import B64MD5, md5_file_b64 ---> 48 from ..lib import retry 49 from ..lib.filenames import DIFF_FNAME, METADATA_FNAME 50 from ..lib.gitlib import GitRepo

File \lib\site-packages\wandb\sdk\lib\retry.py:17 14 import wandb 15 from wandb.util import CheckRetryFnType ---> 17 from .mailbox import ContextCancelledError 19 logger = logging.getLogger(name) 22 # To let tests mock out the retry logic's now()/sleep() funcs, this file 23 # should only use these variables, not call the stdlib funcs directly.

File \lib\site-packages\wandb\sdk\lib\mailbox.py:102 98 self._event.clear() 99 return found --> 102 class _MailboxSlot: 103 _result: Optional[pb.Result] 104 _event: threading.Event

File \lib\site-packages\wandb\sdk\lib\mailbox.py:103, in _MailboxSlot() 102 class _MailboxSlot: --> 103 _result: Optional[pb.Result] 104 _event: threading.Event 105 _lock: threading.Lock

AttributeError: module 'wandb.proto.wandb_internal_pb2' has no attribute 'Result'

[erika-cardenas]

Thanks for reporting it @NasonZ! @CShorten was this version merged into DSPy main?

[erika-cardenas]

Side note: There is a Weave integration in DSPy now if that's what you're looking for!

https://wandb.github.io/weave/guides/integrations/dspy/

[CShorten]

Hey both, I just took a quick look and indeed W&B has been added to DSPy, but not from the branch I was working on -- so I'm not sure the exact internal details.

So that might be relevant to this issue because it seems to be parsing a particular key from W&B, Result -- however, it is still not clear to me if this is a dependency conflict type of thing or something else. I will take another look hopefully this afternoon.

Thanks for opening the issue @NasonZ -- I hate to offer such a naive solution, but given the nature of the internal APIs it's looking for, maybe simply upgrading the w&b library could fix this?

[NasonZ]

Side note: There is a Weave integration in DSPy now if that's what you're looking for!

https://wandb.github.io/weave/guides/integrations/dspy/

Was mainly looking to test prompt optmisation for a RAG application but this intergration defintely makes weaviate a more attricative candidate to be the db.

[CShorten]

cc @rogeriochaves

[erika-cardenas]

If you're all set @NasonZ, I will close the issue!

I'll add a note to the top of the W&B notebook stating the DSPy version and Weave integration.

[NasonZ]

@CShorten Thanks, upgrading from wandb-0.16.5 to wandb-0.16.6 did the trick.