Type Checker keeps giving warnings about almost everything guidance

[ibehnam]

I use guidance according to the examples. But there are several areas where the type-checker yells at me for a variety of reasons:

• Example 1:

• Example 2:

• Example 3:

• Example 4:

I could go on, but I think these examples point to a more fundamental issue with guidance and how it's programmed. I raised these issues in the previous version (https://github.com/guidance-ai/guidance/issues/111). It's disappointing that this issue still exists in the new version.

If there's any workaround, please share here.

[ibehnam]

• Workaround for Example-1:

Use:

from guidance.models.llama_cpp._llama_cpp import LlamaCpp

and then use

LlamaCpp(...)

instead of models.LlamaCpp(...).

[Harsha-Nori]

@ryanpeach made a change in the latest version of guidance that allows this alternative import for when Type Checkers get upset.

from guidance import guidance

Does changing this import work for you?

Note that this isn't a bug, just something weird with how the type checker processes our module...if anyone has tips on code we can add in the library so that we disable this type check on our module decorator, please let us know!

[ibehnam]

• Workaround for Example-4:

Use:

from guidance._guidance import guidance

instead of

from guidance import guidance

[Harsha-Nori]

@ibehnam , if I'm understanding right, both import guidance and from guidance import guidance cause issues with your type checker? Mypy runs on the codebase and I don't have any of these issues with the Python Language Server checks in VS Code, so I'm not sure what's causing it :|.

Can I ask what type checker you have configured?

[ibehnam]

@Harsha-Nori Yes, they cause issues with the type checker. Haven't tried mypy yet. The errors are given by Pylance (Pyright type checker):

"guidance" is not exported from module "guidance"
  Import from "guidance._guidance" insteadPylance[reportPrivateImportUsage](https://github.com/microsoft/pyright/blob/main/docs/configuration.md#reportPrivateImportUsage)

[Harsha-Nori]

@ibehnam thanks! Are you up for helping us fix this issue? I unfortunately can't reproduce on my side even though I'm running the Python Language Server too :(.

I think adding this line: # pyright: ignore to some of the __init__.py lines might solve the type checker problem. From: https://microsoft.github.io/pyright/#/comments?id=line-level-diagnostic-suppression

If you're up for helping us out, you could clone the codebase, do a local install (pip install -e .), and try the diagnostic suppression comment on various lines in the in__init__.py. Would be great to know if this resolves the issue, and we'd be thrilled to have a PR on this.

[ibehnam]

@Harsha-Nori Okay, I'll try to do that and report the results here. Is cloning the repo necessary, or can I just modify the installed guidance files?

[Harsha-Nori]

If you installed guidance via a pre-built .whl on PyPi (e.g. pip install guidance), I'd recommend cloning the repo and running in a fresh environment. There's sometimes ambiguity in where the running code actually sits when you do a pypi/conda install. You get full control over that with a local editable install (and also the benefit of automatically re-installing the package every time you change the code).

[ibehnam]

Thanks! Okay, I will clone the repo and test with Pylance and report here. I just checked from guidance import guidance with mypy and it found no issues with it.

[Harsha-Nori]

Good to hear that mypy works at least!