search

UKGovernmentBEIS / inspect_ai

Inspect: A framework for large language model evaluations
https://inspect.ai-safety-institute.org.uk/
MIT License
567 stars 98 forks source link

Missing __init__.py in subdirectories #18

Closed irenedea closed 4 months ago

irenedea commented 4 months ago

Hello!

Some subdirectories such as src/inspect_ai/_eval should have __init__.py in them to help mark the directory as a python module. I noticed this because pyright had some issues with resolving imports from this project. Thank you!

r-leyshon commented 4 months ago

As of python 3.7 this is no longer required: https://peps.python.org/pep-0420/ , though can be used to control namespace exports.

aisi-inspect commented 4 months ago

Our convention is actually to use _ for "private" modules, so we purposely don't have __init__.py in those modules b/c we don't expect you to import from them. In this case, anything public in inspect_ai/_eval is actually importable from inspect_ai (e.g. Task, eval, etc.). We further butress this convention by having all public APIs explicitly delineated in __all__ objects. For example:

Many (most?) IDEs will only offer auto-imports from __all__ if is is provided. I know that this is how Pylance works, not sure about Pyright. General guideline would be that if there is an _ anywhere in the import path it's a private import, so you should fine the right public place to import from (all imports are at most 1 level deep so either in inspect_ai, inspect_ai.model, inspect_ai.solver, etc.

Going to close in hopes this clarification is all that is required. If there are other issues you think need addressing feel free to re-open and/or rename to something more targeted.