[./pre-commit.sh Error] Found 52 errors in 17 files (checked 365 source files)

[y12uc231]

Getting the following error after executing "./pre-commit.sh"

(decoding_trust_env) skrishna helm % mypy --install-types --non-interactive src scripts
Installing missing stub packages:
/Users/skrishna/anaconda3/envs/decoding_trust_env/bin/python -m pip install types-setuptools types-tabulate

Collecting types-setuptools
  Obtaining dependency information for types-setuptools from https://files.pythonhosted.org/packages/5c/5a/fbfbe3d1db90c59fb0240cf13a84953677b15874d00e80e773425447633c/types_setuptools-68.2.0.0-py3-none-any.whl.metadata
  Using cached types_setuptools-68.2.0.0-py3-none-any.whl.metadata (1.6 kB)
Collecting types-tabulate
  Obtaining dependency information for types-tabulate from https://files.pythonhosted.org/packages/d3/56/eb7d7585e046b75a1fcf137410ea5c0e0e32d1f88e716dcf79943501f06b/types_tabulate-0.9.0.3-py3-none-any.whl.metadata
  Using cached types_tabulate-0.9.0.3-py3-none-any.whl.metadata (1.4 kB)
Using cached types_setuptools-68.2.0.0-py3-none-any.whl (50 kB)
Using cached types_tabulate-0.9.0.3-py3-none-any.whl (3.2 kB)
Installing collected packages: types-tabulate, types-setuptools
Successfully installed types-setuptools-68.2.0.0 types-tabulate-0.9.0.3

scripts/data_overlap/test/test_compute_data_overlap_metrics.py:143: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
scripts/data_overlap/test/test_compute_data_overlap_metrics.py:160: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
scripts/data_overlap/test/test_compute_data_overlap_metrics.py:177: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
scripts/data_overlap/test/test_compute_data_overlap_metrics.py:178: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
scripts/data_overlap/test/test_compute_data_overlap_metrics.py:213: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
scripts/data_overlap/test/test_compute_data_overlap_metrics.py:218: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
scripts/data_overlap/test/test_compute_data_overlap_metrics.py:219: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/test_models.py:5: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
scripts/efficiency/generate_run_specs.py:11: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
scripts/data_overlap/run_data_overlap_beam.py:28: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
scripts/data_overlap/load_documents.py:137: error: Missing return statement  [empty-body]
scripts/data_overlap/data_overlap_beam.py:54: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/accounts.py:250: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/test_huggingface_tokenizer.py:46: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/scenarios/opinions_qa_scenario.py:111: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/scenarios/opinions_qa_scenario.py:120: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/scenarios/opinions_qa_scenario.py:122: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/scenarios/legal_summarization_scenario.py:139: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/scenarios/legal_summarization_scenario.py:141: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/scenarios/legal_summarization_scenario.py:143: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/scenarios/dialogue_scenarios.py:48: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/scenarios/dialogue_scenarios.py:58: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/scenarios/dialogue_scenarios.py:59: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/scenarios/dialogue_scenarios.py:63: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/scenarios/dialogue_scenarios.py:99: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/scenarios/decodingtrust_machine_ethics_scenario.py:474: error: Name "label_text" already defined on line 473  [no-redef]
src/helm/benchmark/scenarios/decodingtrust_fairness_scenario.py:37: error: Name "candidate_label" already defined on line 36  [no-redef]
src/helm/benchmark/scenarios/decodingtrust_fairness_scenario.py:39: error: Argument "text" to "Output" has incompatible type "int"; expected "str"  [arg-type]
src/helm/benchmark/scenarios/decodingtrust_adv_robustness_scenario.py:90: error: Name "candidate_label" already defined on line 89  [no-redef]
src/helm/proxy/test_retry.py:12: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/test_retry.py:34: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/services/service.py:111: error: Missing return statement  [empty-body]
src/helm/proxy/services/service.py:111: note: If the method is meant to be abstract, use @abc.abstractmethod
src/helm/common/cache.py:113: error: Missing return statement  [empty-body]
src/helm/common/cache.py:113: note: If the method is meant to be abstract, use @abc.abstractmethod
src/helm/common/cache.py:312: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/common/cache.py:315: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/augmentations/person_name_perturbation.py:265: error: Missing return statement  [empty-body]
src/helm/benchmark/augmentations/person_name_perturbation.py:265: note: If the method is meant to be abstract, use @abc.abstractmethod
src/helm/benchmark/augmentations/gender_perturbation.py:102: error: Incompatible default for argument "mapping_file_genders" (default has type "None", argument has type "list[str]")  [assignment]
src/helm/benchmark/augmentations/gender_perturbation.py:102: note: PEP 484 prohibits implicit Optional. Accordingly, mypy has changed its default to no_implicit_optional=True
src/helm/benchmark/augmentations/gender_perturbation.py:102: note: Use https://github.com/hauntsaninja/no_implicit_optional to automatically upgrade your codebase
src/helm/benchmark/augmentations/contrast_sets_perturbation.py:85: error: Missing return statement  [empty-body]
src/helm/benchmark/augmentations/contrast_sets_perturbation.py:85: note: If the method is meant to be abstract, use @abc.abstractmethod
src/helm/benchmark/augmentations/contraction_expansion_perturbation.py:111: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/augmentations/contraction_expansion_perturbation.py:112: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/augmentations/contraction_expansion_perturbation.py:151: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/token_counters/test_ai21_token_counter.py:19: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/services/remote_service.py:31: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/perspective_api_client.py:101: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/perspective_api_client.py:109: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/huggingface_client.py:40: error: Incompatible types in assignment (expression has type "Union[int, float]", variable has type "Union[int, LongTensor]")  [assignment]
src/helm/proxy/clients/huggingface_client.py:44: error: Incompatible return value type (got "Tensor", expected "bool")  [return-value]
src/helm/proxy/clients/huggingface_client.py:67: error: Incompatible types in assignment (expression has type "dtype", target has type "str")  [assignment]
src/helm/proxy/clients/huggingface_client.py:69: error: Incompatible types in assignment (expression has type "dtype", target has type "str")  [assignment]
src/helm/proxy/clients/huggingface_client.py:78: error: Item "None" of "Optional[HuggingfaceModelQuantizationConfig]" has no attribute "quant_file"  [union-attr]
src/helm/proxy/clients/huggingface_client.py:86: error: Item "None" of "Optional[HuggingfaceModelQuantizationConfig]" has no attribute "disable_exllama"  [union-attr]
src/helm/proxy/clients/huggingface_client.py:95: error: Item "None" of "Optional[HuggingfaceModelQuantizationConfig]" has no attribute "tokenizer_name"  [union-attr]
src/helm/proxy/clients/huggingface_client.py:96: error: Item "None" of "Optional[HuggingfaceModelQuantizationConfig]" has no attribute "tokenizer_name"  [union-attr]
src/helm/proxy/clients/huggingface_client.py:100: error: Name "tokenizer_name" already defined on line 96  [no-redef]
src/helm/proxy/clients/anthropic_client.py:356: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/anthropic_client.py:357: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/anthropic_client.py:358: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/anthropic_client.py:364: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/anthropic_client.py:370: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/anthropic_client.py:371: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/anthropic_client.py:372: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/anthropic_client.py:386: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/anthropic_client.py:391: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/anthropic_client.py:397: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/anthropic_client.py:404: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/anthropic_client.py:427: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/anthropic_client.py:432: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/aleph_alpha_client.py:136: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/aleph_alpha_client.py:137: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/aleph_alpha_client.py:166: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/ai21_client.py:67: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/ai21_client.py:132: error: Incompatible default for argument "finish_reason" (default has type "None", argument has type "dict[Any, Any]")  [assignment]
src/helm/proxy/clients/ai21_client.py:132: note: PEP 484 prohibits implicit Optional. Accordingly, mypy has changed its default to no_implicit_optional=True
src/helm/proxy/clients/ai21_client.py:132: note: Use https://github.com/hauntsaninja/no_implicit_optional to automatically upgrade your codebase
src/helm/benchmark/metrics/test_statistic.py:10: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/services/test_remote_service.py:95: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/services/test_remote_service.py:96: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/services/test_remote_service.py:97: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/services/test_remote_service.py:124: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/services/test_remote_service.py:129: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/services/test_remote_service.py:135: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/services/test_remote_service.py:163: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/test_together_client.py:14: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/test_ice_tokenizer_client.py:18: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/test_ice_tokenizer_client.py:22: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/test_ice_tokenizer_client.py:23: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/test_ice_tokenizer_client.py:30: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/test_ice_tokenizer_client.py:32: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/test_ice_tokenizer_client.py:38: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/test_ice_tokenizer_client.py:42: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/test_ice_tokenizer_client.py:44: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/test_ice_tokenizer_client.py:49: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/test_ice_tokenizer_client.py:51: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/test_huggingface_client.py:19: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/test_huggingface_client.py:27: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/test_huggingface_client.py:29: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/test_huggingface_client.py:35: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/test_huggingface_client.py:37: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/test_huggingface_client.py:48: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/test_huggingface_client.py:50: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/test_huggingface_client.py:55: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/test_huggingface_client.py:56: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/test_huggingface_client.py:73: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/test_anthropic_client.py:22: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/test_anthropic_client.py:30: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/test_anthropic_client.py:33: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/test_anthropic_client.py:39: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/test_anthropic_client.py:42: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/test_anthropic_client.py:53: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/test_anthropic_client.py:56: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/yalm_tokenizer/test_yalm_tokenizer.py:12: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/yalm_tokenizer/test_yalm_tokenizer.py:13: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/yalm_tokenizer/test_yalm_tokenizer.py:18: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/yalm_tokenizer/test_yalm_tokenizer.py:22: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/augmentations/test_perturbation.py:20: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/augmentations/test_perturbation.py:23: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/augmentations/test_perturbation.py:35: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/augmentations/test_perturbation.py:40: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/augmentations/test_perturbation.py:51: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/augmentations/test_perturbation.py:56: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/augmentations/test_perturbation.py:66: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/augmentations/test_perturbation.py:71: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/augmentations/test_perturbation.py:82: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/augmentations/test_perturbation.py:87: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/augmentations/test_perturbation.py:98: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/augmentations/test_perturbation.py:103: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/augmentations/test_perturbation.py:113: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/augmentations/test_perturbation.py:118: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/augmentations/test_perturbation.py:128: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/augmentations/test_perturbation.py:133: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/augmentations/test_perturbation.py:143: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/augmentations/test_perturbation.py:144: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/augmentations/test_perturbation.py:156: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/augmentations/test_perturbation.py:161: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/augmentations/test_perturbation.py:182: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/augmentations/test_perturbation.py:187: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/augmentations/test_perturbation.py:203: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/augmentations/test_perturbation.py:208: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/augmentations/test_perturbation.py:221: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/augmentations/test_perturbation.py:226: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/adaptation/scenario_state.py:27: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/adaptation/scenario_state.py:30: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/adaptation/scenario_state.py:35: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/token_counters/test_openai_token_counter.py:22: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/token_counters/test_openai_token_counter.py:30: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/test_yalm_tokenizer_client.py:18: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/test_yalm_tokenizer_client.py:21: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/test_yalm_tokenizer_client.py:22: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/test_yalm_tokenizer_client.py:29: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/test_yalm_tokenizer_client.py:31: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/test_yalm_tokenizer_client.py:37: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/test_yalm_tokenizer_client.py:41: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/test_yalm_tokenizer_client.py:45: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/test_yalm_tokenizer_client.py:50: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/test_yalm_tokenizer_client.py:52: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/microsoft_client.py:120: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
scripts/compute_request_limits.py:308: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
scripts/compute_request_limits.py:316: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
scripts/compute_request_limits.py:331: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/services/test_service.py:20: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/services/test_service.py:120: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/metrics/tokens/auto_token_cost_estimator.py:17: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/adaptation/adapters/test_adapter.py:15: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_yalm_window_service.py:11: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_yalm_window_service.py:12: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_yalm_window_service.py:142: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_yalm_window_service.py:146: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_ul2_window_service.py:72: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_ul2_window_service.py:73: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_ul2_window_service.py:159: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_ul2_window_service.py:163: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_t511b_window_service.py:72: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_t511b_window_service.py:73: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_t511b_window_service.py:159: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_t511b_window_service.py:163: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_t0pp_window_service.py:72: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_t0pp_window_service.py:73: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_t0pp_window_service.py:159: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_t0pp_window_service.py:163: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_palmyra_window_service.py:120: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_palmyra_window_service.py:154: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_palmyra_window_service.py:158: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_opt_window_service.py:11: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_opt_window_service.py:12: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_opt_window_service.py:145: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_opt_window_service.py:149: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_openai_window_service.py:11: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_openai_window_service.py:12: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_openai_window_service.py:39: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_openai_window_service.py:43: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_mt_nlg_window_service.py:11: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_mt_nlg_window_service.py:12: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_mt_nlg_window_service.py:39: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_mt_nlg_window_service.py:43: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_ice_window_service.py:66: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_ice_window_service.py:67: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_ice_window_service.py:151: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_ice_window_service.py:155: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_ice_window_service.py:160: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_ice_window_service.py:321: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_ice_window_service.py:322: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_ice_window_service.py:324: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_ice_window_service.py:325: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_gptneox_window_service.py:67: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_gptneox_window_service.py:68: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_gptneox_window_service.py:152: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_gptneox_window_service.py:156: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_gptj_window_service.py:11: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_gptj_window_service.py:12: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_gptj_window_service.py:40: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_gptj_window_service.py:44: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_gpt4_window_service.py:11: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_gpt4_window_service.py:12: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_gpt2_window_service.py:12: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_gpt2_window_service.py:13: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_gpt2_window_service.py:40: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_gpt2_window_service.py:44: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_gpt2_window_service.py:50: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_cohere_window_service.py:18: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_cohere_window_service.py:19: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_cohere_window_service.py:32: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_cohere_window_service.py:34: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_cohere_window_service.py:35: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_cohere_window_service.py:67: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_cohere_window_service.py:71: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_bloom_window_service.py:66: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_bloom_window_service.py:67: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_bloom_window_service.py:152: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_bloom_window_service.py:156: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_anthropic_window_service.py:122: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_anthropic_window_service.py:123: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_anthropic_window_service.py:157: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_anthropic_window_service.py:161: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_ai21_window_service.py:152: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_ai21_window_service.py:156: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/adaptation/adapters/chat_adapter.py:4: error: Module "helm.benchmark.adaptation.adapter_spec" has no attribute "SeparatorStyle"  [attr-defined]
src/helm/benchmark/adaptation/adapters/chat_adapter.py:29: error: Incompatible types in assignment (expression has type "None", variable has type "str")  [assignment]
src/helm/benchmark/adaptation/adapters/chat_adapter.py:31: error: Incompatible types in assignment (expression has type "None", variable has type "str")  [assignment]
src/helm/benchmark/adaptation/adapters/chat_adapter.py:33: error: Incompatible types in assignment (expression has type "None", variable has type "list[int]")  [assignment]
src/helm/benchmark/adaptation/adapters/chat_adapter.py:47: error: Unsupported left operand type for + ("None")  [operator]
src/helm/benchmark/adaptation/adapters/chat_adapter.py:47: note: Left operand is of type "Optional[str]"
src/helm/benchmark/adaptation/adapters/chat_adapter.py:55: error: Unsupported left operand type for + ("None")  [operator]
src/helm/benchmark/adaptation/adapters/chat_adapter.py:55: note: Left operand is of type "Optional[str]"
src/helm/benchmark/adaptation/adapters/chat_adapter.py:63: error: Unsupported left operand type for + ("None")  [operator]
src/helm/benchmark/adaptation/adapters/chat_adapter.py:63: note: Left operand is of type "Optional[str]"
src/helm/benchmark/adaptation/adapters/chat_adapter.py:71: error: Incompatible types in assignment (expression has type "Optional[str]", variable has type "str")  [assignment]
src/helm/benchmark/adaptation/adapters/chat_adapter.py:80: error: Incompatible types in assignment (expression has type "Optional[str]", variable has type "str")  [assignment]
src/helm/benchmark/adaptation/adapters/chat_adapter.py:88: error: Incompatible types in assignment (expression has type "Optional[str]", variable has type "str")  [assignment]
src/helm/benchmark/adaptation/adapters/chat_adapter.py:102: error: Unsupported left operand type for + ("None")  [operator]
src/helm/benchmark/adaptation/adapters/chat_adapter.py:102: note: Left operand is of type "Optional[str]"
src/helm/benchmark/adaptation/adapters/chat_adapter.py:109: error: Incompatible types in assignment (expression has type "Optional[str]", variable has type "str")  [assignment]
src/helm/benchmark/adaptation/adapters/chat_adapter.py:139: error: Unsupported left operand type for + ("None")  [operator]
src/helm/benchmark/adaptation/adapters/chat_adapter.py:139: note: Left operand is of type "Optional[str]"
src/helm/benchmark/adaptation/adapters/chat_adapter.py:149: error: Incompatible types in assignment (expression has type "Optional[str]", variable has type "str")  [assignment]
src/helm/benchmark/adaptation/adapters/chat_adapter.py:160: error: Incompatible types in assignment (expression has type "Optional[str]", variable has type "str")  [assignment]
src/helm/benchmark/adaptation/adapters/chat_adapter.py:170: error: Incompatible types in assignment (expression has type "Optional[str]", variable has type "str")  [assignment]
src/helm/benchmark/adaptation/adapters/chat_adapter.py:178: error: Unsupported left operand type for + ("None")  [operator]
src/helm/benchmark/adaptation/adapters/chat_adapter.py:178: note: Left operand is of type "Optional[str]"
src/helm/benchmark/adaptation/adapters/chat_adapter.py:266: error: "AdapterSpec" has no attribute "sep_style"  [attr-defined]
src/helm/benchmark/adaptation/adapters/chat_adapter.py:267: error: "Input" has no attribute "replace"  [attr-defined]
src/helm/benchmark/adaptation/adapters/chat_adapter.py:270: error: Unsupported operand types for + ("str" and "Input")  [operator]
src/helm/benchmark/adaptation/adapters/chat_adapter.py:275: error: Missing return statement  [empty-body]
src/helm/benchmark/adaptation/adapters/chat_adapter.py:275: note: If the method is meant to be abstract, use @abc.abstractmethod
src/helm/benchmark/window_services/test_flan_t5_window_service.py:10: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/window_services/test_flan_t5_window_service.py:11: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/adaptation/adapters/test_language_modeling_adapter.py:24: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/adaptation/adapters/test_language_modeling_adapter.py:25: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/adaptation/adapters/test_language_modeling_adapter.py:49: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/adaptation/adapters/test_language_modeling_adapter.py:50: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/adaptation/adapters/test_language_modeling_adapter.py:78: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/adaptation/adapters/test_language_modeling_adapter.py:79: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/adaptation/adapters/test_language_modeling_adapter.py:80: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/adaptation/adapters/test_language_modeling_adapter.py:85: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/adaptation/adapters/test_language_modeling_adapter.py:86: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/adaptation/adapters/test_language_modeling_adapter.py:91: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/adaptation/adapters/test_language_modeling_adapter.py:92: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/adaptation/adapters/test_language_modeling_adapter.py:96: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/adaptation/adapters/test_language_modeling_adapter.py:97: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/adaptation/adapters/test_language_modeling_adapter.py:113: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/adaptation/adapters/test_language_modeling_adapter.py:114: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/adaptation/adapters/test_language_modeling_adapter.py:121: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/adaptation/adapters/test_language_modeling_adapter.py:122: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/metrics/decodingtrust_stereotype_bias_metrics.py:95: error: Incompatible types in assignment (expression has type "Optional[RequestResult]", variable has type "RequestResult")  [assignment]
src/helm/benchmark/metrics/decodingtrust_stereotype_bias_metrics.py:96: error: Incompatible types in assignment (expression has type "list[Sequence]", variable has type "list[str]")  [assignment]
src/helm/benchmark/metrics/decodingtrust_privacy_metrics.py:24: error: Item "None" of "Optional[RequestResult]" has no attribute "completions"  [union-attr]
src/helm/benchmark/metrics/decodingtrust_privacy_metrics.py:32: error: Item "None" of "Optional[RequestResult]" has no attribute "completions"  [union-attr]
src/helm/benchmark/metrics/decodingtrust_privacy_metrics.py:66: error: Incompatible types in assignment (expression has type "float", variable has type "int")  [assignment]
src/helm/benchmark/metrics/decodingtrust_privacy_metrics.py:67: error: Incompatible types in assignment (expression has type "float", variable has type "int")  [assignment]
src/helm/benchmark/metrics/decodingtrust_privacy_metrics.py:68: error: Incompatible types in assignment (expression has type "float", variable has type "int")  [assignment]
src/helm/benchmark/metrics/decodingtrust_ood_knowledge_metrics.py:57: error: Value of type "Optional[dict[str, str]]" is not indexable  [index]
src/helm/benchmark/run_expander.py:90: error: Unexpected keyword argument "metrics" for "replace" of "RunSpec"  [call-arg]
src/helm/benchmark/run_expander.py:526: error: Incompatible default for argument "mapping_file_genders" (default has type "None", argument has type "tuple[str]")  [assignment]
src/helm/benchmark/run_expander.py:526: note: PEP 484 prohibits implicit Optional. Accordingly, mypy has changed its default to no_implicit_optional=True
src/helm/benchmark/run_expander.py:526: note: Use https://github.com/hauntsaninja/no_implicit_optional to automatically upgrade your codebase
src/helm/proxy/clients/test_huggingface_model_registry.py:24: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/proxy/clients/test_huggingface_model_registry.py:42: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/presentation/summarize.py:347: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/presentation/summarize.py:348: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/presentation/summarize.py:352: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/presentation/summarize.py:374: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/presentation/summarize.py:375: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/presentation/summarize.py:376: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/presentation/summarize.py:377: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/presentation/summarize.py:445: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/presentation/summarize.py:459: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/presentation/summarize.py:464: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/presentation/summarize.py:517: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/presentation/summarize.py:552: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/presentation/summarize.py:554: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/presentation/summarize.py:1130: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/presentation/summarize.py:1133: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/presentation/summarize.py:1278: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/presentation/summarize.py:1279: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/presentation/summarize.py:1280: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/test_data_preprocessor.py:13: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/test_data_preprocessor.py:15: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/test_data_preprocessor.py:34: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/test_data_preprocessor.py:36: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/run.py:55: error: Argument "num_train_trials" to "replace" of "AdapterSpec" has incompatible type "Optional[int]"; expected "int"  [arg-type]
src/helm/benchmark/run.py:281: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/run.py:303: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/scenarios/test_scenario.py:7: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/presentation/create_plots.py:172: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/presentation/create_plots.py:177: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/presentation/create_plots.py:219: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/presentation/create_plots.py:225: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/presentation/create_plots.py:355: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/presentation/create_plots.py:402: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/presentation/create_plots.py:431: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/presentation/create_plots.py:449: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/presentation/create_plots.py:466: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/presentation/create_plots.py:500: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/presentation/create_plots.py:509: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/presentation/create_plots.py:521: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/presentation/create_plots.py:522: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/presentation/create_plots.py:555: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/presentation/create_plots.py:556: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/adaptation/adapters/test_generation_adapter.py:35: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/adaptation/adapters/test_generation_adapter.py:39: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/adaptation/adapters/test_generation_adapter.py:42: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/adaptation/adapters/test_generation_adapter.py:56: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/adaptation/adapters/test_generation_adapter.py:60: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/adaptation/adapters/test_generation_adapter.py:63: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/adaptation/adapters/test_generation_adapter.py:88: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/adaptation/adapters/test_generation_adapter.py:92: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/adaptation/adapters/test_generation_adapter.py:93: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/adaptation/adapters/test_generation_adapter.py:102: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
src/helm/benchmark/adaptation/adapters/test_generation_adapter.py:119: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
Found 52 errors in 17 files (checked 365 source files)

[boxin-wbx]

Hi y12uc231,

Thank you so much for your interests in our work!

We are sorry for the late response, as we are currently undergoing some code refactoring to make our DT library easier to use.

Currently we have pushed our latest codebase to the main branch. Could you please follow the latest README and tutorial, try installing again from scratch with a clean conda environment, and see if you still have any issues? Hope our latest update could solve the compatibility issues.

Please let us know if you still have further questions. Thank you once again for your interests!

Best, Boxin