When trying to issue a length-exceeded warning while --no_cache is given, the MegatronLMClient errors out with the below stack trace.
The reason being that cache_key is None, which is not checked. The easiest solution would probably be to just omit printing the text strings if cache_key is None (i.e., if caching is disabled).
Traceback (most recent call last):
File "/p/project/opengptx-elm/ebert1/opengpt/lm-evaluation-harness/main.py", line 101, in <module>
main()
File "/p/project/opengptx-elm/ebert1/opengpt/lm-evaluation-harness/main.py", line 67, in main
results = evaluator.simple_evaluate(
File "/p/project/opengptx-elm/ebert1/opengpt/lm-evaluation-harness/lm_eval/utils.py", line 242, in _wrapper
return fn(*args, **kwargs)
File "/p/project/opengptx-elm/ebert1/opengpt/lm-evaluation-harness/lm_eval/evaluator.py", line 103, in simple_evaluate
results = evaluate(
File "/p/project/opengptx-elm/ebert1/opengpt/lm-evaluation-harness/lm_eval/utils.py", line 242, in _wrapper
return fn(*args, **kwargs)
File "/p/project/opengptx-elm/ebert1/opengpt/lm-evaluation-harness/lm_eval/evaluator.py", line 297, in evaluate
resps = getattr(lm, reqtype)([req.args for req in reqs])
File "/p/project/opengptx-elm/ebert1/opengpt/lm-evaluation-harness/lm_eval/base.py", line 210, in loglikelihood_rolling
string_nll = self._loglikelihood_tokens(
File "/p/project/opengptx-elm/ebert1/opengpt/lm-evaluation-harness/lm_eval/models/megatronlm.py", line 187, in _loglikelihood_tokens
f"WARNING: Length of concatenated context ...{repr(cache_key[0][-20:])} and continuation {repr(cache_key[1])} exceeds max length {self.max_length + 1}"
TypeError: 'NoneType' object is not subscriptable
When trying to issue a length-exceeded warning while
--no_cache
is given, theMegatronLMClient
errors out with the below stack trace. The reason being thatcache_key is None
, which is not checked. The easiest solution would probably be to just omit printing the text strings ifcache_key is None
(i.e., if caching is disabled).