facebookresearch / SentEval

A python tool for evaluating the quality of sentence embeddings.
Other
2.09k stars 309 forks source link

script fails when checking the optimizer settings - fix by deleting/commeting line 89-93 #89

Open ulf1 opened 2 years ago

ulf1 commented 2 years ago

Error message

Traceback (most recent call last):
  File "/home/hamster/prj/study-hrp/script.py", line 192, in <module>
    senteval_results = se.eval(senteval_tasks)
  File "/home/hamster/prj/study-hrp/src/senteval/senteval/engine.py", line 59, in eval
    self.results = {x: self.eval(x) for x in name}
  File "/home/hamster/prj/study-hrp/src/senteval/senteval/engine.py", line 59, in <dictcomp>
    self.results = {x: self.eval(x) for x in name}
  File "/home/hamster/prj/study-hrp/src/senteval/senteval/engine.py", line 121, in eval
    self.results = self.evaluation.run(self.params, self.batcher)
  File "/home/hamster/prj/study-hrp/src/senteval/senteval/binary.py", line 57, in run
    devacc, testacc = clf.run()
  File "/home/hamster/prj/study-hrp/src/senteval/senteval/tools/validation.py", line 78, in run
    clf = MLP(self.classifier_config, inputdim=self.featdim,
  File "/home/hamster/prj/study-hrp/src/senteval/senteval/tools/classifier.py", line 200, in __init__
    optim_fn, optim_params = utils.get_optimizer(self.optim)
  File "/home/hamster/prj/study-hrp/src/senteval/senteval/utils.py", line 89, in get_optimizer
    expected_args = inspect.getargspec(optim_fn.__init__)[0]
  File "/home/hamster/miniconda3/envs/gpu-venv-study-hrp/lib/python3.9/inspect.py", line 1122, in getargspec
    raise ValueError("Function has keyword-only parameters or annotations"
ValueError: Function has keyword-only parameters or annotations, use inspect.signature() API which can support them

Quick fix

BrunoKreiner commented 1 year ago

Had the same error running:

params = {'task_path': "../data/senteval/", 'usepytorch': True, 'kfold': 10}
params['classifier'] = {'nhid': 0, 'optim': 'adam', 'batch_size': 32,
                                 'tenacity': 5, 'epoch_size': 10}

se = senteval.engine.SE(params, batcher, prepare)
se.eval(transfer_task)

on ALL transfer tasks EXCEPT these: 'STS12', 'STS13', 'STS14', 'STS15', 'STS16','STSBenchmark', 'SICKRelatedness'

I don't understand the check in utils.py