search

HazyResearch / safari

Convolutions for Sequence Modeling
Apache License 2.0
864 stars 71 forks source link

Hyena YAML parsing fails #20

Closed janEbert closed 1 year ago

janEbert commented 1 year ago

Hey, I was trying to run the Hyena 150B token training according to experiments.md, but the YAML parsing fails and complains about

yaml.scanner.ScannerError: mapping values are not allowed here
  in "[...]/safari/configs/experiment/pile/hyena.yaml", line 9, column 18

This is probably due to the ${eval: [...]} in the mentioned line and the parser assumes eval: [...] should be parsed as a YAML expression. I haven't found this eval: feature in the Hydra or lightning-hydra-template documentation.

I'm probably missing something here due to unfamiliarity with lightning-hydra-template. :) However, if I'm not, has there possibly been dependency breakage or are you maybe using a custom Hydra fork?

Environment

PyTorch 1.12, Lightning 1.8.6, Hydra 1.3.2

I simply installed requirements.txt as specified in README.md.

Command

As specified in experiments.md for the small Hydra training, with two adjustments:

  1. The YAML config name was adjusted to hyena-150b-tokens since hyena-150b does not exist.
  2. experiment/... was changed to experiment=... so the argument is correctly parsed. 
    python -m train experiment=pile/hyena-150b-tokens
janEbert commented 1 year ago

Eric Nguyen found the problem: the YAML syntax had to be ${eval:[...]} without a space. See #22.