Closed norabelrose closed 1 year ago
I did mkdir gpt2
from the elk repo root, and then got the following result:
❯ elk elicit gpt2 amazon_polarity --max_examples 10 --debug
Traceback (most recent call last):
File "/mnt/ssd-2/spar/alexm/miniconda3/envs/elk/bin/elk", line 8, in <module>
sys.exit(run())
File "/mnt/ssd-2/spar/alexm/elk/elk/__main__.py", line 27, in run
run.execute()
File "/mnt/ssd-2/spar/alexm/elk/elk/__main__.py", line 19, in execute
return self.command.execute()
File "/mnt/ssd-2/spar/alexm/elk/elk/run.py", line 50, in execute
self.datasets = [
File "/mnt/ssd-2/spar/alexm/elk/elk/run.py", line 51, in <listcomp>
extract(
File "/mnt/ssd-2/spar/alexm/elk/elk/extraction/extraction.py", line 310, in extract
with temporary_dir_move(cfg.model):
File "/mnt/ssd-2/spar/alexm/miniconda3/envs/elk/lib/python3.10/contextlib.py", line 135, in __enter__
return next(self.gen)
File "/mnt/ssd-2/spar/alexm/elk/elk/utils/data_utils.py", line 64, in temporary_dir_move
path.rename(dest)
File "/mnt/ssd-2/spar/alexm/miniconda3/envs/elk/lib/python3.10/pathlib.py", line 1234, in rename
self._accessor.rename(self, target)
OSError: [Errno 18] Invalid cross-device link: 'gpt2' -> '/tmp/tmpwn7ysohc/gpt2'```
This fixes a very annoying quirk of HuggingFace
from_pretrained
methods: if you try to load, say,huggyllama/llama-7b
when there's a folder namedhuggyllama/llama-7b
in the current working directory, it'll try to load from that folder instead of the HF Hub (or the relevant cache dir). We basically never want this behavior, and it becomes a problem when you happen to becd
ed into theelk-reporters
directory for example.This PR fixes this with a little hack: we just move the offending local directory to a temp dir for a short time while we call
Auto<whatever>.from_pretrained
, then move it back. AFAICT there's not a better way to do this.Also, as a bonus, this PR makes the change that Christy and I talked about like a week ago, where datasets like "super_glue boolq" are now named like super_glue:boolq so that you don't have to wrap it in double quotes on the command line