Unable to run discriminator

[Leo-0906]

Hello there! Great job with this model!

However, I am getting an error while running a run_discrminator of 'utf-8' codec can't decode byte 0xf8 in position 1: invalid start byte' . Can you help me with that? And also I am a little confused about how to run this discriminator so can you guide me with that? Thank You.

[grantnelson]

+1 A Guide/more documentation on running the discriminator would be awesome!

[AiliAili]

Hi there,

I've successfully made it runnable.Script I've used is:

python ./discrimination/run_discrimination.py --input_data=./generator=mega~dataset=p0.94.jsonl --do_train=True --output_dir=./tem --config_file=./lm/configs/base.json

I am running in the root directory of grover, without using pretrained discriminators.

Hope it can help.

[ecrows]

For those trying to use the pretrained models, here's some basic steps (I used the medium* model as an example).

1. Use gsutil to download the following files:

   gs://grover-models/discrimination/generator=medium~discriminator=grover~discsize=medium~dataset=p=0.96/model.ckpt-1562.data-00000-of-00001
   gs://grover-models/discrimination/generator=medium~discriminator=grover~discsize=medium~dataset=p=0.96/model.ckpt-1562.index
   gs://grover-models/discrimination/generator=medium~discriminator=grover~discsize=medium~dataset=p=0.96/model.ckpt-1562.meta

You will also need this one (which isn't currently listed):

gs://grover-models/discrimination/generator=medium~discriminator=grover~discsize=medium~dataset=p=0.96/checkpoint

2. If you need some sample input data download it from the below GCS file.

gs://grover-models/generation_examples/generator=mega~dataset=p0.94.jsonl

Note that each record has a "split" key that determines whether it is "train", "val", or "test" data. When you call the run_discrimination.py script, you can set "predict_val" or "predict_test" to true.

3. Call the script. Remember you need to set your PYTHONPATH first (as in the main README file).

For example:

python ./discrimination/run_discrimination.py --input_data ./generator_mega_dataset_p0.94.jsonl --output_dir out/ --predict_val true --config_file lm/configs/large.json

*One final note, in the "discrimination.py" script, the model called "medium" is actually Grover-Large from the paper, and therefore uses the "lm/configs/large.json" configuration file. The development name is likely because the size corresponds to GPT-2 medium at 355M parameters.

Hope this helps some people!