search

nvtransfer / RULER

This repo contains the source code for RULER: What’s the Real Context Size of Your Long-Context Language Models?
Apache License 2.0
646 stars 43 forks source link

pre_sample in qa code #29

Closed vkaul11 closed 4 months ago

vkaul11 commented 4 months ago

I see parser.add_argument("--pre_samples", type=int, default=0, help='number of samples are already generated') used here for qa input_text, answer = generate_input_output(index + args.pre_samples, used_docs) Why would you ever need this argument?

hsiehjackson commented 4 months ago

You can just ignore this parameter. We use this because we run 500 samples evaluation in parallel, and this parameter can help prevent overlapping.