Closed codecaution closed 3 years ago
We’ll try to reproduce the issue on our side. We’ll keep you posted. Thanks!
We’ll try to reproduce the issue on our side. We’ll keep you posted. Thanks!
Thanks for your help!
We (NVIDIA engineers) were able to reproduce strange perplexity results and we are trying to identify the root cause. We will update you as we know more. Thanks for reporting the issue and for the reproducer.
Hi, I think #12004 is an related issue
Environment info
transformers
version: 4.7.0.dev0Who can help
@jdemouth @LysandreJik @sgugger
Information
Model I am using gpt2(megatron-gpt2-345m):
The problem arises when using:
The tasks I am working on is:
To reproduce
Steps to reproduce the behavior:
Follow the steps given by huggingface to convert the megatron-lm model to huggingface model.
python3 $MYDIR/transformers/src/transformers/models/megatron_gpt2/convert_megatron_gpt2_checkpoint.py $MYDIR/nvidia/megatron-gpt2-345m/checkpoint.zip (Here I meet error: "io.UnsupportedOperation: seek. You can only torch.load from a file that is seekable. Please pre-load the data into a buffer like io.BytesIO and try to load from it instead." And I solve it by,
Expected behavior
I want to convert my megatron-lm model checkpoints into huggingface. Please help me.