Closed gaojingwei closed 2 months ago
Hi, do you also face this issue in megatron_gpt_finetuning.py
if write_predictions_to_file
is set to True? I'm also facing the same error that way
I also meet this issue, w/ a NumPy array is not writable, when the write_predictions_to_file
is True.
I encountered the same issue within 24.05 image
Meet the same issue.
Meet the same issue within 24.01 image
I'm not seeing this issue with our latest dev container so the fix should be alredy in main. It will be included in upcoming 24.07 release container.
I use images:nvcr.io/nvidia/nemo:24.03.framework,and successfully run /NeMo/examples/nlp/language_modeling/tuning/megatron_gpt_finetuning.py but when I run /NeMo/examples/nlp/language_modeling/tuning/megatron_gpt_generate.py, if set model.data.test_ds.write_predictions_to_file=False,it will run success; if set model.data.test_ds.write_predictions_to_file=True,it will run failed。 this is the error log:
this is my conf: