search

microsoft / DeBERTa

The implementation of DeBERTa
MIT License
1.91k stars 215 forks source link

Trying to run rtd_task.py on Windows #137

Open Yuri-Albuquerque opened 1 year ago

Yuri-Albuquerque commented 1 year ago

Issue Description:

I am encountering difficulties with the multiprocessing package. When attempting to run my code using the assync_dataloader on a single GPU in MS Windows 10, I encountered the following error:

File "C:\Users\USER\miniconda3\lib\multiprocessing\context.py", line 336, in _Popen
    return Popen(process_obj)
  File "C:\Users\USER\miniconda3\lib\multiprocessing\popen_spawn_win32.py", line 93, in __init__
    reduction.dump(process_obj, to_child)
  File "C:\Users\USER\miniconda3\lib\multiprocessing\reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'RTDTask.get_feature_fn.<locals>._example_to_feature'

It seems that the issue is related to the RTDTask.get_feature_fn.<locals>._example_to_feature function and its inability to be pickled.

Any assistance in resolving this problem would be greatly appreciated.

BartWesthoff commented 11 months ago

did you end up having a solution? I am facing the same.