czy1999 / MultiTQ

MULTITQ is a large-scale dataset featuring ample relevant facts and multiple temporal granularities.
https://huggingface.co/datasets/chenziyang/MultiTQ
15 stars 3 forks source link

The freeze_support() ERROR #5

Closed nanxfu closed 6 months ago

nanxfu commented 6 months ago

Thanks for your greate work! I completely follow with your step.But when I run python ./train_qa_model.py --model multiqa, the console reoport ERROR like this:

Creating new log file
Starting training
  0%|          | 0/6044 [00:00<?, ?batches/s]
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\L\anaconda3\envs\qtbb\lib\multiprocessing\spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "C:\Users\L\anaconda3\envs\qtbb\lib\multiprocessing\spawn.py", line 125, in _main
    prepare(preparation_data)
  File "C:\Users\L\anaconda3\envs\qtbb\lib\multiprocessing\spawn.py", line 236, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "C:\Users\L\anaconda3\envs\qtbb\lib\multiprocessing\spawn.py", line 287, in _fixup_main_from_path
    main_content = runpy.run_path(main_path,
  File "C:\Users\L\anaconda3\envs\qtbb\lib\runpy.py", line 265, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "C:\Users\L\anaconda3\envs\qtbb\lib\runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "C:\Users\L\anaconda3\envs\qtbb\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "D:\MultiTQ\MultiQA\train_qa_model.py", line 212, in <module>
    train(qa_model, dataset, valid_dataset, args, result_filename=result_filename)
  File "D:\MultiTQ\MultiQA\utils.py", line 505, in train
    for i_batch, a in enumerate(loader):
  File "C:\Users\L\anaconda3\envs\qtbb\lib\site-packages\tqdm\std.py", line 1178, in __iter__
    for obj in iterable:
  File "C:\Users\L\anaconda3\envs\qtbb\lib\site-packages\torch\utils\data\dataloader.py", line 438, in __iter__
    return self._get_iterator()
  File "C:\Users\L\anaconda3\envs\qtbb\lib\site-packages\torch\utils\data\dataloader.py", line 386, in _get_iterator
    return _MultiProcessingDataLoaderIter(self)
  File "C:\Users\L\anaconda3\envs\qtbb\lib\site-packages\torch\utils\data\dataloader.py", line 1039, in __init__
    w.start()
  File "C:\Users\L\anaconda3\envs\qtbb\lib\multiprocessing\process.py", line 121, in start
    self._popen = self._Popen(self)
  File "C:\Users\L\anaconda3\envs\qtbb\lib\multiprocessing\context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "C:\Users\L\anaconda3\envs\qtbb\lib\multiprocessing\context.py", line 327, in _Popen
    return Popen(process_obj)
  File "C:\Users\L\anaconda3\envs\qtbb\lib\multiprocessing\popen_spawn_win32.py", line 45, in __init__
    prep_data = spawn.get_preparation_data(process_obj._name)
  File "C:\Users\L\anaconda3\envs\qtbb\lib\multiprocessing\spawn.py", line 154, in get_preparation_data
    _check_not_importing_main()
  File "C:\Users\L\anaconda3\envs\qtbb\lib\multiprocessing\spawn.py", line 134, in _check_not_importing_main
    raise RuntimeError('''
RuntimeError: 
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.
x
Process finished with exit code -1

Before this error,I encounter The gbk encoding error, but I solved this error by adding the "encoding=UTF-8" in open() function. I think it's not the main reason cause the RuntimeError(but, the origin code literally produce this problem).

So, how do i resolve this RuntimeError?, Does it's caused by different dependencies version?

Thanks for your answering

czy1999 commented 6 months ago

It seems you are using Window system.

To resolve the RuntimeError related to multiprocessing on Windows, ensure your code utilizes the if name == 'main': guard and calls multiprocessing.freeze_support() within it. This ensures proper initialization of subprocesses and prevents conflicts, allowing your multiprocessing code to run smoothly.


import multiprocessing

def main():
    # Your main code here

if __name__ == '__main__':
    multiprocessing.freeze_support()
    main()
nanxfu commented 6 months ago

I just set num_worker = 0 ,it's work for me. I will try your code in my spare time and report for your project. By the way,how to understand the Single/Multiple question, I read the example question in the paper, but still confused with It, and the Equal type is vague for me.

Are there any specific definition or example in other paper?If sure,I could read it until I figure out it thanks for your working again!! image

czy1999 commented 6 months ago

Single questions refer to questions that have only one time constraint, for example, asking what someone or someplace did in a certain year. Multiple questions, on the other hand, involve multiple temporal reasoning constraints, such as a question that requires both equal and first constraints. It needs two rounds of temporal reasoning to solve such a problem.

We first defined these categories in the paper, and as far as we know, there aren't any other papers that have similar definitions.

nanxfu commented 6 months ago

I see.The "First, Last, Before, After"type is easy to understand.However, could you introduce "Equal" type question more specifically?

I recognize it as the "normal type",that is to say:No Before/After,First/Last constraint in the question.Is this understanding correct?

czy1999 commented 6 months ago

For example,

Before: < t
Who was the President of the United States before the year 2005?
After: >t
Who was the President of the United States after the year 2005?
Equal: =t
Who was the President of the United States in 2005?