Open AlexH726 opened 1 year ago
I should also mention the computer is running Windows 10 and python 3.10
Hi @AlexH726 , it looks like the error is not arising from within pygmo code but rather in runa_n.py
. From the look of it, you are doing some multiprocessing the in the implementation of your optimisation problem and Python seems to be complaining that multiple processes are attempting to create a file/directory with the same name at the same time.
I would suggest to try using the tempfile
module if you need to read/write data from/to disk in your optimisation problem:
Hi @bluescarni all I am doing is creating a new file, copying in all the necessary files to run, and a target list, then changing the directory and running from that new file. What is interesting is that I have been able to run a number of native algorithms like de, sga, and aco in addition to some user-defined algorithms without problem. When I try to do this with the asynchronous island model, it seems to set off some sort of perpetual loop periodically when it hits __original_island_init(self, *args) (line 388 in init.py) line in pygmo. It does not seem to matter if I use a temporary directory or a permanent one. If I do let it all run in the file with the original function and just copy in the target list instead of copying everything into a new file I still get an error associated with the multiprocessing library:
Traceback (most recent call last):
File "
This probably means that you are not using fork to start your
child processes and you have forgotten to use the proper idiom
in the main module:
if __name__ == '__main__':
freeze_support()
...
The "freeze_support()" line can be omitted if the program
is not going to be frozen to produce an executable.
@AlexH726 does import
ing your code result in the creation of a multiprocessing pool?
@AlexH726 to expand a bit, a multiprocessing pool should never be created as a result of import
ing a submodule, as that leads to an endless recursion as explained by the error message. See also here:
https://stackoverflow.com/questions/18204782/runtimeerror-on-windows-trying-python-multiprocessing
Note that creating a pygmo mp island indirectly triggers the creation of a multiprocessing pool, so you should never have island-creating code executed as part of an import
statement.
Hi all, I am encountering an issue with the pygmo archipelago pushback command. I have gotten it to run successfully with this code but more often than not when the code hits the pushback lines I get to following error with the multiprocessing library. Any help on fixing this or workarounds would be greatly appreciated. The applicable code is included below as well as the traceback. it is running in a folder called 1runANN_f but when this error pops up it creates another folder by the same name in the original and copies most of the contents of the old folder into the new one. Best, Alex
Code: algos=[pg.algorithm(pg.nsga2(gen=5, cr=0.95, eta_c=10, m=0.01, eta_m=10)), pg.algorithm(pg.de(gen=5))] prob=pg.problem(seqopt(deb, in_prop[20], seed)) archi = pg.archipelago(topology=pg.fully_connected()) archi.push_back(pg.island(algo=algos[0],prob=prob,size=in_prop[20]2)) archi.push_back(pg.island(algo=algos[1],prob=prob,size=in_prop[20]2))
Traceback (most recent call last): File "", line 1, in
File "C:\Users\ahickey7\Anaconda3\envs\DLoc\lib\multiprocessing\spawn.py", line 116, in spawn_main
exitcode = _main(fd, parent_sentinel)
File "C:\Users\ahickey7\Anaconda3\envs\DLoc\lib\multiprocessing\spawn.py", line 125, in _main
prepare(preparation_data)
File "C:\Users\ahickey7\Anaconda3\envs\DLoc\lib\multiprocessing\spawn.py", line 236, in prepare
_fixup_main_from_path(data['init_main_from_path'])
File "C:\Users\ahickey7\Anaconda3\envs\DLoc\lib\multiprocessing\spawn.py", line 287, in _fixup_main_from_path
main_content = runpy.run_path(main_path,
File "C:\Users\ahickey7\Anaconda3\envs\DLoc\lib\runpy.py", line 289, in run_path
return _run_module_code(code, init_globals, run_name,
File "C:\Users\ahickey7\Anaconda3\envs\DLoc\lib\runpy.py", line 96, in _run_module_code
_run_code(code, mod_globals, init_globals,
File "C:\Users\ahickey7\Anaconda3\envs\DLoc\lib\runpy.py", line 86, in _run_code
exec(code, run_globals)
File "c:\users\ahickey7\documents\dissertation\runa_n.py", line 13, in
os.mkdir(path)
FileExistsError: [WinError 183] Cannot create a file when that file already exists: '1runANN_f/.'
Traceback (most recent call last):
File "", line 1, in
File "C:\Users\ahickey7\Anaconda3\envs\DLoc\lib\multiprocessing\spawn.py", line 116, in spawn_main
exitcode = _main(fd, parent_sentinel)
File "C:\Users\ahickey7\Anaconda3\envs\DLoc\lib\multiprocessing\spawn.py", line 125, in _main
prepare(preparation_data)
File "C:\Users\ahickey7\Anaconda3\envs\DLoc\lib\multiprocessing\spawn.py", line 236, in prepare
_fixup_main_from_path(data['init_main_from_path'])
File "C:\Users\ahickey7\Anaconda3\envs\DLoc\lib\multiprocessing\spawn.py", line 287, in _fixup_main_from_path
main_content = runpy.run_path(main_path,
File "C:\Users\ahickey7\Anaconda3\envs\DLoc\lib\runpy.py", line 289, in run_path
return _run_module_code(code, init_globals, run_name,
File "C:\Users\ahickey7\Anaconda3\envs\DLoc\lib\runpy.py", line 96, in _run_module_code
_run_code(code, mod_globals, init_globals,
File "C:\Users\ahickey7\Anaconda3\envs\DLoc\lib\runpy.py", line 86, in _run_code
exec(code, run_globals)
File "c:\users\ahickey7\documents\dissertation\runa_n.py", line 13, in
os.mkdir(path)
FileExistsError: [WinError 183] Cannot create a file when that file already exists: '1runANN_f/.'
Traceback (most recent call last):
File "", line 1, in
File "C:\Users\ahickey7\Anaconda3\envs\DLoc\lib\multiprocessing\spawn.py", line 116, in spawn_main
exitcode = _main(fd, parent_sentinel)
File "C:\Users\ahickey7\Anaconda3\envs\DLoc\lib\multiprocessing\spawn.py", line 125, in _main
prepare(preparation_data)
File "C:\Users\ahickey7\Anaconda3\envs\DLoc\lib\multiprocessing\spawn.py", line 236, in prepare
_fixup_main_from_path(data['init_main_from_path'])
File "C:\Users\ahickey7\Anaconda3\envs\DLoc\lib\multiprocessing\spawn.py", line 287, in _fixup_main_from_path
main_content = runpy.run_path(main_path,
File "C:\Users\ahickey7\Anaconda3\envs\DLoc\lib\runpy.py", line 289, in run_path
return _run_module_code(code, init_globals, run_name,
File "C:\Users\ahickey7\Anaconda3\envs\DLoc\lib\runpy.py", line 96, in _run_module_code
_run_code(code, mod_globals, init_globals,
File "C:\Users\ahickey7\Anaconda3\envs\DLoc\lib\runpy.py", line 86, in _run_code
exec(code, run_globals)
File "c:\users\ahickey7\documents\dissertation\runa_n.py", line 13, in
os.mkdir(path)
FileExistsError: [WinError 183] Cannot create a file when that file already exists: '1runANN_f/.'
Traceback (most recent call last):
File "", line 1, in
File "C:\Users\ahickey7\Anaconda3\envs\DLoc\lib\multiprocessing\spawn.py", line 116, in spawn_main
exitcode = _main(fd, parent_sentinel)
File "C:\Users\ahickey7\Anaconda3\envs\DLoc\lib\multiprocessing\spawn.py", line 125, in _main
prepare(preparation_data)
File "C:\Users\ahickey7\Anaconda3\envs\DLoc\lib\multiprocessing\spawn.py", line 236, in prepare
_fixup_main_from_path(data['init_main_from_path'])
File "C:\Users\ahickey7\Anaconda3\envs\DLoc\lib\multiprocessing\spawn.py", line 287, in _fixup_main_from_path
main_content = runpy.run_path(main_path,
File "C:\Users\ahickey7\Anaconda3\envs\DLoc\lib\runpy.py", line 289, in run_path
return _run_module_code(code, init_globals, run_name,
File "C:\Users\ahickey7\Anaconda3\envs\DLoc\lib\runpy.py", line 96, in _run_module_code
_run_code(code, mod_globals, init_globals,
File "C:\Users\ahickey7\Anaconda3\envs\DLoc\lib\runpy.py", line 86, in _run_code
exec(code, run_globals)
File "c:\users\ahickey7\documents\dissertation\runa_n.py", line 13, in
os.mkdir(path)
FileExistsError: [WinError 183] Cannot create a file when that file already exists: '1runANN_f/.'
Traceback (most recent call last):
File "", line 1, in
File "C:\Users\ahickey7\Anaconda3\envs\DLoc\lib\multiprocessing\spawn.py", line 116, in spawn_main
exitcode = _main(fd, parent_sentinel)
File "C:\Users\ahickey7\Anaconda3\envs\DLoc\lib\multiprocessing\spawn.py", line 125, in _main
prepare(preparation_data)
File "C:\Users\ahickey7\Anaconda3\envs\DLoc\lib\multiprocessing\spawn.py", line 236, in prepare
_fixup_main_from_path(data['init_main_from_path'])
File "C:\Users\ahickey7\Anaconda3\envs\DLoc\lib\multiprocessing\spawn.py", line 287, in _fixup_main_from_path
main_content = runpy.run_path(main_path,
File "C:\Users\ahickey7\Anaconda3\envs\DLoc\lib\runpy.py", line 289, in run_path
return _run_module_code(code, init_globals, run_name,
File "C:\Users\ahickey7\Anaconda3\envs\DLoc\lib\runpy.py", line 96, in _run_module_code
_run_code(code, mod_globals, init_globals,
File "C:\Users\ahickey7\Anaconda3\envs\DLoc\lib\runpy.py", line 86, in _run_code
exec(code, run_globals)
File "c:\users\ahickey7\documents\dissertation\runa_n.py", line 13, in
os.mkdir(path)
FileExistsError: [WinError 183] Cannot create a file when that file already exists: '1runANN_f/.'
Traceback (most recent call last): File "", line 1, in
File "C:\Users\ahickey7\Anaconda3\envs\DLoc\lib\multiprocessing\spawn.py", line 116, in spawn_main
exitcode = _main(fd, parent_sentinel)
File "C:\Users\ahickey7\Anaconda3\envs\DLoc\lib\multiprocessing\spawn.py", line 125, in _main
prepare(preparation_data)
File "C:\Users\ahickey7\Anaconda3\envs\DLoc\lib\multiprocessing\spawn.py", line 236, in prepare
_fixup_main_from_path(data['init_main_from_path'])
File "C:\Users\ahickey7\Anaconda3\envs\DLoc\lib\multiprocessing\spawn.py", line 287, in _fixup_main_from_path
main_content = runpy.run_path(main_path,
File "C:\Users\ahickey7\Anaconda3\envs\DLoc\lib\runpy.py", line 289, in run_path
return _run_module_code(code, init_globals, run_name,
File "C:\Users\ahickey7\Anaconda3\envs\DLoc\lib\runpy.py", line 96, in _run_module_code
_run_code(code, mod_globals, init_globals,
File "C:\Users\ahickey7\Anaconda3\envs\DLoc\lib\runpy.py", line 86, in _run_code
exec(code, run_globals)
File "c:\users\ahickey7\documents\dissertation\runa_n.py", line 13, in
os.mkdir(path)
FileExistsError: [WinError 183] Cannot create a file when that file already exists: '1runANN_f/.'