patnr / HistoryMatching

Notebooks on production optimisation and history matching
MIT License
22 stars 13 forks source link

Forecasting issue #7

Closed mspringfield1 closed 1 year ago

mspringfield1 commented 1 year ago

I'm trying to use this file in my own enviroment. I've set up the enviroment, installed the requirements.txt and have opened it up in jupyter notebook just as the README suggests but when I run the code, its holding up just after the forward model process. The line that it's crashing at is

(wsat.past.Prior, prod.past.Prior) = forward_model(nTime, wsat.init.Prior, perm.Prior)

The error message I'm receiving is

A Jupyter widget could not be displayed because the widget state could not be found. This could happen if the kernel storing the widget is no longer available, or if the widget state was not saved in the notebook. You may be able to create the widget by running the appropriate cells.

RemoteTraceback Traceback (most recent call last) RemoteTraceback: """ Traceback (most recent call last): File "C:*\Anaconda3\lib\site-packages\multiprocess\pool.py", line 125, in worker result = (True, func(args, kwds)) File "C:*\Anaconda3\lib\site-packages\pathos\helpers\mp_helper.py", line 15, in func = lambda args: f(args) File "C:\\AppData\Local\Temp\ipykernel_4264\2723154157.py", line 10, in run1 NameError: name 'copy' is not defined """

The above exception was the direct cause of the following exception:

NameError Traceback (most recent call last) ~\AppData\Local\Temp\ipykernel_4264\682821882.py in 1 (wsat.past.Prior, ----> 2 prod.past.Prior) = forward_model(nTime, wsat.init.Prior, perm.Prior)

~\AppData\Local\Temp\ipykernel_4264\2723154157.py in forward_model(nTime, *variables, **kwargs) 31 32 # Dispatch jobs ---> 33 Ef = mp(run1, E, total=len(variables[0]), 34 desc="Ens-run"+kwargs.get("desc", ""), leave=kwargs.get("leave", True)) 35

~**\Documents\HistoryMatching-master\tools\utils.py in mp(fun, args, desc, total, leave) 155 156 from p_tqdm import p_map --> 157 return p_map(fun, list(args), desc=desc, num_cpus=nCores, leave=leave) 158 else: 159 return progbar(map(fun, args), desc=desc, total=total, leave=leave)

~\Anaconda3\lib\site-packages\p_tqdm\p_tqdm.py in p_map(function, *iterables, *kwargs) 63 ordered = True 64 generator = _parallel(ordered, function, iterables, **kwargs) ---> 65 result = list(generator) 66 67 return result

~\Anaconda3\lib\site-packages\p_tqdm\p_tqdm.py in _parallel(ordered, function, *iterables, *kwargs) 52 tqdm_func = kwargs.pop('tqdm', tqdm) 53 ---> 54 for item in tqdm_func(map_func(function, iterables), total=length, **kwargs): 55 yield item 56

~\Anaconda3\lib\site-packages\tqdm\notebook.py in iter(self) 252 try: 253 it = super(tqdm_notebook, self).iter() --> 254 for obj in it: 255 # return super(tqdm...) will not catch exception 256 yield obj

~\Anaconda3\lib\site-packages\tqdm\std.py in iter(self) 1176 1177 try: -> 1178 for obj in iterable: 1179 yield obj 1180 # Update and possibly print the progressbar.

~\Anaconda3\lib\site-packages\multiprocess\pool.py in next(self, timeout) 868 if success: 869 return value --> 870 raise value 871 872 next = next # XXX

NameError: name 'copy' is not defined

I;ve also ran the MAIN python file in VS Code and ran the debugger and it's resulting in an error in the utils.py file on line 157.

return p_map(fun, list(args), desc=desc, num_cpus=nCores, leave=leave)

Exception has occurred: RuntimeError (note: full exception trace is shown but execution is paused at: )

    An attempt has been made to start a new process before the
    current process has finished its bootstrapping phase.

    This probably means that you are not using fork to start your
    child processes and you have forgotten to use the proper idiom
    in the main module:

        if __name__ == '__main__':
            freeze_support()
            ...

    The "freeze_support()" line can be omitted if the program
    is not going to be frozen to produce an executable.

File "C:**\Documents\HistoryMatching-master\tools\utils.py", line 157, in mp return p_map(fun, list(args), desc=desc, num_cpus=nCores, leave=leave) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:**\Documents\HistoryMatching-master\MAIN.py", line 408, in forward_model Ef = mp(run1, E, total=len(variables[0]), ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:**\Documents\HistoryMatching-master\MAIN.py", line 441, in prod.past.Prior) = forward_model(nTime, wsat.init.Prior, perm.Prior) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "", line 1, in (Current frame) RuntimeError: An attempt has been made to start a new process before the current process has finished its bootstrapping phase.

    This probably means that you are not using fork to start your
    child processes and you have forgotten to use the proper idiom
    in the main module:

        if __name__ == '__main__':
            freeze_support()
            ...

    The "freeze_support()" line can be omitted if the program
    is not going to be frozen to produce an executable.

I'm not sure why this result is happening because I followed all the steps to run the code on my own server. Any help would be great.

patnr commented 1 year ago

I'm guessing this is a Windows/Python/MPL/Multiprocessing issue. Cannot debug it in detail at the moment, but you can probably get around it by replacing the line

mp = get_map(multiprocessing=4)
mp = get_map(multiprocessing=False)

or with

mp = map
mspringfield1 commented 1 year ago

Yes, it's a windows issue. The code you gave (mp = get_map(multiprocessing=False)) ending up working to have the code run without interruption. I'm now trying to figure out why some of the plots wont show when I run the code. When I launch jupyter notebook it defaults to microsoft edge which may be the problem but I'm not sure.