lanl / PyBNF

An application for parameterization of biological models available in SBML and BNGL formats. Features include parallelization, metaheuristic optimization algorithms, and an adaptive Markov chain Monte Carlo (MCMC) sampling algorithm.
Other
22 stars 18 forks source link

installation issue with dask.distributed? #260

Closed psthiagu closed 5 years ago

psthiagu commented 5 years ago

Hi, I have installed PyBNF. When I run the simple job "demo_bng.conf" I get the follwoing error. I am also attaching the log file.


Traceback (most recent call last): File "", line 1, in File "/home/thiagu/anaconda3/lib/python3.7/multiprocessing/forkserver.py", line 186, in main with socket.socket(socket.AF_UNIX, fileno=listener_fd) as listener, \ File "/home/thiagu/anaconda3/lib/python3.7/socket.py", line 151, in init _socket.socket.init(self, family, type, proto, fileno) OSError: [Errno 22] Invalid argument: 'protocol' Sorry, an unknown error occurred: EOFError: unexpected EOF


Best, Thiagu psthiagu@gmail.com

bnf_20190109-171007.log

emitra17 commented 5 years ago

It might be an issue with tornado versions? Could you run python and check:

import tornado
tornado.version
tornado.__file__
psthiagu commented 5 years ago

Thanks for the quick response.

I see:

tornado.version = '4.5.3'

tornado.file = '/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/init.py'

Which version of tornado should I be using?

On Wed, Jan 9, 2019 at 9:29 PM Eshan Mitra notifications@github.com wrote:

It might be an issue with tornado versions? Could you run python and check:

import tornado tornado.version tornado.file

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/lanl/PyBNF/issues/260#issuecomment-452748805, or mute the thread https://github.com/notifications/unsubscribe-auth/AQCvXte2NCXk3mcAOtw9y2GHB1uDOilIks5vBhHQgaJpZM4Z3VJj .

emitra17 commented 5 years ago

Ok, that looks correct.

Let's check something else, does this work without crashing?

import distributed
distributed.Client()
psthiagu commented 5 years ago

Indeed it throws up a long error message. I am appending what I could capture from the screen.

best,

Thiagu

On Wed, Jan 9, 2019 at 10:44 PM Eshan Mitra notifications@github.com wrote:

Ok, that looks correct.

Let's check something else, does this work without crashing?

import distributed distributed.Client()

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/lanl/PyBNF/issues/260#issuecomment-452768198, or mute the thread https://github.com/notifications/unsubscribe-auth/AQCvXgHQzz4hea5VBb_IxfnLbFv_tfu-ks5vBiN0gaJpZM4Z3VJj .

File "", line 4, in raise_exc_info File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/gen.py", line 1063, in run yielded = self.gen.throw(exc_info) File "/home/thiagu/anaconda3/lib/python3.7/site-packages/distributed/nanny.py", line 157, in _start response = yield self.instantiate() File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/gen.py", line 1055, in run value = future.result() File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/concurrent.py", line 238, in result raise_exc_info(self._exc_info) File "", line 4, in raise_exc_info File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/gen.py", line 1063, in run yielded = self.gen.throw(exc_info) File "/home/thiagu/anaconda3/lib/python3.7/site-packages/distributed/nanny.py", line 226, in instantiate self.process.start() File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/gen.py", line 1055, in run value = future.result() File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/concurrent.py", line 238, in result raise_exc_info(self._exc_info) File "", line 4, in raise_exc_info File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/gen.py", line 1063, in run yielded = self.gen.throw(exc_info) File "/home/thiagu/anaconda3/lib/python3.7/site-packages/distributed/nanny.py", line 370, in start yield self.process.start() File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/gen.py", line 1055, in run value = future.result() File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/concurrent.py", line 238, in result raise_exc_info(self._exc_info) File "", line 4, in raise_exc_info File "/home/thiagu/anaconda3/lib/python3.7/site-packages/distributed/process.py", line 35, in _call_and_set_future res = func(args, *kwargs) File "/home/thiagu/anaconda3/lib/python3.7/site-packages/distributed/process.py", line 184, in _start process.start() File "/home/thiagu/anaconda3/lib/python3.7/multiprocessing/process.py", line 112, in start self._popen = self._Popen(self) File "/home/thiagu/anaconda3/lib/python3.7/multiprocessing/context.py", line 291, in _Popen return Popen(process_obj) File "/home/thiagu/anaconda3/lib/python3.7/multiprocessing/popen_forkserver.py", line 35, in init super().init(process_obj) File "/home/thiagu/anaconda3/lib/python3.7/multiprocessing/popen_fork.py", line 20, in init self._launch(process_obj) File "/home/thiagu/anaconda3/lib/python3.7/multiprocessing/popen_forkserver.py", line 55, in _launch self.pid = forkserver.read_signed(self.sentinel) File "/home/thiagu/anaconda3/lib/python3.7/multiprocessing/forkserver.py", line 312, in read_signed raise EOFError('unexpected EOF') EOFError: unexpected EOF Traceback (most recent call last): File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/gen.py", line 828, in callback result_list.append(f.result()) File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/concurrent.py", line 238, in result raise_exc_info(self._exc_info) File "", line 4, in raise_exc_info File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/gen.py", line 1063, in run yielded = self.gen.throw(exc_info) File "/home/thiagu/anaconda3/lib/python3.7/site-packages/distributed/deploy/local.py", line 208, in _start_worker yield w._start() File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/gen.py", line 1055, in run value = future.result() File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/concurrent.py", line 238, in result raise_exc_info(self._exc_info) File "", line 4, in raise_exc_info File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/gen.py", line 1063, in run yielded = self.gen.throw(exc_info) File "/home/thiagu/anaconda3/lib/python3.7/site-packages/distributed/nanny.py", line 157, in _start response = yield self.instantiate() File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/gen.py", line 1055, in run value = future.result() File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/concurrent.py", line 238, in result raise_exc_info(self._exc_info) File "", line 4, in raise_exc_info File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/gen.py", line 1063, in run yielded = self.gen.throw(exc_info) File "/home/thiagu/anaconda3/lib/python3.7/site-packages/distributed/nanny.py", line 226, in instantiate self.process.start() File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/gen.py", line 1055, in run value = future.result() File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/concurrent.py", line 238, in result raise_exc_info(self._exc_info) File "", line 4, in raise_exc_info File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/gen.py", line 1063, in run yielded = self.gen.throw(exc_info) File "/home/thiagu/anaconda3/lib/python3.7/site-packages/distributed/nanny.py", line 370, in start yield self.process.start() File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/gen.py", line 1055, in run value = future.result() File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/concurrent.py", line 238, in result raise_exc_info(self._exc_info) File "", line 4, in raise_exc_info File "/home/thiagu/anaconda3/lib/python3.7/site-packages/distributed/process.py", line 35, in _call_and_set_future res = func(args, **kwargs) File "/home/thiagu/anaconda3/lib/python3.7/site-packages/distributed/process.py", line 184, in _start process.start() File "/home/thiagu/anaconda3/lib/python3.7/multiprocessing/process.py", line 112, in start self._popen = self._Popen(self) File "/home/thiagu/anaconda3/lib/python3.7/multiprocessing/context.py", line 291, in _Popen return Popen(process_obj) File "/home/thiagu/anaconda3/lib/python3.7/multiprocessing/popen_forkserver.py", line 35, in init super().init(process_obj) File "/home/thiagu/anaconda3/lib/python3.7/multiprocessing/popen_fork.py", line 20, in init self._launch(process_obj) File "/home/thiagu/anaconda3/lib/python3.7/multiprocessing/popen_forkserver.py", line 55, in _launch self.pid = forkserver.read_signed(self.sentinel) File "/home/thiagu/anaconda3/lib/python3.7/multiprocessing/forkserver.py", line 312, in read_signed raise EOFError('unexpected EOF') EOFError: unexpected EOF

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "", line 1, in File "/home/thiagu/anaconda3/lib/python3.7/site-packages/distributed/client.py", line 628, in init self.start(timeout=timeout) File "/home/thiagu/anaconda3/lib/python3.7/site-packages/distributed/client.py", line 751, in start sync(self.loop, self._start, kwargs) File "/home/thiagu/anaconda3/lib/python3.7/site-packages/distributed/utils.py", line 277, in sync six.reraise(error[0]) File "/home/thiagu/anaconda3/lib/python3.7/site-packages/six.py", line 693, in reraise raise value File "/home/thiagu/anaconda3/lib/python3.7/site-packages/distributed/utils.py", line 262, in f result[0] = yield future File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/gen.py", line 1055, in run value = future.result() File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/concurrent.py", line 238, in result raise_exc_info(self._exc_info) File "", line 4, in raise_exc_info File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/gen.py", line 1063, in run yielded = self.gen.throw(exc_info) File "/home/thiagu/anaconda3/lib/python3.7/site-packages/distributed/client.py", line 814, in _start yield self.cluster File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/gen.py", line 1055, in run value = future.result() File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/concurrent.py", line 238, in result raise_exc_info(self._exc_info) File "", line 4, in raise_exc_info File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/gen.py", line 1063, in run yielded = self.gen.throw(exc_info) File "", line 6, in _wrap_awaitable File "", line 3, in await File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/gen.py", line 1055, in run value = future.result() File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/concurrent.py", line 238, in result raise_exc_info(self._exc_info) File "", line 4, in raise_exc_info File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/gen.py", line 1063, in run yielded = self.gen.throw(exc_info) File "/home/thiagu/anaconda3/lib/python3.7/site-packages/distributed/deploy/local.py", line 191, in _start yield [self._start_worker(self.worker_kwargs) for i in range(n_workers)] File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/gen.py", line 1055, in run value = future.result() File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/concurrent.py", line 238, in result raise_exc_info(self._exc_info) File "", line 4, in raise_exc_info File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/gen.py", line 828, in callback result_list.append(f.result()) File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/concurrent.py", line 238, in result raise_exc_info(self._exc_info) File "", line 4, in raise_exc_info File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/gen.py", line 1063, in run yielded = self.gen.throw(exc_info) File "/home/thiagu/anaconda3/lib/python3.7/site-packages/distributed/deploy/local.py", line 208, in _start_worker yield w._start() File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/gen.py", line 1055, in run value = future.result() File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/concurrent.py", line 238, in result raise_exc_info(self._exc_info) File "", line 4, in raise_exc_info File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/gen.py", line 1063, in run yielded = self.gen.throw(exc_info) File "/home/thiagu/anaconda3/lib/python3.7/site-packages/distributed/nanny.py", line 157, in _start response = yield self.instantiate() File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/gen.py", line 1055, in run value = future.result() File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/concurrent.py", line 238, in result raise_exc_info(self._exc_info) File "", line 4, in raise_exc_info File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/gen.py", line 1063, in run yielded = self.gen.throw(exc_info) File "/home/thiagu/anaconda3/lib/python3.7/site-packages/distributed/nanny.py", line 226, in instantiate self.process.start() File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/gen.py", line 1055, in run value = future.result() File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/concurrent.py", line 238, in result raise_exc_info(self._exc_info) File "", line 4, in raise_exc_info File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/gen.py", line 1063, in run yielded = self.gen.throw(exc_info) File "/home/thiagu/anaconda3/lib/python3.7/site-packages/distributed/nanny.py", line 370, in start yield self.process.start() File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/gen.py", line 1055, in run value = future.result() File "/home/thiagu/anaconda3/lib/python3.7/site-packages/tornado/concurrent.py", line 238, in result raise_exc_info(self._exc_info) File "", line 4, in raise_exc_info File "/home/thiagu/anaconda3/lib/python3.7/site-packages/distributed/process.py", line 35, in _call_and_set_future res = func(*args, **kwargs) File "/home/thiagu/anaconda3/lib/python3.7/site-packages/distributed/process.py", line 184, in _start process.start() File "/home/thiagu/anaconda3/lib/python3.7/multiprocessing/process.py", line 112, in start self._popen = self._Popen(self) File "/home/thiagu/anaconda3/lib/python3.7/multiprocessing/context.py", line 291, in _Popen return Popen(process_obj) File "/home/thiagu/anaconda3/lib/python3.7/multiprocessing/popen_forkserver.py", line 35, in init super().init(process_obj) File "/home/thiagu/anaconda3/lib/python3.7/multiprocessing/popen_fork.py", line 20, in init self._launch(process_obj) File "/home/thiagu/anaconda3/lib/python3.7/multiprocessing/popen_forkserver.py", line 55, in _launch self.pid = forkserver.read_signed(self.sentinel) File "/home/thiagu/anaconda3/lib/python3.7/multiprocessing/forkserver.py", line 312, in read_signed raise EOFError('unexpected EOF') EOFError: unexpected EOF

emitra17 commented 5 years ago

So it looks like an issue with the distributed installation. PyBNF internally calls distributed.Client(), so will crash if that isn't working.

As a quick thing to try, see if it helps to reinstall: pip install distributed==1.24.0 dask==0.20.0

Beyond that, the developers of https://github.com/dask/distributed would probably have better expertise here, so you could try submitting an issue to them.

psthiagu commented 5 years ago

Tried the quick fix. Didn't work. Will contact the the developers.

(probably a) dumb question: Might it help if I ran PyBNF under python3.5?

Best,

Thiagu

On Thu, Jan 10, 2019 at 3:45 AM Eshan Mitra notifications@github.com wrote:

So it looks like an issue with the distributed installation. PyBNF internally calls distributed.Client(), so will crash if that isn't working.

As a quick thing to try, see if it helps to reinstall: pip install distributed==1.24.0 dask==0.20.0

Beyond that, the developers of https://github.com/dask/distributed would probably have better expertise here, so you could try submitting an issue to them.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/lanl/PyBNF/issues/260#issuecomment-452885187, or mute the thread https://github.com/notifications/unsubscribe-auth/AQCvXuSCqA4YOaX747-KiLK2A2R6yCc1ks5vBmbHgaJpZM4Z3VJj .

psthiagu commented 5 years ago

I am running ubuntu on windows10 (windows subsystem for linux). Could this be the problem?

On Thu, Jan 10, 2019 at 3:45 AM Eshan Mitra notifications@github.com wrote:

So it looks like an issue with the distributed installation. PyBNF internally calls distributed.Client(), so will crash if that isn't working.

As a quick thing to try, see if it helps to reinstall: pip install distributed==1.24.0 dask==0.20.0

Beyond that, the developers of https://github.com/dask/distributed would probably have better expertise here, so you could try submitting an issue to them.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/lanl/PyBNF/issues/260#issuecomment-452885187, or mute the thread https://github.com/notifications/unsubscribe-auth/AQCvXuSCqA4YOaX747-KiLK2A2R6yCc1ks5vBmbHgaJpZM4Z3VJj .

emitra17 commented 5 years ago

Running on a windows10 subsystem could be the issue - we haven't tested it on that type of system configuration. I don't think Python 3.5 vs 3.7 will make a difference.

emitra17 commented 5 years ago

Opened issue is dask/distributed#2455

psthiagu commented 5 years ago

Thanks. I will try to migrate to first class Ubuntu.

On Thu, 10 Jan 2019, 20:36 Eshan Mitra <notifications@github.com wrote:

Running on a windows10 subsystem could be the issue - we haven't tested it on that type of system configuration. I don't think Python 3.5 vs 3.7 will make a difference.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/lanl/PyBNF/issues/260#issuecomment-453127253, or mute the thread https://github.com/notifications/unsubscribe-auth/AQCvXljUhZzbyyryBim0naY6B9F4yjNWks5vB1bbgaJpZM4Z3VJj .