Parsl / parsl

Parsl - a Python parallel scripting library
http://parsl-project.org
Apache License 2.0
506 stars 195 forks source link

Monitoring broken on MacOS #616

Open annawoodard opened 6 years ago

annawoodard commented 6 years ago

Running this test script from @garri1105 works on midway but not on my local machine (macOS 10.13.3). Note that if I make the DB elsewhere and copy it locally I'm able to run visualize without issue.

import parsl
from parsl.app.app import python_app
from parsl.monitoring.db_logger import MonitoringConfig
import random

threads_config = parsl.config.Config(
    executors=[parsl.executors.threads.ThreadPoolExecutor(label='threads', max_threads=4)],
    monitoring_config=MonitoringConfig(
        database_type='local_database',
        logger_name='parsl_db_logger',
        eng_link='sqlite:///DB2.db',
        web_app_host='http://localhost',
        web_app_port=8000,
        resource_loop_sleep_duration=2
    )
)

dfk = parsl.load(threads_config)

import time

@python_app
def bitcoin_mining(seconds):
    time.sleep(seconds)
    if random.randint(0, 10) >= 8:
        raise AssertionError("Just an Error")
    return seconds

@python_app
def feeding_the_poor(seconds):
    time.sleep(seconds)
    if random.randint(0, 10) >= 9:
        raise AssertionError("Just an Error")
    return seconds

tasks = [bitcoin_mining(random.random() * 3) for _ in range(0, 100)] + [feeding_the_poor(random.random() * 3) for _ in range(0, 100)]
for task in tasks:
    print(task)
    try:
        task.result()
    except: pass

Fails with:

 >python make_db.py
 * Visualizing DB2.db
 * Running on http://localhost:8001
Process Process-2:
Traceback (most recent call last):
  File "/Users/awoodard/software/miniconda3/envs/parsl/lib/python3.7/asyncio/selector_events.py", line 262, in _add_reader
    key = self._selector.get_key(fd)
  File "/Users/awoodard/software/miniconda3/envs/parsl/lib/python3.7/selectors.py", line 192, in get_key
    raise KeyError("{!r} is not registered".format(fileobj)) from None
KeyError: '14 is not registered'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/awoodard/software/miniconda3/envs/parsl/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/Users/awoodard/software/miniconda3/envs/parsl/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/awoodard/software/miniconda3/envs/parsl/lib/python3.7/site-packages/parsl/monitoring/logging_server.py", line 44, in run
    app.listen(monitoring_config.web_app_port)
  File "/Users/awoodard/software/miniconda3/envs/parsl/lib/python3.7/site-packages/tornado/web.py", line 2042, in listen
    server.listen(port, address)
  File "/Users/awoodard/software/miniconda3/envs/parsl/lib/python3.7/site-packages/tornado/tcpserver.py", line 144, in listen
    self.add_sockets(sockets)
  File "/Users/awoodard/software/miniconda3/envs/parsl/lib/python3.7/site-packages/tornado/tcpserver.py", line 158, in add_sockets
    sock, self._handle_connection)
  File "/Users/awoodard/software/miniconda3/envs/parsl/lib/python3.7/site-packages/tornado/netutil.py", line 268, in add_accept_handler
    io_loop.add_handler(sock, accept_handler, IOLoop.READ)
  File "/Users/awoodard/software/miniconda3/envs/parsl/lib/python3.7/site-packages/tornado/platform/asyncio.py", line 80, in add_handler
    fd, self._handle_events, fd, IOLoop.READ)
  File "/Users/awoodard/software/miniconda3/envs/parsl/lib/python3.7/asyncio/selector_events.py", line 335, in add_reader
    return self._add_reader(fd, callback, *args)
  File "/Users/awoodard/software/miniconda3/envs/parsl/lib/python3.7/asyncio/selector_events.py", line 265, in _add_reader
    (handle, None))
  File "/Users/awoodard/software/miniconda3/envs/parsl/lib/python3.7/selectors.py", line 522, in register
    self._selector.control([kev], 0, 0)
OSError: [Errno 9] Bad file descriptor
benclifford commented 6 years ago

Note that if I make the DB elsewhere and copy it locally I'm able to run visualize without issue.

This sounds like it's related to DB collection rather than to visualisation? If so, you could attach a sample broken database to this issue.