It4innovations / hyperqueue

Scheduler for sub-node tasks for HPC systems with batch scheduling
https://it4innovations.github.io/hyperqueue
MIT License
275 stars 21 forks source link

Python API + client with dashboard #671

Open grzanka opened 8 months ago

grzanka commented 8 months ago

Is there a working combination where I could use hyperqueue server with running dashboard and connect to it from Python API ?

As reported in https://github.com/It4innovations/hyperqueue/issues/670 only v0.17.0-liberec server version has dashboard. I've tried to install hyperqueue pip package from pypi using pip install hyperqueue. It seems to install v0.17.0, not v0.17.0-liberec. When connecting to server with simple python script I get the message:

(venv) [ares][plgkongruencj@ac0040 2022-krakow-lgad]$ python src/ares/submit_lv2_v3.py 
Traceback (most recent call last):
  File "/net/people/plgrid/plgkongruencj/2022-krakow-lgad/src/ares/submit_lv2_v3.py", line 9, in <module>
    client = Client(hq_server_path)
             ^^^^^^^^^^^^^^^^^^^^^^
  File "/memfs/7515829/venv/lib/python3.11/site-packages/hyperqueue/client.py", line 69, in __init__
    self.connection = ClientConnection(server_dir)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/memfs/7515829/venv/lib/python3.11/site-packages/hyperqueue/ffi/client.py", line 28, in __init__
    self.ctx: HqClientContext = ffi.connect_to_server(directory)
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Hyperqueue version mismatch detected.
Access record contains version v0.17.0-liberec, but the current version is v0.17.0.

Here is my python code:

from hyperqueue import Client, Job
from pathlib import Path

hq_server_path = Path.home() / ".hq-server/hq-current"

def test_fun():
    print("Hello World!")

client = Client(hq_server_path)

job = Job(test_fun)

client.submit(job)

Do you have a working combination of server with dashboard and python API ? I wasn't able to figureout how to install v0.17.0-liberec python package.