marsupialtail / quokka

Making data lake work for time series
https://marsupialtail.github.io/quokka/
Apache License 2.0
1.1k stars 60 forks source link

Unable to use redis running in docker #57

Closed gkaretka closed 1 year ago

gkaretka commented 1 year ago

Description

Currently it is not possible to run redis server for local development in docker:

Initializing local Quokka cluster.
2023-06-08 08:01:39,652 INFO worker.py:1625 -- Started a local Ray instance.
Traceback (most recent call last):
  File "/Users/x/Documents/Projects/quokka-test/main.py", line 8, in <module>
    local_cluster = LocalCluster()
  File "/Users/x/Documents/Projects/quokka-test/venv/lib/python3.10/site-packages/pyquokka/utils.py", line 148, in __init__
    self.redis_process = subprocess.Popen(["redis-server" , pyquokka_loc + "redis.conf", "--port 6800", "--protected-mode no"], preexec_fn=preexec_function)
  File "/Users/x/.pyenv/versions/3.10.5/lib/python3.10/subprocess.py", line 969, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/Users/x/.pyenv/versions/3.10.5/lib/python3.10/subprocess.py", line 1845, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'redis-server'

Solution

Add optional **kwargs to LocalCluster, so it is backward compatible.

options = {
    'docker_redis_enabled': True,
}

local_cluster = LocalCluster(**options)
qc = QuokkaContext(local_cluster)
marsupialtail commented 1 year ago

@gkaretka sorry about this -- just merged your PR. I really need to set up email notifications from this Github.