search

AutoMQ / automq

AutoMQ is a cloud-first alternative to Kafka by decoupling durability to S3 and EBS. 10x Cost-Effective. No Cross-AZ Traffic Cost. Autoscale in seconds. Single-digit ms latency.
https://www.automq.com/docs
Other
3.81k stars 220 forks source link

[BUG] The localstack container can't be started when I install the automq with install_run.sh #1349

Open xuqingkang opened 5 months ago

xuqingkang commented 5 months ago

Version & Environment

What went wrong?

I can't install the automq in my vmware's virtual host。

What should have happened instead?

How to reproduce the issue?

  1. download the install_run.sh with wget ...，and call the shell to install automq，but the localstack container can't be started. wget https://download.automq.com/community_edition/standalone_deployment/install_run.sh 
    [root@rhel75-170 automq]# ./install_run.sh 
Welcome to the installation of AutoMQ, A Cloud-Native fork of Kafka by separating storage to S3.
If you encounter any problems, please report them at https://github.com/AutoMQ/automq/issues.
[WARN] Directory automq-standalone-deployment already exists.
% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                             Dload  Upload   Total   Spent    Left  Speed
100  5158  100  5158    0     0   6350      0 --:--:-- --:--:-- --:--:--  6352
Please enter your password for sudo:
WARN[0000] /tmp/automq-standalone-deployment/docker-compose.yaml: `version` is obsolete 
[+] Running 1/1
✘ Container localstack  Error                                                                                                                                                                                      13.9s 
dependency failed to start: container localstack exited (1)
  2. the error log as following， How can i fix it? 
    [root@rhel75-170 automq]# docker logs -t d3fb1bcb0e34
2024-06-04T00:28:47.809155851Z 
2024-06-04T00:28:47.809245524Z LocalStack version: 3.3.0
2024-06-04T00:28:47.819012573Z LocalStack Docker container id: d3fb1bcb0e34
2024-06-04T00:28:47.819063932Z LocalStack build date: 2024-03-28
2024-06-04T00:28:47.819071963Z LocalStack build git hash: 5567b0b3c
2024-06-04T00:28:47.819077694Z 
2024-06-04T00:28:48.821629892Z 2024-06-04T00:28:48.810  WARN --- [  MainThread] localstack.dns.plugins     : Unable to start DNS: can't start new thread
2024-06-04T00:28:48.821673212Z Unexpected exception while starting infrastructure: can't start new thread Traceback (most recent call last):
2024-06-04T00:28:48.821678849Z   File "/opt/code/localstack/localstack/services/infra.py", line 247, in start_infra
2024-06-04T00:28:48.821683780Z     hooks.on_infra_start.run()
2024-06-04T00:28:48.821688113Z   File "/opt/code/localstack/localstack/runtime/hooks.py", line 64, in run_in_order
2024-06-04T00:28:48.821692459Z     fn_plugin(*args, **kwargs)
2024-06-04T00:28:48.821696301Z   File "/opt/code/localstack/.venv/lib/python3.11/site-packages/plux/core/plugin.py", line 226, in __call__
2024-06-04T00:28:48.821700530Z     return self.fn(*args, **kwargs)
2024-06-04T00:28:48.821704519Z            ^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-04T00:28:48.821708358Z   File "/opt/code/localstack/localstack/runtime/analytics.py", line 86, in _publish_config_as_analytics_event
2024-06-04T00:28:48.821712615Z     log.event("config", env_vars=env_vars, set_vars=present_env_vars)
2024-06-04T00:28:48.821717166Z   File "/opt/code/localstack/localstack/utils/analytics/logger.py", line 39, in event
2024-06-04T00:28:48.821721249Z     self._log(event, payload=payload)
2024-06-04T00:28:48.821724861Z   File "/opt/code/localstack/localstack/utils/analytics/logger.py", line 42, in _log
2024-06-04T00:28:48.821729377Z     self.handler.handle(Event(name=event, metadata=self._metadata(), payload=payload))
2024-06-04T00:28:48.821733210Z   File "/opt/code/localstack/localstack/utils/analytics/publisher.py", line 235, in handle
2024-06-04T00:28:48.821737394Z     self._start()
2024-06-04T00:28:48.821740931Z   File "/opt/code/localstack/localstack/utils/analytics/publisher.py", line 250, in _start
2024-06-04T00:28:48.821744945Z     start_worker_thread(self._do_start_retry)
2024-06-04T00:28:48.821748798Z   File "/opt/code/localstack/localstack/utils/threads.py", line 111, in start_worker_thread
2024-06-04T00:28:48.821753045Z     return start_thread(method, *args, _shutdown_hook=False, **kwargs)
2024-06-04T00:28:48.821756804Z            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-04T00:28:48.821760604Z   File "/opt/code/localstack/localstack/utils/threads.py", line 103, in start_thread
2024-06-04T00:28:48.821764614Z     thread.start()
2024-06-04T00:28:48.821768194Z   File "/usr/local/lib/python3.11/threading.py", line 964, in start
2024-06-04T00:28:48.821789656Z     _start_new_thread(self._bootstrap, ())
2024-06-04T00:28:48.821794241Z RuntimeError: can't start new thread
2024-06-04T00:28:48.821798005Z 
2024-06-04T00:28:52.362081169Z 2024-06-04T00:28:52.355  INFO --- [  MainThread] l.runtime.shutdown         : [shutdown] Stopping all services
2024-06-04T00:28:52.362130882Z 2024-06-04T00:28:52.360  WARN --- [  MainThread] localstack.dns.server      : resolv.conf file to restore not found.
2024-06-04T00:28:52.384911544Z Traceback (most recent call last):
2024-06-04T00:28:52.384964974Z   File "<frozen runpy>", line 198, in _run_module_as_main
2024-06-04T00:28:52.384973896Z   File "<frozen runpy>", line 88, in _run_code
2024-06-04T00:28:52.385218770Z   File "/opt/code/localstack/localstack/runtime/main.py", line 29, in <module>
2024-06-04T00:28:52.393623478Z     main()
2024-06-04T00:28:52.393985464Z   File "/opt/code/localstack/localstack/runtime/main.py", line 21, in main
2024-06-04T00:28:52.394005703Z     infra.start_infra(asynchronous=False)
2024-06-04T00:28:52.394012979Z   File "/opt/code/localstack/localstack/services/infra.py", line 269, in start_infra
2024-06-04T00:28:52.394019546Z     raise e
2024-06-04T00:28:52.394025438Z   File "/opt/code/localstack/localstack/services/infra.py", line 247, in start_infra
2024-06-04T00:28:52.394032389Z     hooks.on_infra_start.run()
2024-06-04T00:28:52.394038203Z   File "/opt/code/localstack/localstack/runtime/hooks.py", line 64, in run_in_order
2024-06-04T00:28:52.394133280Z     fn_plugin(*args, **kwargs)
2024-06-04T00:28:52.394178651Z   File "/opt/code/localstack/.venv/lib/python3.11/site-packages/plux/core/plugin.py", line 226, in __call__
2024-06-04T00:28:52.394187609Z     return self.fn(*args, **kwargs)
2024-06-04T00:28:52.394193938Z            ^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-04T00:28:52.394199961Z   File "/opt/code/localstack/localstack/runtime/analytics.py", line 86, in _publish_config_as_analytics_event
2024-06-04T00:28:52.394206564Z     log.event("config", env_vars=env_vars, set_vars=present_env_vars)
2024-06-04T00:28:52.394212617Z   File "/opt/code/localstack/localstack/utils/analytics/logger.py", line 39, in event
2024-06-04T00:28:52.394218765Z     self._log(event, payload=payload)
2024-06-04T00:28:52.394224345Z   File "/opt/code/localstack/localstack/utils/analytics/logger.py", line 42, in _log
2024-06-04T00:28:52.394230377Z     self.handler.handle(Event(name=event, metadata=self._metadata(), payload=payload))
2024-06-04T00:28:52.394236350Z   File "/opt/code/localstack/localstack/utils/analytics/publisher.py", line 235, in handle
2024-06-04T00:28:52.394242490Z     self._start()
2024-06-04T00:28:52.394248017Z   File "/opt/code/localstack/localstack/utils/analytics/publisher.py", line 250, in _start
2024-06-04T00:28:52.394254102Z     start_worker_thread(self._do_start_retry)
2024-06-04T00:28:52.394287186Z   File "/opt/code/localstack/localstack/utils/threads.py", line 111, in start_worker_thread
2024-06-04T00:28:52.394294439Z     return start_thread(method, *args, _shutdown_hook=False, **kwargs)
2024-06-04T00:28:52.394300626Z            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-04T00:28:52.394306570Z   File "/opt/code/localstack/localstack/utils/threads.py", line 103, in start_thread
2024-06-04T00:28:52.394312551Z     thread.start()
2024-06-04T00:28:52.394318074Z   File "/usr/local/lib/python3.11/threading.py", line 964, in start
2024-06-04T00:28:52.394323931Z     _start_new_thread(self._bootstrap, ())
2024-06-04T00:28:52.394329692Z RuntimeError: can't start new thread
2024-06-04T03:44:36.778924622Z 
2024-06-04T03:44:36.779299351Z LocalStack version: 3.3.0
2024-06-04T03:44:36.793836598Z LocalStack Docker container id: d3fb1bcb0e34
2024-06-04T03:44:36.793885069Z LocalStack build date: 2024-03-28
2024-06-04T03:44:36.793892962Z LocalStack build git hash: 5567b0b3c
2024-06-04T03:44:36.793898864Z 
2024-06-04T03:44:37.792978777Z 2024-06-04T03:44:37.791  WARN --- [  MainThread] localstack.dns.plugins     : Unable to start DNS: can't start new thread
2024-06-04T03:44:37.808574648Z Unexpected exception while starting infrastructure: can't start new thread Traceback (most recent call last):
2024-06-04T03:44:37.808632273Z   File "/opt/code/localstack/localstack/services/infra.py", line 247, in start_infra
2024-06-04T03:44:37.808641762Z     hooks.on_infra_start.run()
2024-06-04T03:44:37.808647946Z   File "/opt/code/localstack/localstack/runtime/hooks.py", line 64, in run_in_order
2024-06-04T03:44:37.808654661Z     fn_plugin(*args, **kwargs)
2024-06-04T03:44:37.808660914Z   File "/opt/code/localstack/.venv/lib/python3.11/site-packages/plux/core/plugin.py", line 226, in __call__
2024-06-04T03:44:37.808667291Z     return self.fn(*args, **kwargs)
2024-06-04T03:44:37.808673326Z            ^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-04T03:44:37.808679828Z   File "/opt/code/localstack/localstack/runtime/analytics.py", line 86, in _publish_config_as_analytics_event
2024-06-04T03:44:37.808686562Z     log.event("config", env_vars=env_vars, set_vars=present_env_vars)
2024-06-04T03:44:37.808692735Z   File "/opt/code/localstack/localstack/utils/analytics/logger.py", line 39, in event
2024-06-04T03:44:37.808699235Z     self._log(event, payload=payload)
2024-06-04T03:44:37.808705078Z   File "/opt/code/localstack/localstack/utils/analytics/logger.py", line 42, in _log
2024-06-04T03:44:37.808711215Z     self.handler.handle(Event(name=event, metadata=self._metadata(), payload=payload))
2024-06-04T03:44:37.808717103Z   File "/opt/code/localstack/localstack/utils/analytics/publisher.py", line 235, in handle
2024-06-04T03:44:37.808723596Z     self._start()
2024-06-04T03:44:37.808729418Z   File "/opt/code/localstack/localstack/utils/analytics/publisher.py", line 250, in _start
2024-06-04T03:44:37.808891432Z     start_worker_thread(self._do_start_retry)
2024-06-04T03:44:37.808935070Z   File "/opt/code/localstack/localstack/utils/threads.py", line 111, in start_worker_thread
2024-06-04T03:44:37.808943619Z     return start_thread(method, *args, _shutdown_hook=False, **kwargs)
2024-06-04T03:44:37.808949101Z            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-04T03:44:37.808954472Z   File "/opt/code/localstack/localstack/utils/threads.py", line 103, in start_thread
2024-06-04T03:44:37.808960466Z     thread.start()
2024-06-04T03:44:37.808965776Z   File "/usr/local/lib/python3.11/threading.py", line 964, in start
2024-06-04T03:44:37.808972041Z     _start_new_thread(self._bootstrap, ())
2024-06-04T03:44:37.808977829Z RuntimeError: can't start new thread
2024-06-04T03:44:37.808983477Z 
2024-06-04T03:44:43.319681664Z 2024-06-04T03:44:43.318  INFO --- [  MainThread] l.runtime.shutdown         : [shutdown] Stopping all services
2024-06-04T03:44:43.332122596Z 2024-06-04T03:44:43.331  WARN --- [  MainThread] localstack.dns.server      : resolv.conf file to restore not found.
2024-06-04T03:44:43.352322837Z Traceback (most recent call last):
2024-06-04T03:44:43.352363529Z   File "<frozen runpy>", line 198, in _run_module_as_main
2024-06-04T03:44:43.352372826Z   File "<frozen runpy>", line 88, in _run_code
2024-06-04T03:44:43.352378953Z   File "/opt/code/localstack/localstack/runtime/main.py", line 29, in <module>
2024-06-04T03:44:43.357473354Z     main()
2024-06-04T03:44:43.357503368Z   File "/opt/code/localstack/localstack/runtime/main.py", line 21, in main
2024-06-04T03:44:43.357629369Z     infra.start_infra(asynchronous=False)
2024-06-04T03:44:43.357771026Z   File "/opt/code/localstack/localstack/services/infra.py", line 269, in start_infra
2024-06-04T03:44:43.358047529Z     raise e
2024-06-04T03:44:43.358130580Z   File "/opt/code/localstack/localstack/services/infra.py", line 247, in start_infra
2024-06-04T03:44:43.358309883Z     hooks.on_infra_start.run()
2024-06-04T03:44:43.358373025Z   File "/opt/code/localstack/localstack/runtime/hooks.py", line 64, in run_in_order
2024-06-04T03:44:43.358531497Z     fn_plugin(*args, **kwargs)
2024-06-04T03:44:43.358541239Z   File "/opt/code/localstack/.venv/lib/python3.11/site-packages/plux/core/plugin.py", line 226, in __call__
2024-06-04T03:44:43.358884202Z     return self.fn(*args, **kwargs)
2024-06-04T03:44:43.358896059Z            ^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-04T03:44:43.358902280Z   File "/opt/code/localstack/localstack/runtime/analytics.py", line 86, in _publish_config_as_analytics_event
2024-06-04T03:44:43.359213197Z     log.event("config", env_vars=env_vars, set_vars=present_env_vars)
2024-06-04T03:44:43.359292757Z   File "/opt/code/localstack/localstack/utils/analytics/logger.py", line 39, in event
2024-06-04T03:44:43.359451851Z     self._log(event, payload=payload)
2024-06-04T03:44:43.359462594Z   File "/opt/code/localstack/localstack/utils/analytics/logger.py", line 42, in _log
2024-06-04T03:44:43.359845176Z     self.handler.handle(Event(name=event, metadata=self._metadata(), payload=payload))
2024-06-04T03:44:43.359858489Z   File "/opt/code/localstack/localstack/utils/analytics/publisher.py", line 235, in handle
2024-06-04T03:44:43.360059491Z     self._start()
2024-06-04T03:44:43.360069939Z   File "/opt/code/localstack/localstack/utils/analytics/publisher.py", line 250, in _start
2024-06-04T03:44:43.360403231Z     start_worker_thread(self._do_start_retry)
2024-06-04T03:44:43.360417478Z   File "/opt/code/localstack/localstack/utils/threads.py", line 111, in start_worker_thread
2024-06-04T03:44:43.360785706Z     return start_thread(method, *args, _shutdown_hook=False, **kwargs)
2024-06-04T03:44:43.360796117Z            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-04T03:44:43.360858564Z   File "/opt/code/localstack/localstack/utils/threads.py", line 103, in start_thread
2024-06-04T03:44:43.361012151Z     thread.start()
2024-06-04T03:44:43.361021708Z   File "/usr/local/lib/python3.11/threading.py", line 964, in start
2024-06-04T03:44:43.361559199Z     _start_new_thread(self._bootstrap, ())
2024-06-04T03:44:43.361584205Z RuntimeError: can't start new thread

Additional information

My local machine infomation as following 1、The Virtual Machine installed by vmware use the NAT network mode. 2、Docker is the 18.09， is this version too low?

[root@rhel75-170 ~]# cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 7.5 (Maipo)
[root@rhel75-170 ~]# 
[root@rhel75-170 ~]# 
[root@rhel75-170 ~]# uname -a
Linux rhel75-170 3.10.0-862.el7.x86_64 #1 SMP Wed Mar 21 18:14:51 EDT 2018 x86_64 x86_64 x86_64 GNU/Linux
[root@rhel75-170 ~]# 
[root@rhel75-170 ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.32.172 rhel75-172
192.168.32.171 rhel75-171
192.168.32.170 rhel75-170
[root@rhel75-170 ~]# 
[root@rhel75-170 ~]# 
[root@rhel75-170 ~]# hostname
rhel75-170
[root@rhel75-170 ~]# 
[root@rhel75-170 ~]# docker -v
Docker version 18.09.6, build 481bc77156
[root@rhel75-170 ~]# 
[root@rhel75-170 ~]# 
[root@rhel75-170 ~]# 
[root@rhel75-170 ~]# docker-compose -v
Docker Compose version v2.27.0

Please attach any relevant logs, backtraces, or metric charts.

KaimingWan commented 4 months ago

It seems that the LocalStack container did not start properly. We haven't tested it on VMware virtual machines. You can try running it directly on your Windows, Mac, or Linux machine, provided that Docker and Docker Compose are installed. Docker itself is cross-platform, so you don't need to deploy AutoMQ on VMware. If you encounter any further issues, feel free to leave a message.