SOLVED celery wont run in docker (windows 10)

[renschni]

⚠️ Check for existing issues before proceeding. ⚠️

• [X] I have searched the existing issues, and there is no existing issue for my problem

Where are you using SuperAGI?

Windows

Which branch of SuperAGI are you using?

Main

Do you use OpenAI GPT-3.5 or GPT-4?

GPT-3.5

Which area covers your issue best?

Installation and setup

Describe your issue.

Installation with the latest git (on windows 10) worked fine, docker composing worked fine, but celery exits right away, therefore SuperAgi wont run. Here is the docker log for the celery image:

2023-07-07 00:11:29 Usage: celery [OPTIONS] COMMAND [ARGS]... 2023-07-07 00:11:29 Try 'celery --help' for help. 2023-07-07 00:11:29 2023-07-07 00:11:29 Error: Invalid value for '-A' / '--app': 2023-07-07 00:11:29 Unable to load celery application. 2023-07-07 00:11:29 While trying to load the module superagi.worker the following error occurred: 2023-07-07 00:11:29 Traceback (most recent call last): 2023-07-07 00:11:29 File "/usr/local/lib/python3.9/site-packages/celery/bin/celery.py", line 57, in convert 2023-07-07 00:11:29 return find_app(value) 2023-07-07 00:11:29 File "/usr/local/lib/python3.9/site-packages/celery/app/utils.py", line 384, in find_app 2023-07-07 00:11:29 sym = symbol_by_name(app, imp=imp) 2023-07-07 00:11:29 File "/usr/local/lib/python3.9/site-packages/kombu/utils/imports.py", line 56, in symbol_by_name 2023-07-07 00:11:29 module = imp(module_name, package=package, **kwargs) 2023-07-07 00:11:29 File "/usr/local/lib/python3.9/site-packages/celery/utils/imports.py", line 105, in import_from_cwd 2023-07-07 00:11:29 return imp(module, package=package) 2023-07-07 00:11:29 File "/usr/local/lib/python3.9/importlib/init.py", line 127, in import_module 2023-07-07 00:11:29 return _bootstrap._gcd_import(name[level:], package, level) 2023-07-07 00:11:29 File "", line 1030, in _gcd_import 2023-07-07 00:11:29 File "", line 1007, in _find_and_load 2023-07-07 00:11:29 File "", line 984, in _find_and_load_unlocked 2023-07-07 00:11:29 ModuleNotFoundError: No module named 'superagi' 2023-07-07 00:11:29

everything else runs in docker just fine. Any ideas what exactly causes the problem? Any fixes possible? Thx for your help!

How to replicate your Issue?

No response

Upload Error Log Content

ModuleNotFoundError: No module named 'superagi'

[neelayan7]

Are you running docker from the root directory?

[iguy0]

I'm experiencing a similar issue with Celery as well, trying to deploy using docker on Ubuntu Linux.

Steps: cd SuperAGI/ nano config.yaml docker compose up --build -d

and i have this for erros in the container: Usage: celery [OPTIONS] COMMAND [ARGS]... Try 'celery --help' for help. Error: Invalid value for '-A' / '--app': Unable to load celery application. While trying to load the module superagi.worker the following error occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/celery/app/utils.py", line 384, in find_app sym = symbol_by_name(app, imp=imp) File "/usr/local/lib/python3.9/site-packages/kombu/utils/imports.py", line 61, in symbol_by_name return getattr(module, cls_name) if cls_name else module AttributeError: module 'superagi' has no attribute 'worker' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/celery/bin/celery.py", line 57, in convert return find_app(value) File "/usr/local/lib/python3.9/site-packages/celery/app/utils.py", line 387, in find_app sym = imp(app) File "/usr/local/lib/python3.9/site-packages/celery/utils/imports.py", line 105, in import_from_cwd return imp(module, package=package) File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1030, in _gcd_import File "<frozen importlib._bootstrap>", line 1007, in _find_and_load File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 680, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 850, in exec_module File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed File "/app/superagi/worker.py", line 5, in <module> from superagi.helper.tool_helper import handle_tools_import File "/app/superagi/helper/tool_helper.py", line 11, in <module> from superagi.config.config import get_config File "/app/superagi/config/config.py", line 47, in <module> _config_instance = Config(ROOT_DIR + "/" + CONFIG_FILE) File "/app/superagi/config/config.py", line 39, in __init__ config_data = self.load_config(config_file) File "/app/superagi/config/config.py", line 20, in load_config config_data = yaml.safe_load(file) File "/usr/local/lib/python3.9/site-packages/yaml/__init__.py", line 125, in safe_load return load(stream, SafeLoader) File "/usr/local/lib/python3.9/site-packages/yaml/__init__.py", line 81, in load return loader.get_single_data() File "/usr/local/lib/python3.9/site-packages/yaml/constructor.py", line 49, in get_single_data node = self.get_single_node() File "/usr/local/lib/python3.9/site-packages/yaml/composer.py", line 36, in get_single_node document = self.compose_document() File "/usr/local/lib/python3.9/site-packages/yaml/composer.py", line 55, in compose_document node = self.compose_node(None, None) File "/usr/local/lib/python3.9/site-packages/yaml/composer.py", line 84, in compose_node node = self.compose_mapping_node(anchor) File "/usr/local/lib/python3.9/site-packages/yaml/composer.py", line 127, in compose_mapping_node while not self.check_event(MappingEndEvent): File "/usr/local/lib/python3.9/site-packages/yaml/parser.py", line 98, in check_event self.current_event = self.state() File "/usr/local/lib/python3.9/site-packages/yaml/parser.py", line 428, in parse_block_mapping_key if self.check_token(KeyToken): File "/usr/local/lib/python3.9/site-packages/yaml/scanner.py", line 115, in check_token while self.need_more_tokens(): File "/usr/local/lib/python3.9/site-packages/yaml/scanner.py", line 152, in need_more_tokens self.stale_possible_simple_keys() File "/usr/local/lib/python3.9/site-packages/yaml/scanner.py", line 291, in stale_possible_simple_keys raise ScannerError("while scanning a simple key", key.mark, yaml.scanner.ScannerError: while scanning a simple key in "/app/config.yaml", line 11, column 1 could not find expected ':' in "/app/config.yaml", line 13, column 1

[renschni]

Are you running docker from the root directory?

started the docker app then running the build comand in terminal from the superAGI install directory. But again, everything else builds and runs just fine, no errors, no path issues. config.yaml in place of course. It's just the celery module that wont run. I'm not familiar with celery but trying to find informations and solution I found that celery does not 100% support windows after version 4. I asume the development machine for SuperAGI is linux and the docker files work just fine when deployed on linux clients and thus the error cant be replicated on linux machines. I think a good start would be to identify this problem as a windows specific one. My machine is rock solid and even oobabooga runs flawless in a conda environment, heck, I run 30b models with 8k token window locally that exceed vram without OOM issues. Just mentioning this to stress that I try to run SuperAGI on a dedicated windows AI workstation that is proper set up.

[dicksensei69]

Make sure that you check that you have copied the config_template.yaml to config.yaml in your main superagi dir. Thats what had me hung up for a second. I missed it in the instructions. It looks like an error with celery when it happens. I think it's installing correctly now and I hope that is all that is wrong with your guy's installs.

[renschni]

Make sure that you check that you have copied the config_template.yaml to config.yaml in your main superagi dir.

Of course I did, as I mentioned earlier. And look at the specifics of the problem: All archives/modules that get started in docker after the build run smoothly. Except celery that exits right away. The others keep running. Maybe there is something in the config.yaml that needs to get edited to solve the problem/bug, maybe it's just a path problem, but I don't know. All I do know is that celery doesn't run and I have no means to debug that. So, any constructive ideas, anybody?

[renschni]

SOLVED Assuming a path problem I tried out some install variations and found one that works! creating a SuperAGI directory in root and installing the files there leads to the problem as described. creating another SuperAGI directory within the SuperAGI directory and installing from there works! I'd like to stress once more that in both casses all archives (except celery) run in docker without problems, but only celery joins the club when being away from root one more directory level. Computers, what fun...