Celery worker: No module named 'schedule.clean_embedding_cache_task'

Sakura4036 commented 1 day ago

Self Checks

[X] This is only for bug report, if you would like to ask a question, please head to Discussions.
[X] I have searched for existing issues search for existing issues, including closed ones.
[X] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
[X] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
[X] Please do not modify this template :) and fill in all the required fields.

Dify version

0.9.2

Cloud or Self Hosted

Self Hosted (Source)

Steps to reproduce

update to dify v0.9.2

cd dify/api
celery -A app.celery worker -P gevent -c 1 -Q dataset,generation,mail --loglevel INFO

✔️ Expected Behavior

celery working

❌ Actual Behavior

celery -A app.celery worker -P gevent -c 1 -Q dataset,generation,mail --loglevel INFO
None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.
sagemaker.config INFO - Not applying SDK defaults from location: /etc/xdg/sagemaker/config.yaml
sagemaker.config INFO - Not applying SDK defaults from location: /home/.config/sagemaker/config.yaml
Fontconfig error: Cannot load default config file: No such file: (null)
Traceback (most recent call last):
  File "/home/anaconda3/envs/dify/bin/celery", line 8, in <module>
    sys.exit(main())
  File "/home/anaconda3/envs/dify/lib/python3.10/site-packages/celery/__main__.py", line 15, in main
    sys.exit(_main())
  File "/home/anaconda3/envs/dify/lib/python3.10/site-packages/celery/bin/celery.py", line 236, in main
    return celery(auto_envvar_prefix="CELERY")
  File "/home/anaconda3/envs/dify/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/home/anaconda3/envs/dify/lib/python3.10/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/home/anaconda3/envs/dify/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/anaconda3/envs/dify/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/anaconda3/envs/dify/lib/python3.10/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/home/anaconda3/envs/dify/lib/python3.10/site-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/anaconda3/envs/dify/lib/python3.10/site-packages/celery/bin/base.py", line 134, in caller
    return f(ctx, *args, **kwargs)
  File "/home/anaconda3/envs/dify/lib/python3.10/site-packages/celery/bin/worker.py", line 348, in worker
    worker = app.Worker(
  File "/home/anaconda3/envs/dify/lib/python3.10/site-packages/celery/worker/worker.py", line 93, in __init__
    self.app.loader.init_worker()
  File "/home/anaconda3/envs/dify/lib/python3.10/site-packages/celery/loaders/base.py", line 110, in init_worker
    self.import_default_modules()
  File "/home/anaconda3/envs/dify/lib/python3.10/site-packages/celery/loaders/base.py", line 105, in import_default_modules
    return [self.import_task_module(m) for m in self.default_modules]
  File "/home/anaconda3/envs/dify/lib/python3.10/site-packages/celery/loaders/base.py", line 105, in <listcomp>
    return [self.import_task_module(m) for m in self.default_modules]
  File "/home/anaconda3/envs/dify/lib/python3.10/site-packages/celery/loaders/base.py", line 85, in import_task_module
    return self.import_from_cwd(module)
  File "/home/anaconda3/envs/dify/lib/python3.10/site-packages/celery/loaders/base.py", line 91, in import_from_cwd
    return import_from_cwd(
  File "/home/anaconda3/envs/dify/lib/python3.10/site-packages/celery/utils/imports.py", line 109, in import_from_cwd
    return imp(module, package=package)
  File "/home/anaconda3/envs/dify/lib/python3.10/site-packages/celery/loaders/base.py", line 88, in import_module
    return importlib.import_module(module, package=package)
  File "/home/anaconda3/envs/dify/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1004, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'schedule.clean_embedding_cache_task'

dosubot[bot] commented 1 day ago

To resolve the "No module named 'schedule.clean_embedding_cache_task'" error when running a Celery worker in Dify version 0.9.2, follow these steps:

Verify Module Existence: Ensure that the schedule directory exists and contains a file named clean_embedding_cache_task.py. This file should define the task clean_embedding_cache_task ^[1].
Check Python Path: Make sure that the directory containing the schedule module is included in your Python path. You can check this by printing sys.path in your script or by setting the PYTHONPATH environment variable.
Correct Import Path: If the module is located in a different directory, adjust the import path accordingly in the Celery configuration.
Install Required Packages: If schedule is part of an external package, ensure it is installed in your environment.

These steps should help you resolve the import error and get your Celery worker running correctly.

To continue talking to Dosu, mention @dosu.

crazywoola commented 1 day ago

I can not reproduce this on latest branch.

langgenius / dify