search

celery / celery

Distributed Task Queue (development branch)
https://docs.celeryq.dev
Other
24.64k stars 4.65k forks source link

Can't pickle local object 'Pool.__init__.<locals>.Process' #3559

Closed cemremengu closed 6 years ago

cemremengu commented 7 years ago

My configuration is as follows (running as administrator)

software -> celery:4.0.0 (latentcall) kombu:4.0.0 py:3.5.2
            billiard:3.5.0.2 py-amqp:2.1.1
platform -> system:Windows arch:32bit, WindowsPE imp:CPython
loader   -> celery.loaders.app.AppLoader
settings -> transport:amqp results:disabled

broker_url: 'amqp://guest:********@localhost:5672//'

Steps to reproduce (tasks.py)

from celery import Celery

app = Celery('tasks', broker='amqp://guest:guest@localhost:5672//')

@app.task(ignore_result=True, serializer='json')
def print_hello():
    print('hello there')

@app.task(serializer='json')
def gen_prime(x):
    multiples = []
    results = []
    for i in xrange(2, x+1):
        if i not in multiples:
            results.append(i)
            for j in xrange(i*i, x+1, i):
                multiples.append(j)
    return results

celery worker -A tasks

Expected behavior

Worker running successfully

Actual behavior

 -------------- celery@CMENGU-PC v4.0.0 (latentcall)
---- **** -----
--- * ***  * -- Windows-7-6.1.7601-SP1 2016-11-07 23:25:16
-- * - **** ---
- ** ---------- [config]
- ** ---------- .> app:         tasks:0x34f0790
- ** ---------- .> transport:   amqp://guest:**@localhost:5672//
- ** ---------- .> results:     disabled://
- *** --- * --- .> concurrency: 8 (prefork)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** -----
 -------------- [queues]
                .> celery           exchange=celery(direct) key=celery

[2016-11-07 23:25:16,511: CRITICAL/MainProcess] Unrecoverable error: AttributeError("Can't pickle loca
l object 'Pool.__init__.<locals>.Process'",)
Traceback (most recent call last):
  File "c:\users\cmengu\appdata\local\programs\python\python35-32\lib\site-packages\celery\worker\work
er.py", line 203, in start
    self.blueprint.start(self)
  File "c:\users\cmengu\appdata\local\programs\python\python35-32\lib\site-packages\celery\bootsteps.p
y", line 119, in start
    step.start(parent)
  File "c:\users\cmengu\appdata\local\programs\python\python35-32\lib\site-packages\celery\bootsteps.p
y", line 370, in start
    return self.obj.start()
  File "c:\users\cmengu\appdata\local\programs\python\python35-32\lib\site-packages\celery\concurrency
\base.py", line 131, in start
    self.on_start()
  File "c:\users\cmengu\appdata\local\programs\python\python35-32\lib\site-packages\celery\concurrency
\prefork.py", line 112, in on_start
    **self.options)
  File "c:\users\cmengu\appdata\local\programs\python\python35-32\lib\site-packages\billiard\pool.py",
 line 1008, in __init__
    self._create_worker_process(i)
  File "c:\users\cmengu\appdata\local\programs\python\python35-32\lib\site-packages\billiard\pool.py",
 line 1117, in _create_worker_process
    w.start()
  File "c:\users\cmengu\appdata\local\programs\python\python35-32\lib\site-packages\billiard\process.p
y", line 122, in start
    self._popen = self._Popen(self)
  File "c:\users\cmengu\appdata\local\programs\python\python35-32\lib\site-packages\billiard\context.p
y", line 383, in _Popen
    return Popen(process_obj)
  File "c:\users\cmengu\appdata\local\programs\python\python35-32\lib\site-packages\billiard\popen_spa
wn_win32.py", line 79, in __init__
    reduction.dump(process_obj, to_child)
  File "c:\users\cmengu\appdata\local\programs\python\python35-32\lib\site-packages\billiard\reduction
.py", line 99, in dump
    ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'Pool.__init__.<locals>.Process'

C:\Users\CMENGU\Desktop\celery-app\messaging>Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "c:\users\cmengu\appdata\local\programs\python\python35-32\lib\site-packages\billiard\spawn.py"
, line 159, in spawn_main
    new_handle = steal_handle(parent_pid, pipe_handle)
  File "c:\users\cmengu\appdata\local\programs\python\python35-32\lib\site-packages\billiard\reduction
.py", line 126, in steal_handle
    _winapi.DUPLICATE_SAME_ACCESS | _winapi.DUPLICATE_CLOSE_SOURCE)
PermissionError: [WinError 5] Access is denied

It says a permission error but I am running on admin privileges. Not really sure whats going on.

artemistomaras commented 7 years ago

I am having the exact error as well with almost identical setup as @cemremengu

gangofmumbai commented 7 years ago

I did face same issue and it seems that Celery Version mismatch on workers of same cluster is causing the problem. When I tried to add new worker to the cluster I simply installed latest celery from pip without realizing that my existing workers are on older celery version (3.1.20). After installing correct version on new worker I was able to start task without any problem.

debnet commented 7 years ago

Same error here.

(nouvelleoffre) C:\Git\nouvelleoffre-back>celery worker -A nouvelleoffre -l DEBUG -E
[2017-01-02 14:48:04,422: DEBUG/MainProcess] | Worker: Preparing bootsteps.
[2017-01-02 14:48:04,424: DEBUG/MainProcess] | Worker: Building graph...
[2017-01-02 14:48:04,424: DEBUG/MainProcess] | Worker: New boot order: {Timer, Hub, Pool, Autoscaler, Beat, StateDB, Consumer}
[2017-01-02 14:48:04,446: DEBUG/MainProcess] | Consumer: Preparing bootsteps.
[2017-01-02 14:48:04,447: DEBUG/MainProcess] | Consumer: Building graph...
[2017-01-02 14:48:04,468: DEBUG/MainProcess] | Consumer: New boot order: {Connection, Events, Mingle, Tasks, Control, Agent, Gossip, Heart, event loop}

 -------------- celery@DELL-MARC-D-P v4.0.2 (latentcall)
---- **** -----
--- * ***  * -- Windows-8.1-6.3.9600-SP0 2017-01-02 14:48:04
-- * - **** ---
- ** ---------- [config]
- ** ---------- .> app:         nouvelleoffre:0x3a6391160
- ** ---------- .> transport:   redis://localhost:6379/0
- ** ---------- .> results:
- *** --- * --- .> concurrency: 4 (prefork)
-- ******* ---- .> task events: ON
--- ***** -----
 -------------- [queues]
                .> celery           exchange=celery(direct) key=celery

[tasks]
  . celery.accumulate
  . celery.backend_cleanup
  . celery.chain
  . celery.chord
  . celery.chord_unlock
  . celery.chunks
  . celery.group
  . celery.map
  . celery.starmap
  . common.log_delete
  . common.log_m2m
  . common.log_save
  . common.notify_changes
  . crm.creer_fichier_crm
  . crm.creer_lot
  . crm.export
  . crm.generer
  . crm.import
  . dsn.annulation
  . dsn.consolidation
  . dsn.historiques
  . dsn.integration
  . dsn.volumetrie
  . fdp.export
  . fdp.exporter_lot_fdp
  . fdp.generer_fdp
  . fdp.import
  . fdp.import_lot
  . fdp.import_tous_types
  . gpj.notifier_changements
  . gpj.relancer_dossier
  . merge
  . query.execute
  . tarificateur.calcul

[2017-01-02 14:48:04,491: DEBUG/MainProcess] | Worker: Starting Pool
[2017-01-02 14:48:04,511: CRITICAL/MainProcess] Unrecoverable error: AttributeError("Can't pickle local object 'Pool.__i
nit__.<locals>.Process'",)
Traceback (most recent call last):
  File "c:\virtualenvs\nouvelleoffre\lib\site-packages\celery\worker\worker.py", line 203, in start
    self.blueprint.start(self)
  File "c:\virtualenvs\nouvelleoffre\lib\site-packages\celery\bootsteps.py", line 119, in start
    step.start(parent)
  File "c:\virtualenvs\nouvelleoffre\lib\site-packages\celery\bootsteps.py", line 370, in start
    return self.obj.start()
  File "c:\virtualenvs\nouvelleoffre\lib\site-packages\celery\concurrency\base.py", line 131, in start
    self.on_start()
  File "c:\virtualenvs\nouvelleoffre\lib\site-packages\celery\concurrency\prefork.py", line 112, in on_start
    **self.options)
  File "c:\virtualenvs\nouvelleoffre\lib\site-packages\billiard\pool.py", line 1008, in __init__
    self._create_worker_process(i)
  File "c:\virtualenvs\nouvelleoffre\lib\site-packages\billiard\pool.py", line 1117, in _create_worker_process
    w.start()
  File "c:\virtualenvs\nouvelleoffre\lib\site-packages\billiard\process.py", line 122, in start
    self._popen = self._Popen(self)
  File "c:\virtualenvs\nouvelleoffre\lib\site-packages\billiard\context.py", line 383, in _Popen
    return Popen(process_obj)
  File "c:\virtualenvs\nouvelleoffre\lib\site-packages\billiard\popen_spawn_win32.py", line 79, in __init__
    reduction.dump(process_obj, to_child)
  File "c:\virtualenvs\nouvelleoffre\lib\site-packages\billiard\reduction.py", line 99, in dump
    ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'Pool.__init__.<locals>.Process'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "c:\virtualenvs\nouvelleoffre\lib\site-packages\billiard\spawn.py", line 165, in spawn_main
    exitcode = _main(fd)
  File "c:\virtualenvs\nouvelleoffre\lib\site-packages\billiard\spawn.py", line 207, in _main
    self = pickle.load(from_parent)
EOFError: Ran out of input
auvipy commented 6 years ago

is the bug still live in master?

auvipy commented 6 years ago

hope this is fixed in master.