UnicodeEncodeError when using send_task() with message containing german umlauts

[fgmehlin]

It seems sending a task with german umlauts triggers a : UnicodeEncodeError: 'ascii' codec can't encode character '\xe4' in position xx: ordinal not in range(128)

Working example :

# Note the 'o', replacing 'note'
extract_tasks = [(arg1, note_id, 'o') for note_id, note in
                         enumerate(some_dict[arg1])]

results = [extractor_app.send_task('extractor.app.extract', x,
                queue='extractor', exchange='extractor',
                key='extractor') for x in extract_tasks]

Failing example :

# Note the 'ö', replacing 'note'
extract_tasks = [(arg1, note_id, 'ö') for note_id, note in
                         enumerate(some_dict[arg1])]

results = [extractor_app.send_task('extractor.app.extract', x,
                queue='extractor', exchange='extractor',
                key='extractor') for x in extract_tasks]

triggers :

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/local/lib/python3.6/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/src/app/patient_processing/patient_process_orchestrator.py", line 67, in preprocess_patient
    patient_concepts_dict = self._pool_extract(extract_tasks)
  File "/usr/src/app/patient_processing/patient_process_orchestrator.py", line 236, in _pool_extract
    key='extractor') for x in extract_tasks]
  File "/usr/src/app/patient_processing/patient_process_orchestrator.py", line 236, in <listcomp>
    key='extractor') for x in extract_tasks]
  File "/usr/local/lib/python3.6/site-packages/celery/app/base.py", line 737, in send_task
    amqp.send_task_message(P, name, message, **options)
  File "/usr/local/lib/python3.6/site-packages/celery/app/amqp.py", line 554, in send_task_message
    **properties
  File "/usr/local/lib/python3.6/site-packages/kombu/messaging.py", line 181, in publish
    exchange_name, declare,
  File "/usr/local/lib/python3.6/site-packages/kombu/connection.py", line 494, in _ensured
    return fun(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/kombu/messaging.py", line 203, in _publish
    mandatory=mandatory, immediate=immediate,
  File "/usr/local/lib/python3.6/site-packages/librabbitmq-2.0.0-py3.6-linux-x86_64.egg/librabbitmq/__init__.py", line 122, in basic_publish
    mandatory or False, immediate or False,
UnicodeEncodeError: 'ascii' codec can't encode character '\xe4' in position 155: ordinal not in range(128)

Using python3 and librabbitmq==2.0.0 celery==4.1.0

[auvipy]

pyamqp is the recommended package. try that to see the issue is gone

[fgmehlin]

@auvipy pyamqp indeed solved the issue. Thanks for the quick answer!

Cheers !