can't abort job

[yuliym]

Hello, We have running AWX cluster 9.0.1 we just discovered that we can't abort running job. When I'm clicking on 'Cancel the job' I see following pop-up error alert

Here is error from awx web container

2020-01-21 20:17:23,556 ERROR django.request Internal Server Error: /api/v2/jobs/21/cancel/ Traceback (most recent call last): File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/core/handlers/exception.py", line 34, in inner response = get_response(request) File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/core/handlers/base.py", line 115, in _get_response response = self.process_exception_by_middleware(e, request) File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/core/handlers/base.py", line 113, in _get_response response = wrapped_callback(request, *callback_args, callback_kwargs) File "/usr/lib64/python3.6/contextlib.py", line 52, in inner return func(*args, *kwds) File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view return view_func(args, kwargs) File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/views/generic/base.py", line 71, in view return self.dispatch(request, *args, kwargs) File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/api/generics.py", line 297, in dispatch return super(APIView, self).dispatch(request, *args, *kwargs) File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/rest_framework/views.py", line 495, in dispatch response = self.handle_exception(exc) File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/rest_framework/views.py", line 455, in handle_exception self.raise_uncaught_exception(exc) File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/rest_framework/views.py", line 492, in dispatch response = handler(request, args, kwargs) File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/api/views/init.py", line 3601, in post obj.cancel() File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/main/models/unified_jobs.py", line 1354, in cancel if self.status == 'running' and not self.actually_running: File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/main/models/unified_jobs.py", line 1323, in actually_running ).running(timeout=timeout) File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/main/dispatch/control.py", line 37, in running return self.control_with_reply('running', *args, *kwargs) File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/main/dispatch/control.py", line 44, in control_with_reply with Consumer(conn, reply_queue, callbacks=[self.process_message], no_ack=True): File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/kombu/messaging.py", line 386, in init self.revive(self.channel) File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/kombu/messaging.py", line 408, in revive self.declare() File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/kombu/messaging.py", line 421, in declare queue.declare() File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/kombu/entity.py", line 608, in declare self._create_queue(nowait=nowait, channel=channel) File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/kombu/entity.py", line 617, in _create_queue self.queue_declare(nowait=nowait, passive=False, channel=channel) File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/kombu/entity.py", line 652, in queue_declare nowait=nowait, File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/main/dispatch/kombu.py", line 23, in queue_declare return super(_Channel, self).queue_declare(queue, args, *kwargs) File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/amqp/channel.py", line 1154, in queue_declare spec.Queue.DeclareOk, returns_tuple=True, File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/amqp/abstract_channel.py", line 80, in wait self.connection.drain_events(timeout=timeout) File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/amqp/connection.py", line 500, in drain_events while not self.blocking_read(timeout): File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/amqp/connection.py", line 506, in blocking_read return self.on_inbound_frame(frame) File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/amqp/method_framing.py", line 55, in on_frame callback(channel, method_sig, buf, None) File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/amqp/connection.py", line 510, in on_inbound_method method_sig, payload, content, File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/amqp/abstract_channel.py", line 126, in dispatch_method listener(args) File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/amqp/channel.py", line 282, in _on_close reply_code, reply_text, (class_id, method_id), ChannelError, amqp.exceptions.AccessRefused: Queue.declare: (403) ACCESS_REFUSED - queue name 'amq.rabbitmq.reply-to' contains reserved prefix 'amq.*'

[sujiar37]

@yuliym , the new release[V9.2.0] is out and I wasn't able to reproduce this. Please take a look.