TypeError: get_auth_attempts_at_port() got multiple values for argument 'nas_ip_address'

[agdsn-sentry[bot]]

Sentry Issue: PYCROFT-2W

TypeError: get_auth_attempts_at_port() got multiple values for argument 'nas_ip_address'
(13 additional frame(s) were not displayed)
...
  File "celery/backends/asynchronous.py", line 223, in wait_for_pending
    return result.maybe_throw(callback=callback, propagate=propagate)
  File "celery/result.py", line 336, in maybe_throw
    self.throw(value, self._to_remote_traceback(tb))
  File "celery/result.py", line 329, in throw
    self.on_ready.throw(*args, **kwargs)
  File "vine/promises.py", line 234, in throw
    reraise(type(exc), exc, tb)
  File "vine/utils.py", line 30, in reraise
    raise value

[lukasjuhrich]

Minimal working repro on the deployed instance: (FLASK_APP=web.app flask shell)

hl = app.extensions["hades_logs"]
res = hl.celery.signature(
    f"hades.agent.rpc.get_auth_attempts_at_port",
    args=(),
    kwargs=dict(
        nas_ip_address="<IP>",
        nas_port_id="<PORT>",
        limit=5,
    ),
).apply_async().wait(timeout=2)

Note that this example already has the definitions of HadesLogs.signature, HadesLogs.celery.main, HadesLogs.create_task inlined.

[lukasjuhrich]

The celery events dump says:

Task details for 41a62e42-b5a9-4b56-b73d-25b0f900255f                                                               

args: ()                                                                                                            
kwargs: {'nas_ip_address': '10.160.0.65', 'nas_port_id': '1/1/3', 'limit': 100}                                     
retries: 0                                                                                                          
exception: TypeError("get_auth_attempts_at_port() got multiple values for argument 'nas_ip_address'")               
root_id: 41a62e42-b5a9-4b56-b73d-25b0f900255f                                                                       
state: FAILURE 

Task Exception Traceback for 41a62e42-b5a9-4b56-b73d-25b0f900255f                           

Traceback (most recent call last):                                                          
  File "/usr/lib/python3/dist-packages/celery/app/trace.py", line 409, in trace_task        
    R = retval = fun(*args, **kwargs)                                                       
  File "/usr/lib/python3/dist-packages/celery/app/trace.py", line 701, in __protected_call__
    return self.run(*args, **kwargs)                                                        
TypeError: get_auth_attempts_at_port() got multiple values for argument 'nas_ip_address'    

[lukasjuhrich]

Which is very weird because args are clearly empty, and the kwargs clearly don't contain a duplicate (how can they, they're a dict)…

[lukasjuhrich]

(Pdb) l
386             self.push_request(args=args, kwargs=kwargs)
387             try:
388  >>             return self.run(*args, **kwargs)
389             finally:
390                 self.pop_request()
391  ->             _task_stack.pop()
392  
393         def __reduce__(self):
394             # - tasks are pickled into the name of the task only, and the receiver
395             # - simply grabs it from the local registry.
396             # - in later versions the module of the task is also included,
(Pdb) self.run
<bound method get_auth_attempts_at_port of <@task: hades.agent.rpc.get_auth_attempts_at_port of hades.agent at 0x7f207495ad60>>
(Pdb) import inspect as i
(Pdb) i.signature(self.run)
<Signature (nas_port_id: 'str', when: 'Optional[TimestampRange]' = None, limit: 'Optional[int]' = 100) -> 'List[Tuple[str, str, Groups, Attributes, float]]'>
(Pdb) self.run.__self__
<@task: hades.agent.rpc.get_auth_attempts_at_port of hades.agent at 0x7f207495ad60>

Okay, something goes wrong with the task wrapping; the signature is correct but the method gets bound to the task class, i.e. nas_ip_address as the first argument binds to __self__.

Transferring to hades.

[lukasjuhrich]

fixed by 7aeb7f665024ff23af5bc03f4f571baa25307302