search

fermi-ad / acsys-python

Python module to access the Fermilab Control System
MIT License
8 stars 2 forks source link

ACNET error 1 -6 on enable_settings #49

Closed beauremus closed 2 years ago

beauremus commented 2 years ago

We have a report that sometimes (unreliably replicable) when enable_settings is requested, an ACNET_REQTMO is returned.

We believe this is happening at the ACNET level because we are requesting that the ACNET request for enable_settings times out after 1000ms.

I've attempted shortening the timeout to provoke the error to no avail.

Our hypothesis is that sometimes the KDC is taking more than 1s to respond. So, this is only replicable when the KDC is slow which we don't have control over.

beauremus commented 2 years ago

Currently, we have a function, _retryable_request, that retries the request if an ACNET_REQTMO status is returned. We aren't sure what the motivation for this was originally. We likely saw ACNET_REQTMO, during testing, with some regularity, and this was a band-aid.

Unfortunately, this function was not used for the auth step.

This issue has sparked a conversation about how this is best handled because ACNET will wait for us without retrying. So, we are considering offloading the waiting to ACNET and allowing users to wait as long as they deem reasonable for their request.

beauremus commented 2 years ago

ab5cfce9087dd7e121b3ac019f57abb4dfee49ea and bc4c0da5df5a0fe3a7f3c2d69d62ceaacf754ce4 resolves this issue by allowing the user to decide how long to wait.