Open ozamani9gh opened 2 months ago
Update from Joshua: • The removal of the output delay creates confusion for clients if an examiner were to catch a mistake and go back to fix it (generates multiple different results emails to clients, who then call or email in confusion which creates a cascade effect). Even a few minutes delay for output generation would help reduce this.
@Mihai-QuickSilverDev reach out to Joshua and get the exact delay he wants, we suggest 10 or 15 minutes.
Message sent to Joshua on June 28, 2024, 9 AM:
Hi Joshua,
Regarding this topic below, could you please let us know what would be a desirable duration for this delay? We could do 5, 10, 15 minutes, whatever would best fit the Name Examiners needs.
Update from Joshua: • The removal of the output delay creates confusion for clients if an examiner were to catch a mistake and go back to fix it (generates multiple different results emails to clients, who then call or email in confusion which creates a cascade effect). Even a few minutes delay for output generation would help reduce this.
Thank you,
Joshua stated 5 minutes is sufficient
Clients are emailed about their NRs in the following scenarios:
Async Operations with Flask Framework: https://flask.palletsprojects.com/en/3.0.x/async-await/# Async Operations with Gunicorn Server: https://docs.gunicorn.org/en/latest/design.html
Gunicorn WSGI Server
Why its not possible to us async/await on each email for 5 minutes Flask on its own is single-threaded (blocking) and does not natively support asynchronous operations across requests - only within them (can await a database operation, but cant schedule it - it will be cancelled once the request ends since flask handles 1 request at a time).
workers
that run independently of each other
workers
can be sync
(default) or gevent
(green threads)sync
is the default worker type and the one that is used in our API
Background Tasks
, it states that this is not possible on a WSGI server (1 request-response cycle per worker).Change the Gunicorn worker to be of type gevent
to allow for async operations
When to use Quart instead
in the Flask docs seem to imply this is more of a work-around.Spawn a separate thread
for async operations
Use Celery
to get the async effect
Use a Cron Job
to get the async effect
A good resource for this topic: https://testdriven.io/blog/flask-async/
It is not really an option to change the functionality of the queue. The queue module is located here: https://github.com/bcgov/namex/blob/main/services/pubsub/gcp_queue/pubsub.py
Reasons why changing this queue is not an option
The queue also runs on a Flask Server and has the same problem of not being able to perform async operations.
It cannot wait for 5 minutes in between requests as this would block the queue for 5 minutes or the task would be cancelled once the request ends. This is the code that the API calls to publish emails:
def publish(self, topic: str, payload: bytes):
"""Send payload to the queue."""
if not (publisher := self.publisher):
raise Exception("missing setup arguments") # pylint: disable=W0719
try:
future = publisher.publish(topic, payload) # Send email to gcp emailer
return future.result() # Wait for the completion and then move on
except (CancelledError, TimeoutError) as error:
raise Exception("Unable to post to queue", error) from error # pylint: disable=W0719
Priority 3: Email delay-previously emails generated by examining a name had a 5-minute delay. This was helpful when examining as any mistakes (forgot to put condition on, made error in examining, etc.) wouldn’t result in multiple outputs that are confusing for the client. Preference for a short delay that only generates the last email. (Frequency 100%) Time cost: approx. 20-30min/week