search

deploifai / meteorite

A fast and simple web server for machine learning models
https://pypi.org/project/meteorite
MIT License
8 stars 1 forks source link

Add job queue to process requests #13

Closed utkarsh867 closed 11 months ago

utkarsh867 commented 1 year ago

This is a major change in the way Meteorite processes requests from clients.

After this change, Meteorite will use a job queue to keep a track of tasks sent to it as a request. Any client making a request to a Meteorite server immediately receives a response with the job ID. The intention behind adding this ID is to be able to poll the status of the job using it.

At the moment the Job ID is not persistent. Ideally, something like Redis or any other kind of database could potentially help make job IDs persistent.

This change was required due to the nature of ML models requiring some time to complete their computation.

utkarsh867 commented 1 year ago

There is some merit in being able to enable and disable queueing. However, that is not available in this PR at the moment.

utkarsh867 commented 11 months ago

@98sean98 Pending on your review only.

I have limited this PR to adding the webhook as you suggested previously! Seems like this works out really well.

In the future, it will be great to add an option to switch between different approaches to do job queues.

We can merge this, and implement a client for sending requests to this server since its not a simple HTTP request.