fogfish / esq

simple persistent queues for Erlang
Apache License 2.0
56 stars 10 forks source link

Head functionality to handle multiple requests #19

Closed AbhilashR2020 closed 4 years ago

AbhilashR2020 commented 4 years ago

Hi,

currently with deq/2, i can get multiple elements dequeued at a time. I would like to request, is there anyway i can read multiple request at a time using head operation, similar to the way deq works.

fogfish commented 4 years ago

Hello,

esq:deq/2 returns messages "ownership" to client, it is physically extracts messages from file segments and destructs segments. In principle you can put whatever large N to deq all messages from your queue.

esq:head/1 retains message ownership within the queue. Physical files do not support any kind of rollback or scans atm. Therefore, head ops cannot look ahead an extract more than loaded into memory. Therefore N cannot be large then C.

Would it be confusing that deq has visibility to all message, while head only to memory part?

Can you please elaborate more about your use-case? May be we can find a compromise solution.

RAMAABHI commented 4 years ago

Hi,

Thank for the response.

Basically I want to first read the message and then deque the message. Due to which I am using esq:head/1 and then esq:deq/1.

But the problem is, I wanted to read group of messages. And deque group of message, which might allow my application to make operations faster.

For esq:deq/2 allows us to deq multiple messages. But there is no option for head/1 to retrieve multiple messages. Though I have enabled "capacity" to a bigger value which will allow my head to contain more than 1 message to be stored in in_flight_qieue.

fogfish commented 4 years ago

It looks you are doing: dequeue - process - acknowledge. You are trying to achieve "dequeue" with esq:head feature. Should you try this with In-flight capabilities?

fogfish commented 4 years ago

Do you have any feedback so far? If not, I am about to close the issue.