openbullet / OpenBullet2

OpenBullet reinvented
https://docs.openbullet.dev/
MIT License
1.74k stars 464 forks source link

[REQUEST] Add posibility to put back item at the end of the queue in case of unhandled exception in the work Function | RuriLib.Parallelization #322

Open ghost opened 3 years ago

ghost commented 3 years ago

Hello, First of all I have to say this library is very useful and works very well. I came across a problem, let's say I have the following code:

Func<int, CancellationToken, Task<string>> workFunction = new(async (number, token) =>
            {
                var res = DoSomeWork(number);
                if (res == true)
                {
                    // this function can throw an exception in some cases (let's say it is an http request for exemple)
                    DoSomeOtherWork(number);
                    return "WhatEver result";
                }
                else
                {
                    return "WhatEver result";
                }
            });

            var parallelizer = ParallelizerFactory<int, string>.Create(
                type: ParallelizerType.TaskBased, // Use task-based (it's better)
                workItems: Enumerable.Range(1, 100), // The work items are all integers from 1 to 100
                workFunction: workFunction, // Use the work function we defined above
                degreeOfParallelism: 5, // Use 5 concurrent tasks at most
                totalAmount: 100, // The total amount of tasks you expect to have, used for calculating progress
                skip: 0); // How many items to skip from the start of the provided enumerable
                parallelizer.TaskError += OnTaskError;
                await parallelizer.Start();

In this workFunction when the do DoSomeOtherWork throw an exception it will trigger TaskError event but the item is considered as "completed" and not put back in the queue so if I want to process it I would have to recreate a Parallelizer only with the failed items. this would be easier if there was possibility to put directly back in the queue when the workFunction got an unhandled exception. This feature could be possibly activated or deactivated by a simple parameter in the ctor.

openbullet commented 3 years ago

Wouldn't it be easier to just do something like

int maxTries = 10;
int currentTry = 1;
START:
try
{
    var res = DoSomeWork(number);
    // etc...
}
catch (Exception ex)
{
    Console.WriteLine(ex.Message);
    if (++currentTry < maxTries)
    {
        goto START;
    }
}

In the OB2 code, for example, I use the goto to loop until one of these conditions happen

If you give me a reason why this behaviour wouldn't be suitable, I will implement the feature that you asked for.

ghost commented 3 years ago

Thanks for responding. Of couse yes that would work but let's say the item you're processing is "banned" (because of some cooldowns for exemple) so you retry again and again but as the item is "banned" so it will never end with a good status. With this feature as the item is put at the end of the queue, while the other items are getting processed, the item that was put at the end of the queue is getting unbanned, so now reprocessing it will not give a bad status. I hope you understand what I mean

openbullet commented 3 years ago

I'll see if I can find some time to do this, but I'm marking it as low priority for now since there are important performance issues to address first.

ghost commented 3 years ago

No problems, take your time. I wish I could help you but I'm not experienced enough in c# and don't understand fully the code.