equinor / witsml-explorer

Witsml Explorer data management tool.
Apache License 2.0
45 stars 50 forks source link

Try to optimize the log data comparison by setting up multiple threads #2104

Open janmarius opened 1 year ago

janmarius commented 1 year ago

Is your feature request related to a problem? Please describe. Try to optimize the log data comparison by setting up multiple threads. The CompareLogDataWorker.cs currently compares each mnemonic sequentially, which may not be the most efficient method.

Describe the solution you'd like I want the worker to compare data for each mnemonic concurrently, by using the WhenAll() method on line 79 in the CompareLogDataWorker.cs.

Some pseudocode for a possible solution:

var tasks = new List<Task<List<CompareLogDataItem>>();

foreach (var item in items)
{
    var task = Add.Data()
    tasks.Add(task);
}

var myResults = await Task.WhenAll(tasks);
eliasbruvik commented 12 months ago

As discussed with Stian, we should set a limit to the number of concurrent calls in order to avoid overloading both the server and the API memory usage.