neisbut / EntityFramework.MemoryJoin

Extension for EntityFramework for joins to in-memory data
MIT License
56 stars 24 forks source link

update to .netcore 3 #10

Closed geebacsi closed 4 years ago

neisbut commented 4 years ago

Hi @geebacsi , thanks for this!

geebacsi commented 4 years ago

Dear Anton,

I think this solution is not the best. We tested our site, and it boost up the processor to 100%. Could you check this?

Sorry for that, this was my first pull request to github.

BR, Gábor

On Sun, Sep 29, 2019 at 3:34 PM Anton Shkuratov notifications@github.com wrote:

Hi @geebacsi https://github.com/geebacsi , thanks for this!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/neisbut/EntityFramework.MemoryJoin/pull/10?email_source=notifications&email_token=AA7T33WPPDVDVFWFGMSK64LQMCVGZA5CNFSM4IZ6KYBKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD73U3MA#issuecomment-536300976, or mute the thread https://github.com/notifications/unsubscribe-auth/AA7T33VQB575I6GJCR3SNETQMCVGZANCNFSM4IZ6KYBA .

-- Üdvözlettel: Majoros Gábor 70/284 9010 <(70)%20284%209010>

neisbut commented 4 years ago

Hi @geebacsi,

I think there is nothing wrong with your PR, it was fine. Could you please give a bit more details on what query you are making? Problem might be in the query itself and how EF Core optimized it. For example, EF 6 always tries to convert query to single SQL command. And MemoryJoin works very well with EF 6. But EF Core might do some job locally (or can perform multiple SQL queries) and I faced cases when usage MemoryJoin was useless because of such EF Core "optimizations". In my case I did a join of big table to 10k records like this:

var localPrices = context.FromLocalList(GetPricesToCheck());
var query = from t in context.Prices
                   from p in localPrices.Where(
                        x => x.SecurityId == t.SecurityId && x.TradedOn <= t.PriceDate)
                    .OrderByDescending(x => x.TradedOn)
                    .Take(1)
                select p;

This worked very slow.. but when I changed the order of execution like this:

var query = from t in localPrices
                   from p in context.Prices.Where(
                        x => x.SecurityId == t.SecurityId && x.TradedOn <= t.PriceDate)
                    .OrderByDescending(x => x.TradedOn)
                    .Take(1)
                select p;

It started to work very fast. So, I think you may try to rewrite your query differently. This might help you.

Thanks, Anton