Query involving List type initialization

[cjmairair]

If you write a query involving a "local list" and involving an initialization of that list type, there's a good chance you'll get this exception:

"The type 'XYZ' appears in two structurally incompatible initializations within a single LINQ to Entities query. A type can be initialized in two places in the same query, but only if the same properties are set in both places and those properties are set in the same order."

For example, there's the chance of this exception when running this query, because of the "new Key" initialization part.

context.FromLocalList(keyList)
                        .GroupJoin(
                                context.Table2,
                                k => k,
                                x => new Key { Id1 = t.Field1, Id2 = t.Field2 },
....
);

public class Key
{
  public int Id1;
  public int Id2;
}

I'm guessing that the reason for this error is that the MemoryJoin code is generating some kind of projection (like using Select), like:

.Select(new Key { ...})

[neisbut]

Hi @cjmairair, yes, you are absolutely correct, internally it uses a projection. Basically it has to use it. And honesly I don't have an idea how to handle it internally of MemoryJoin. Basically fields initialization in new Key { Id1 = t.Field1, Id2 = t.Field2 } should go in exactly same order (as they described in Key class). Maybe you have an idea how to solve it?

[cjmairair]

Yeah, I haven't come up with an idea of how to solve it.

On Mon, Mar 18, 2019 at 11:15 PM Anton Shkuratov notifications@github.com wrote:

Hi @cjmairair https://github.com/cjmairair, yes, you are absolutely correct, internally it uses a projection. Basically it has to use it. And honesly I don't have an idea how to handle it internally of MemoryJoin. Basically fields initialization in new Key { Id1 = t.Field1, Id2 = t.Field2 } should go in exactly same order (as they described in Key class). Maybe you have an idea how to solve it?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/neisbut/EntityFramework.MemoryJoin/issues/5#issuecomment-474211943, or mute the thread https://github.com/notifications/unsubscribe-auth/AchMQ2b4JFBQ4sEDqkIPFN4llRVHuaQrks5vYIBkgaJpZM4b7Ct1 .

[cjmairair]

Idea:

Maybe this will work. I haven't tested anything though.

You can project into a subclass of the list type - instead of projecting into the list type. And you can create that subclass dynamically. (See https://stackoverflow.com/questions/3862226/how-to-dynamically-create-a-class-in-c). And then cast it to the list type. Like:

context.ToLocalList<T>(myList) becomes context.DummyEntities.Select(x => new DynamicallyGeneratedSubclassOfT { Id1 = x.Int1, Id2 = x.String1 }).Cast<T>()

(You can of course test whether this will work without doing that dynamic class generation thing.)

[neisbut]

Hi @cjmairair, I tried and result is unsatisfactory. With introducing cast I started to get 'Unable to cast the type '...KeyEntityDerived' to type '...KeyEntity'. LINQ to Entities only supports casting EDM primitive or enumeration types.'

I also tried the following code in different variations (of types, initialization order, etc) and no luck:

context.Addresses.GroupJoin(context.Addresses,
    x => new KeyEntity() { HouseNumber = x.HouseNumber, StreetName = x.StreetName },
    x => new KeyEntityDerived() { StreetName = x.StreetName, HouseNumber = x.HouseNumber },
    (x, y) => x).FirstOrDefault();

There is only way possible way to solve it which I see for now: use some ExpressionVisitor which could resort field initializations and make them consistent. But that will demand calling of some extension methods or ExpressionVisitor itself.

[cjmairair]

Okay, but then wouldn't the programmer need to make sure that he always writes those fields in the "correct" order whenever he wants to do an initialization in a query?

On Mon, Apr 1, 2019 at 8:17 PM Anton Shkuratov notifications@github.com wrote:

Hi @cjmairair https://github.com/cjmairair, I tried and result is unsatisfactory. With introducing cast I started to get 'Unable to cast the type '...KeyEntityDerived' to type '...KeyEntity'. LINQ to Entities only supports casting EDM primitive or enumeration types.'

I also tried the following code in different variations (of types, initialization order, etc) and no luck:

context.Addresses.GroupJoin(context.Addresses, x => new KeyEntity() { HouseNumber = x.HouseNumber, StreetName = x.StreetName }, x => new KeyEntityDerived() { StreetName = x.StreetName, HouseNumber = x.HouseNumber }, (x, y) => x).FirstOrDefault();

There is only way possible way to solve it which I see for now: use some ExpressionVisitor which could resort field initializations and make them consistent. But that will demand calling of some extension methods or ExpressionVisitor itself.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/neisbut/EntityFramework.MemoryJoin/issues/5#issuecomment-478828155, or mute the thread https://github.com/notifications/unsubscribe-auth/AchMQ6DjiVC0FdCeBOA6DGHUDrhLOLuDks5vcsuzgaJpZM4b7Ct1 .

[cjmairair]

This will work. But see comment for the bad part.

`
// Mimics ToLocalList
IQueryable<Key> keyQ = context.FakeTable.Select(x => new KeySub
                        {
                            Id1 = x.String1,
                            Id2 = x.Int1
                        });

            var mainQ = keyQ.GroupJoin(context.Things,
            x => new Key() { Id1 = x.Id1, Id2 = x.Id2 },     // But you have to expand this out. You can't just write x => x. So, this idea might trip people up.
            x => new Key() { Id1 = x.Field1, Id2 = x.Field2 },
            (key, pss) => new
            {
                .....
            })
            .ToList();
`

class Key
{
  public string Id1;
  public int Id2;
}

class KeySub : Key
{
}

[neisbut]

Hi @cjmairair, yes, you are right. That "bad" part is most important because provides structurally compatible initializations. Actually you don't even need derived class, so the following will also work.

        // Mimics ToLocalList
        // I can even skip Select here!! This part is not that important, you can you Select for your convinience. OR it can come from ToLocalList of course.
        IQueryable<Key> keyQ = context.FakeTable;

        var mainQ = keyQ.GroupJoin(context.Things,
        x => new Key() { Id1 = x.String1, Id2 = x.Int1 },      // Same init order here
        x => new Key() { Id1 = x.Field1, Id2 = x.Field2 },     // ... and here
        (key, pss) => new
        {
            .....
        })
        .ToList();

So using these 2 selects above is a guerantee that this query will work the same way always (even if Key class is changed, like when properties resorted, new ones are added, etc)

[cjmairair]

I think there is some misunderstanding about what is going on here. So, just to be clear...

When I want to do something like this...

`
IQueryable<Key> keyQ = context.ToLocalList(myKeyList);

            var mainQ = keyQ.GroupJoin(context.Things,
            x => new Key() { Id1 = x.Id1, Id2 = x.Id2 },   // or x => x, I think will run okay also
            x => new Key() { Id1 = x.Field1, Id2 = x.Field2 },
            (key, pss) => new
            {
                .....
            })
            .ToList();
`

class Key
{
  public string Id1;
  public int Id2;
}

... then there is the possibility of getting this EF error:

"The type 'Key' appears in two structurally incompatible initializations within a single LINQ to Entities query. A type can be initialized in two places in the same query, but only if the same properties are set in both places and those properties are set in the same order."

This error will happen if, in your MemoryJoin code, you build the projection-selection-initializer thing with Id2 1st, and then Id1 2nd. Like this:

// Below would be equivalent to: IQueryable<Key> keyQ = context.ToLocalList(myKeyList);

IQueryable<Key> keyQ = context.QueryModelClass.Select(x => new Key
                        {
                            Id2 = x.Int1,
                            Id1 = x.String1
                        });

One way to fix this problem is like this:

`
// This is what you would need to in ToLocalList (but using Expression trees instead)
// In other words, this code is what this would be: IQueryable<Key> keyQ = context.ToLocalList(myKeyList);

IQueryable<Key> keyQ = context.QueryModelClass.Select(x => new KeySub
                        {
                            Id2 = x.String1,   // Notice that these fields can now be initialized in a different order than in the rest of the query below
                            Id1 = x.Int1
                        });

            var mainQ = keyQ.GroupJoin(context.Things,
            x => new Key() { Id1 = x.Id1, Id2 = x.Id2 },     // But you have to expand this out. You can't just write x => x. So, this idea might trip people up.
            x => new Key() { Id1 = x.Field1, Id2 = x.Field2 },
            (key, pss) => new
            {
                .....
            })
            .ToList();
`

class Key
{
  public string Id1;
  public int Id2;
}

// This class would be internally generated at runtime by MemoryJoin
class KeySub : Key
{
}

Also, instead of using a subclass (KeySub), you can also do it like this, but I don't know why you would want to do it this way - I don't see any advantages.

`
IQueryable<Key> keyQ = context.QueryModelClass.Select(x => new Key2
                        {
                            Id2 = x.String1,  
                            Id1 = x.Int1
                        });

`

class Key
{
  public string Id1;
  public int Id2;
}

// This class would be internally generated at runtime by MemoryJoin
// It has the exact same stuff in it as the Key class.
class Key2 : Key
{
  public string Id1;
  public int Id2;
}

[neisbut]

Hi @cjmairair, yes, I think I got you point. It is not a problem to generate a new KeySub type dynamically, but I see some problems with usage. I mean when FromLocalList<Key>(..) the result must be IQueryable<Key>, not IQueryable<KeySub> (because it doesn't yet exists), right? I could use 2 approaches (for using projection to KeySub and casting to Key):

context.QueryModelClass.Select(x => new KeySub { ... }).Cast<Key>();

OR

context.QueryModelClass.Select(x => (Key)new KeySub { ... });

1 - Then if I try to use it like this:

 var mainQ = keyQ.GroupJoin(context.Things,
      x => x                                                 // IMPORTANT
      x => new Key() { Id1 = x.Field1, Id2 = x.Field2 },
      (key, pss) => new
      {
          .....
      })
      .ToList();

In BOTH approaches an exception will be thrown: 'Unable to cast the type '...KeySub' to type '...Key'. LINQ to Entities only supports casting EDM primitive or enumeration types.'

2 - At the same time if I use the following query:

 var mainQ = keyQ.GroupJoin(context.Things,
      x => new Key() { Id1 = x.Field1, Id2 = x.Field2 },   // IMPORTANT
      x => new Key() { Id1 = x.Field1, Id2 = x.Field2 },
      (key, pss) => new
      {
          .....
      })
      .ToList();

.. then it works fine, but derived class doen't play any role here: above construction works fine when I use projection to Key in FromLocalList method.

These 2 results makes me think that dynamic type generation doesn't provide any benefit.

So recomendation is to use syntax with expansions in joins.

[cjmairair]

You asked:

I mean when FromLocalList<Key>(..) the result must be IQueryable<Key>, not IQueryable<KeySub> (because it doesn't yet exists), right?"

Right. IQueryable<Key>

1 - Then if I try to use it like this:

 var mainQ = keyQ.GroupJoin(context.Things,
      x => x                                                 // IMPORTANT
      x => new Key() { Id1 = x.Field1, Id2 = x.Field2 },
      (key, pss) => new
      {
          .....
      })
      .ToList();

In BOTH approaches an exception will be thrown: 'Unable to cast the type '...KeySub' to type '...Key'. LINQ to Entities only supports casting EDM primitive or enumeration types.'

Right, I realize now that you can't use Cast().

2 - At the same time if I use the following query:

 var mainQ = keyQ.GroupJoin(context.Things,
      x => new Key() { Id1 = x.Field1, Id2 = x.Field2 },   // IMPORTANT
      x => new Key() { Id1 = x.Field1, Id2 = x.Field2 },
      (key, pss) => new
      {
          .....
      })
      .ToList();

.. then it works fine, but derived class doen't play any role here: above construction works fine when I use projection to Key in FromLocalList method.

These 2 results makes me think that dynamic type generation doesn't provide any benefit.

No. There is a benefit to #2. It gets rid of the "parameters out of order" exception. (You said that you didn't get an exception when doing this query with the current implementation of ToLocalList. That's because, in your query, you, luckily, ordered the initialization parameters in the same order as MemoryJoin ordered them in the generated expression tree.)

`
// This is what you would need to in ToLocalList (but using Expression trees instead)
// In other words, this code is what this would be: IQueryable<Key> keyQ = context.ToLocalList(myKeyList);

IQueryable<Key> keyQ = context.QueryModelClass.Select(x => new KeySub    // IF YOU DO THIS WITH Key instead of KeySub, you will get that "initialization parameters out of order" exception
                        {
                            Id2 = x.String1,   // Notice that these fields can now be initialized in a different order than in the rest of the query below
                            Id1 = x.Int1
                        });

            var mainQ = keyQ.GroupJoin(context.Things,
            x => new Key() { Id1 = x.Id1, Id2 = x.Id2 },     // But you have to expand this out. You can't just write x => x. So, this idea might trip people up.
            x => new Key() { Id1 = x.Field1, Id2 = x.Field2 },
            (key, pss) => new
            {
                .....
            })
            .ToList();
`

If you write your query like this, you will get the exception.

x => new Key() { Id2 = x.Id2, Id1 = x.Id1 }, x => new Key() { Id2 = x.Field2, Id1 = x.Field1 },

[neisbut]

Hi @cjmairair, let's formalize this in a form of unit tests.

We have 2 options for FromLocalList:

Option 1:

IQueryable<T> FromLocalList(IList<T> list) 
{
     return context.DummyData.Select(x => new T() { ... });
}

Option 2:

IQueryable<T> FromLocalList(IList<T> list) 
{
     // generate sub class TSub
     return context.DummyData.Select(x => new TSub() { ... });
}

Next we have the following use cases:

Case 1:

var keyQ = context.FromLocalList(keyList);
var mainQ = keyQ.GroupJoin(context.Things,
    x => x
    x => new Key() { Id1 = x.Field1, Id2 = x.Field2 },
    ....).ToList();

Case 2:

var keyQ = context.FromLocalList(keyList);
var mainQ = keyQ.GroupJoin(context.Things,
    x => new Key() { Id1 = x.Id1, Id2 = x.Id2 },
    x => new Key() { Id1 = x.Field1, Id2 = x.Field2 },
    ....).ToList();

Here what result I get:

Option 1: Case 1: May work and may not work throwing ...appears in two structurally incompatible initializations... :question: Case 2: Works fine always. :heavy_check_mark:

Option 2: Case 1: Doesn't work, throwing ... unexpected type ... :x: Case 2: Works fine always. :heavy_check_mark:

Am I missing something?

[cjmairair]

I don't know when/if I'll have time to get to this again. But, yes, you are missing the main problem.

You need a Case 3. Which is the same as Case 2 except like this.

Case 2:

var keyQ = context.FromLocalList(keyList);

var mainQ = keyQ.GroupJoin(context.Things,

x => new Key() { Id1 = x.Id1, Id2 = x.Id2 },

x => new Key() { Id2 = x.Field2, Id1 = x.Field1 },

....).ToList();

You see, you don't even need to do a group join. You don't need to have those 2 initialization lines above to get the problem. The 2nd line is enough.

All you need in order to get the exception is for the user to have an initialization in their query whose parameters are ordered differently than the parameters in the query which you internally generate in your FromLocalList.

Got it?

On Fri, Apr 19, 2019 at 7:01 PM Anton Shkuratov notifications@github.com wrote:

Hi @cjmairair https://github.com/cjmairair, let's formalize this in a form of unit tests.

We have 2 options for FromLocalList:

Option 1:

IQueryable FromLocalList(IList list)

{

 return context.DummyData.Select(x => new T() { ... });

}

Option 2:

IQueryable FromLocalList(IList list)

{

 // generate sub class TSub

 return context.DummyData.Select(x => new TSub() { ... });

}

Next we have the following use cases:

Case 1:

var keyQ = context.FromLocalList(keyList);

var mainQ = keyQ.GroupJoin(context.Things,

x => x

x => new Key() { Id1 = x.Field1, Id2 = x.Field2 },

....).ToList();

Case 2:

var keyQ = context.FromLocalList(keyList);

var mainQ = keyQ.GroupJoin(context.Things,

x => new Key() { Id1 = x.Id1, Id2 = x.Id2 },

x => new Key() { Id1 = x.Field1, Id2 = x.Field2 },

....).ToList();

Here what result I get:

Option 1: Case 1: May work and may not work throwing ...appears in two structurally incompatible initializations... ❓ Case 2: Works fine always. ✔️

Option 2: Case 1: Doesn't work, throwing ... unexpected type ... ❌ Case 2: Works fine always. ✔️

Am I missing something?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/neisbut/EntityFramework.MemoryJoin/issues/5#issuecomment-485050064, or mute the thread https://github.com/notifications/unsubscribe-auth/AHEEYQZQXFC4D3BBZLEC5F3PRJ2OTANCNFSM4G7MFN2Q .

[cjmairair]

Correction:

var mainQ = keyQ.GroupJoin(context.Things,

x => new Key() { Id2 = x.Id2, Id1 = x.Id1 },

x => new Key() { Id2 = x.Field2, Id1 = x.Field1 },

....).ToList();

The query above would normally be okay. But with your FromLocalList() you will get that exception.

On Wed, May 8, 2019 at 7:13 PM CJM Airstrip cjm.airstrip@gmail.com wrote:

I don't know when/if I'll have time to get to this again. But, yes, you are missing the main problem.

You need a Case 3. Which is the same as Case 2 except like this.

Case 2:

var keyQ = context.FromLocalList(keyList);

var mainQ = keyQ.GroupJoin(context.Things,

x => new Key() { Id1 = x.Id1, Id2 = x.Id2 },

x => new Key() { Id2 = x.Field2, Id1 = x.Field1 },

....).ToList();

You see, you don't even need to do a group join. You don't need to have those 2 initialization lines above to get the problem. The 2nd line is enough.

All you need in order to get the exception is for the user to have an initialization in their query whose parameters are ordered differently than the parameters in the query which you internally generate in your FromLocalList.

Got it?

On Fri, Apr 19, 2019 at 7:01 PM Anton Shkuratov notifications@github.com wrote:

Hi @cjmairair https://github.com/cjmairair, let's formalize this in a form of unit tests.

We have 2 options for FromLocalList:

Option 1:

IQueryable FromLocalList(IList list)

{

 return context.DummyData.Select(x => new T() { ... });

}

Option 2:

IQueryable FromLocalList(IList list)

{

 // generate sub class TSub

 return context.DummyData.Select(x => new TSub() { ... });

}

Next we have the following use cases:

Case 1:

var keyQ = context.FromLocalList(keyList);

var mainQ = keyQ.GroupJoin(context.Things,

x => x

x => new Key() { Id1 = x.Field1, Id2 = x.Field2 },

....).ToList();

Case 2:

var keyQ = context.FromLocalList(keyList);

var mainQ = keyQ.GroupJoin(context.Things,

x => new Key() { Id1 = x.Id1, Id2 = x.Id2 },

x => new Key() { Id1 = x.Field1, Id2 = x.Field2 },

....).ToList();

Here what result I get:

Option 1: Case 1: May work and may not work throwing ...appears in two structurally incompatible initializations... ❓ Case 2: Works fine always. ✔️

Option 2: Case 1: Doesn't work, throwing ... unexpected type ... ❌ Case 2: Works fine always. ✔️

Am I missing something?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/neisbut/EntityFramework.MemoryJoin/issues/5#issuecomment-485050064, or mute the thread https://github.com/notifications/unsubscribe-auth/AHEEYQZQXFC4D3BBZLEC5F3PRJ2OTANCNFSM4G7MFN2Q .

[neisbut]

Decided not to fix this. Nobody else complained.