Cannot see NpgsqlException details when applying migrations

[roji]

From @BenHerron on August 16, 2015 22:48

I'm using latest nuget packages 3.0.0 with entity framework (6.1.3) and entity framework code first migrations and am using package manager console to "update-database" I have some seed data that's attempting to be added and something in that data is causing it to crash than i'm getting the following error in package manager console.

System.Runtime.Serialization.SerializationException: Type 'Npgsql.NpgsqlException' in assembly 'Npgsql, Version=3.0.0.0, Culture=neutral, PublicKeyToken=5d8b90d52f46fda7' is not marked as serializable.
   at System.AppDomain.DoCallBack(CrossAppDomainDelegate callBackDelegate)
   at System.Data.Entity.Migrations.Design.ToolingFacade.Run(BaseRunner runner)
   at System.Data.Entity.Migrations.Design.ToolingFacade.Update(String targetMigration, Boolean force)
   at System.Data.Entity.Migrations.UpdateDatabaseCommand.<>c__DisplayClass2.<.ctor>b__0()
   at System.Data.Entity.Migrations.MigrationsDomainCommand.Execute(Action command)
Type 'Npgsql.NpgsqlException' in assembly 'Npgsql, Version=3.0.0.0, Culture=neutral, PublicKeyToken=5d8b90d52f46fda7' is not marked as serializable.

If you remove permissions to the user to create tables and run "update-database" you also get the same error.

Copied from original issue: npgsql/npgsql#714

[roji]

Thanks, will fix for 3.0.1 and release in a few days.

[roji]

From @BenHerron on August 17, 2015 3:14

Thanks!

[roji]

@BenHerron, can you please grab the latest 3.0.1 build from our unstable feed (or directly from the build server) and confirm that the issue is resolve?

[roji]

From @BenHerron on August 18, 2015 0:0

@roji Sure, will have a look at it now.

[roji]

From @BenHerron on August 18, 2015 0:13

@roji I have tested on 3.1.0-unstable0146, I tested by removing the permissions of the user to access the schema.

So before what I was getting was.

Now I'm getting

System.TypeLoadException: Could not load type 'System.Data.Entity.Migrations.Model.RenameIndexOperation' from assembly 'EntityFramework, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'.
   at Npgsql.NpgsqlMigrationSqlGenerator.Convert(IEnumerable`1 operations)
   at Npgsql.NpgsqlMigrationSqlGenerator.Generate(IEnumerable`1 migrationOperations, String providerManifestToken) in C:\dev\TeamCity\buildAgent\work\e61e4d1c63e73d70\src\EntityFramework6.Npgsql\NpgsqlMigrationSqlGenerator.cs:line 57
   at System.Data.Entity.Migrations.DbMigrator.GenerateStatements(IList`1 operations, String migrationId)
   at System.Data.Entity.Migrations.Infrastructure.MigratorBase.GenerateStatements(IList`1 operations, String migrationId)
   at System.Data.Entity.Migrations.DbMigrator.ExecuteOperations(String migrationId, XDocument targetModel, IEnumerable`1 operations, IEnumerable`1 systemOperations, Boolean downgrading, Boolean auto)
   at System.Data.Entity.Migrations.DbMigrator.ApplyMigration(DbMigration migration, DbMigration lastMigration)
   at System.Data.Entity.Migrations.Infrastructure.MigratorLoggingDecorator.ApplyMigration(DbMigration migration, DbMigration lastMigration)
   at System.Data.Entity.Migrations.DbMigrator.Upgrade(IEnumerable`1 pendingMigrations, String targetMigrationId, String lastMigrationId)
   at System.Data.Entity.Migrations.Infrastructure.MigratorLoggingDecorator.Upgrade(IEnumerable`1 pendingMigrations, String targetMigrationId, String lastMigrationId)
   at System.Data.Entity.Migrations.DbMigrator.UpdateInternal(String targetMigration)
   at System.Data.Entity.Migrations.DbMigrator.<>c__DisplayClassc.<Update>b__b()
   at System.Data.Entity.Migrations.DbMigrator.EnsureDatabaseExists(Action mustSucceedToKeepDatabase)
   at System.Data.Entity.Migrations.Infrastructure.MigratorBase.EnsureDatabaseExists(Action mustSucceedToKeepDatabase)
   at System.Data.Entity.Migrations.DbMigrator.Update(String targetMigration)
   at System.Data.Entity.Migrations.Infrastructure.MigratorBase.Update(String targetMigration)
   at System.Data.Entity.Migrations.Design.ToolingFacade.UpdateRunner.Run()
   at System.AppDomain.DoCallBack(CrossAppDomainDelegate callBackDelegate)
   at System.AppDomain.DoCallBack(CrossAppDomainDelegate callBackDelegate)
   at System.Data.Entity.Migrations.Design.ToolingFacade.Run(BaseRunner runner)
   at System.Data.Entity.Migrations.Design.ToolingFacade.Update(String targetMigration, Boolean force)
   at System.Data.Entity.Migrations.UpdateDatabaseCommand.<>c__DisplayClass2.<.ctor>b__0()
   at System.Data.Entity.Migrations.MigrationsDomainCommand.Execute(Action command)
Could not load type 'System.Data.Entity.Migrations.Model.RenameIndexOperation' from assembly 'EntityFramework, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'.

So I think the initial problem I raised is solved, but that there might be another problem associated with internal operations and not having the right permission.

[roji]

@BenHerron thanks!

It seems you tested against a 3.1.0 unstable version, and the new bug you're seeing (RenameIndexOperation) is related to #384 which I merged there. Could you please open a separate option about the new bug?

Regardless, you can test against nuget package 3.0.1-beta0001, which is the 3.0.1 version (only bugfixes for 3.0.0 that will be published in a few days). This should fix the NpgsqlException serializable bug without adding the RenameIndexOperation bug. It would be great if you could confirm that.

Thanks for the help!

[roji]

From @BenHerron on August 18, 2015 9:26

@roji

Sorry was on auto pilot mode, I will raise a separate issue for that (done #720) .

Just tested on 3.0.1-beta0001 and got the follow error.

System.Runtime.Serialization.SerializationException: Type is not resolved for member 'Npgsql.NpgsqlException,Npgsql, Version=3.0.1.0, Culture=neutral, PublicKeyToken=5d8b90d52f46fda7'.
   at System.AppDomain.DoCallBack(CrossAppDomainDelegate callBackDelegate)
   at System.Data.Entity.Migrations.Design.ToolingFacade.Run(BaseRunner runner)
   at System.Data.Entity.Migrations.Design.ToolingFacade.Update(String targetMigration, Boolean force)
   at System.Data.Entity.Migrations.UpdateDatabaseCommand.<>c__DisplayClass2.<.ctor>b__0()
   at System.Data.Entity.Migrations.MigrationsDomainCommand.Execute(Action command)
Type is not resolved for member 'Npgsql.NpgsqlException,Npgsql, Version=3.0.1.0, Culture=neutral, PublicKeyToken=5d8b90d52f46fda7'.

[roji]

That's really weird... Will take a look ASAP.

[roji]

@BenHerron, I've tried to look into this.

First, I got the Type is not resolved for member 'Npgsql.NpgsqlException,Npgsql...' even when using Npgsql 2.2, and could only work around it by installing Npgsql.dll into the GAC (gacutil /i Npgsql.dll as administrator). I suspect it might be the only way to solve the issue aside from copying Npgsql.dll somewhere into Visual Studio as part of a VSIX (we're actually working on this approach at the moment).

After doing that I had to deploy another fix to allow NpgsqlException to serialize. However, I still ran into a weird Exception calling "CreateInstanceFrom" with "8" argument(s): "Object reference not set to an instance of an object.", which I suspect might be an issue with my environment.

So it would be great if you could give 3.0.1 yet another test. If it still doesn't work I'll try looking into it some more, but will probably release 3.0.1 without the fix as it's taking a while...

Thanks again for all your time!

[roji]

From @BenHerron on August 18, 2015 21:32

@roji

I got the following exception on 3.0.1-beta0001

System.Runtime.Serialization.SerializationException: Type is not resolved for member 'Npgsql.NpgsqlException,Npgsql, Version=3.0.1.0, Culture=neutral, PublicKeyToken=5d8b90d52f46fda7'.
   at System.AppDomain.DoCallBack(CrossAppDomainDelegate callBackDelegate)
   at System.Data.Entity.Migrations.Design.ToolingFacade.Run(BaseRunner runner)
   at System.Data.Entity.Migrations.Design.ToolingFacade.Update(String targetMigration, Boolean force)
   at System.Data.Entity.Migrations.UpdateDatabaseCommand.<>c__DisplayClass2.<.ctor>b__0()
   at System.Data.Entity.Migrations.MigrationsDomainCommand.Execute(Action command)
Type is not resolved for member 'Npgsql.NpgsqlException,Npgsql, Version=3.0.1.0, Culture=neutral, PublicKeyToken=5d8b90d52f46fda7'.

I than added Npgsql.dll into the GAC and got the following error

System.Runtime.Serialization.SerializationException: The constructor to deserialize an object of type 'Npgsql.NpgsqlException' was not found.
   at System.AppDomain.DoCallBack(CrossAppDomainDelegate callBackDelegate)
   at System.Data.Entity.Migrations.Design.ToolingFacade.Run(BaseRunner runner)
   at System.Data.Entity.Migrations.Design.ToolingFacade.Update(String targetMigration, Boolean force)
   at System.Data.Entity.Migrations.UpdateDatabaseCommand.<>c__DisplayClass2.<.ctor>b__0()
   at System.Data.Entity.Migrations.MigrationsDomainCommand.Execute(Action command)
The constructor to deserialize an object of type 'Npgsql.NpgsqlException' was not found.

I agree it's best not to release this just yet. I'm not blocked by it at the moment so happy to wait until it's done right, it just makes debugging migrations more difficult.

[roji]

@BenHerron, are you sure you reinstalled the latest 3.0.1-beta0001? I published a new version today with the constructor necessary to deserialize. Note it's a bit tricky because every time I push a new commit to the 3.0.1 branch the same nuget version is released (3.0.1-beta0001), so you have to make sure you delete Npgsql from your nuget cache and get the latest from myget.

[roji]

From @BenHerron on August 18, 2015 22:4

@roji

My process was to roll back my code changes (using sourcetree) which causes the project to reset back to the 3.0.0 released package.

I than went into Nuget manager and upgraded to the 3.0.1-beta0001 version on the unstable feed. Which uninstalled the old versions and than installs the new one.

I wasn't aware of the cache, I will delete and retest now.

[roji]

From @BenHerron on August 18, 2015 22:23

@roji

Ok I now get the same error as you.

Exception calling "CreateInstanceFrom" with "8" argument(s): "Object reference not set to an instance of an object."
At <solutionpath>\packages\EntityFramework.6.0.0\tools\EntityFramework.psm1:698 char:5
+     $domain.CreateInstanceFrom(
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : NullReferenceException

[roji]

OK, thanks for all the testing. I'll try to investigate this further for 3.0.2.

[roji]

From @theroc on August 20, 2015 8:26

Further to the other stuff above, I thought I'd add my error to the mix:

This occurs in the same context when running a code first migration using the Update-Database command on the latest 3.0.1 nugget package:

System.Runtime.Serialization.SerializationException: Member 'errors' was not found. at System.Runtime.Serialization.SerializationInfo.GetElement(String name, Type& foundType) at System.Runtime.Serialization.SerializationInfo.GetValue(String name, Type type) at Npgsql.NpgsqlException..ctor(SerializationInfo info, StreamingContext context)

at System.AppDomain.DoCallBack(CrossAppDomainDelegate callBackDelegate) at System.Data.Entity.Migrations.Design.ToolingFacade.Run(BaseRunner runner) at System.Data.Entity.Migrations.Design.ToolingFacade.Update(String targetMigration, Boolean force) at System.Data.Entity.Migrations.UpdateDatabaseCommand.<>cDisplayClass2.<.ctor>b0() at System.Data.Entity.Migrations.MigrationsDomainCommand.Execute(Action command) Member 'errors' was not found.

[roji]

From @sparkyshark on August 22, 2015 12:12

I got the following exception on Npgsql 3.0.1 when try execute Update-Database:

Exception calling "CreateInstanceFrom" with "8" argument(s): "Object reference not set to an instance of an object." At ...packages\EntityFramework.6.1.3\tools\EntityFramework.psm1:809 char:5

• $domain.CreateInstanceFrom(

• 
  - CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
  - FullyQualifiedErrorId : NullReferenceException

[roji]

From @theroc on August 25, 2015 10:52

Ok, some more info folks.

Had to write a whole host of debug stuff in the Entity Framework powershell stuff to get to the root of this one and here it is:

System.NullReferenceException: Object reference not set to an instance of an object. at Npgsql.NpgsqlException.get_Message() at System.Data.Entity.Migrations.MigrationsDomainCommand.Throw(Exception ex) at System.Data.Entity.Migrations.MigrationsDomainCommand.Execute(Action command) at System.Data.Entity.Migrations.UpdateDatabaseCommand..ctor(String sourceMigration, String targetMigration, Boolean script, Boolean force, Boolean verbose)

[roji]

From @CezarCretu on August 25, 2015 11:6

Out of curiosity, can you provide some insight into what exactly you had to do to get the full stack trace?

[roji]

From @theroc on August 25, 2015 11:12

Sure,

1. Open the package folder for the EF 6.1.3 release
2. In the tools folder open the EntityFramework.psm1 script.

3. Between line 607-610 (It may differ as I have a bunch of other stuff in mine) in the Update-Database command add the following:

   Catch [system.exception] { Write-Verbose $_.Exception.ToString() throw }

4. Run a migration that contains a deliberate issue with the -Verbose command.

[roji]

From @theroc on August 25, 2015 11:19

It looks like the _msg field in the NpgsqlException is null at the get_message stage as it appears to only be set by the Internal constructor.

[roji]

Thanks for looking into this @theroc. It makes some sense: for 3.0 I redid NpgsqlException and did not take into account scenarios where it is serialized/deserialized, which is what happens with the EF6 scripts which transfer NpgsqlException instances across appdomain boundaries.

If you (or anyone else) wants to take a stab at fixing the issue(s) please go ahead - I'll be glad to accept a PR. If not I'll definitely try to fix this myself for 3.0.2.

[roji]

From @BenHerron on September 14, 2015 7:49

@roji Not sure that the title is correct, i'm not using EF7. This is issue is causing me lots of grief at the moment.

[roji]

@BenHerron, sorry, I forgot about this issue for a bit, promise to try to get it fixed ASAP for 3.0.3. I know it's EF6-related (and not EF7)

[roji]

From @BenHerron on September 14, 2015 8:5

@roji Thanks!

[roji]

This was actually pretty trivial, NpgsqlException was simply not handling serialization/deserialization properly. Sorry it took this long.

You can download the nuget for 3.0.3-beta.9 from our unstable feed right away (a confirmation that the bug is gone would be great).

[roji]

From @BenHerron on September 14, 2015 21:25

@roji I can't seem to find the 3.0.3-beta.9 is there something special i need to do to access it?

[roji]

Ah, seems like the 3.1.0 unstable release have taken over the entire feed. Will open a separate feed for 3.0.x changes.

[roji]

You should now be able to download 3.0.3-beta0010 from the new MyGet stable feed (for patch nugets): https://www.myget.org/F/npgsql/api/v2

[roji]

From @BenHerron on September 16, 2015 0:44

@roji running 3.0.3-beta0010 I get the following error.

Exception calling "CreateInstanceFrom" with "8" argument(s): "Object reference not set to an instance of an object."
At C:\development\project\packages\EntityFramework.6.0.0\tools\EntityFramework.psm1:698 char:5
+     $domain.CreateInstanceFrom(
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : NullReferenceException

[roji]

Very strange, will look into it

[roji]

@BenHerron don't forget you need to install the new Npgsql.dll in your GAC as well as in your nugget, can you confirm you did this and are still getting the error?

[roji]

From @BenHerron on September 16, 2015 21:46

@roji I hadn't installed to GAC. But have now done that and retested and still get same error.

[roji]

@BenHerron I just tested again with a virgin project with 3.0.3-beta0011 (should be the same as 3.0.3-beta0010), and the issue is gone for me.

There's also something weird with your experience - if I don't have Npgsql in my GAC I get:

Type is not resolved for member 'Npgsql.NpgsqlException,Npgsql, Version=3.0.3.0, Culture=neutral, PublicKeyToken=5d8b90d52f46fda7'.

So maybe some older version of Npgsql (without the fix) is lingering somewhere in your setup. I suggest you try to clean up until you get the above error, at which point you know there's no npgsql anywhere. Then you can install the 3.0.3-beta0011 Npgsql.dll in your GAC and retest.

Let me know!

[roji]

From @BenHerron on September 18, 2015 7:11

@roji I will retest in the next couple of days and get back to you.

[roji]

@BenHerron am pretty close to releasing 3.0.3 and it would be great to get confirmation that this bug is gone.

[roji]

From @BenHerron on September 23, 2015 5:54

Sorry been busy, will get on to it today.

[roji]

From @BenHerron on September 23, 2015 6:33

@roji Tested 3.0.3 Beta 11, everything looks good to me.

I removed permissions to the database and got the following error which is what I would expect.

Npgsql.NpgsqlException (0x80004005): 42501: permission denied for database "x"
   at Npgsql.NpgsqlConnector.DoReadSingleMessage(DataRowLoadingMode dataRowLoadingMode, Boolean returnNullForAsyncMessage, Boolean isPrependedMessage)
   at Npgsql.NpgsqlConnector.ReadSingleMessage(DataRowLoadingMode dataRowLoadingMode, Boolean returnNullForAsyncMessage)
   at Npgsql.NpgsqlConnector.HandleAuthentication()
   at Npgsql.NpgsqlConnector.Open()
   at Npgsql.NpgsqlConnectorPool.GetPooledConnector(NpgsqlConnection Connection)
   at Npgsql.NpgsqlConnectorPool.RequestConnector(NpgsqlConnection connection)
   at Npgsql.NpgsqlConnection.Open()
   at System.Data.Entity.Migrations.DbMigrator.ExecuteStatementsInternal(IEnumerable`1 migrationStatements, DbConnection connection)
   at System.Data.Entity.Migrations.DbMigrator.<>c__DisplayClass32.<ExecuteStatements>b__2e()
   at System.Data.Entity.Infrastructure.DefaultExecutionStrategy.Execute(Action operation)
   at System.Data.Entity.Migrations.DbMigrator.ExecuteStatements(IEnumerable`1 migrationStatements)
   at System.Data.Entity.Migrations.Infrastructure.MigratorBase.ExecuteStatements(IEnumerable`1 migrationStatements)
   at System.Data.Entity.Migrations.DbMigrator.ExecuteOperations(String migrationId, XDocument targetModel, IEnumerable`1 operations, IEnumerable`1 systemOperations, Boolean downgrading, Boolean auto)
   at System.Data.Entity.Migrations.DbMigrator.ApplyMigration(DbMigration migration, DbMigration lastMigration)
   at System.Data.Entity.Migrations.Infrastructure.MigratorLoggingDecorator.ApplyMigration(DbMigration migration, DbMigration lastMigration)
   at System.Data.Entity.Migrations.DbMigrator.Upgrade(IEnumerable`1 pendingMigrations, String targetMigrationId, String lastMigrationId)
   at System.Data.Entity.Migrations.Infrastructure.MigratorLoggingDecorator.Upgrade(IEnumerable`1 pendingMigrations, String targetMigrationId, String lastMigrationId)
   at System.Data.Entity.Migrations.DbMigrator.UpdateInternal(String targetMigration)
   at System.Data.Entity.Migrations.DbMigrator.<>c__DisplayClassc.<Update>b__b()
   at System.Data.Entity.Migrations.DbMigrator.EnsureDatabaseExists(Action mustSucceedToKeepDatabase)
   at System.Data.Entity.Migrations.Infrastructure.MigratorBase.EnsureDatabaseExists(Action mustSucceedToKeepDatabase)
   at System.Data.Entity.Migrations.DbMigrator.Update(String targetMigration)
   at System.Data.Entity.Migrations.Infrastructure.MigratorBase.Update(String targetMigration)
   at System.Data.Entity.Migrations.Design.ToolingFacade.UpdateRunner.Run()
   at System.AppDomain.DoCallBack(CrossAppDomainDelegate callBackDelegate)
   at System.AppDomain.DoCallBack(CrossAppDomainDelegate callBackDelegate)
   at System.Data.Entity.Migrations.Design.ToolingFacade.Run(BaseRunner runner)
   at System.Data.Entity.Migrations.Design.ToolingFacade.Update(String targetMigration, Boolean force)
   at System.Data.Entity.Migrations.UpdateDatabaseCommand.<>c__DisplayClass2.<.ctor>b__0()
   at System.Data.Entity.Migrations.MigrationsDomainCommand.Execute(Action command)
42501: permission denied for database "x"

[roji]

Great, thanks for confirming!

[roji]

From @andreypesoshin on December 17, 2015

Hi guys! Just updated to EntityFramework6.Npgsql 3.0.4.0. Unfortunately still can see no pgsql exception details:

PM> Update-Database
Specify the '-Verbose' flag to view the SQL statements being applied to the target database.
System.Runtime.Serialization.SerializationException: Type is not resolved for member 'Npgsql.NpgsqlException,Npgsql, Version=3.0.4.0, Culture=neutral, PublicKeyToken=5d8b90d52f46fda7'.
   at System.AppDomain.DoCallBack(CrossAppDomainDelegate callBackDelegate)
   at System.Data.Entity.Migrations.Design.ToolingFacade.Run(BaseRunner runner)
   at System.Data.Entity.Migrations.Design.ToolingFacade.Update(String targetMigration, Boolean force)
   at System.Data.Entity.Migrations.UpdateDatabaseCommand.<>c__DisplayClass2.<.ctor>b__0()
   at System.Data.Entity.Migrations.MigrationsDomainCommand.Execute(Action command)
Type is not resolved for member 'Npgsql.NpgsqlException,Npgsql, Version=3.0.4.0, Culture=neutral, PublicKeyToken=5d8b90d52f46fda7'.

[roji]

@andreypesoshin sorry it took so long to respond.

Can you confirm you have Npgsql installed in your GAC?