Closed bjarnef closed 8 months ago
On the existing live environment it had records in index, but I noticed on rebuild I get an error:
Umbraco 10.8.2 and Forms 10.5.2
@AndyButland have you noticed this before?
No, but the errors look to me that you are missing some database migrations. NodeId
is a new field added in 12.2 and Culture
was added in a minor before that. I'm not sure why your migrations haven't run. Maybe the logs will reveal something, or maybe you can compare to a clean install and manually apply the fields you are missing.
I tried deleting the migration for Umbraco Forms here:
after restart of application, it is re-added, but I got some log errors
I think it is related to https://github.com/umbraco/Umbraco.Forms.Issues/issues/1129
Microsoft.Data.SqlClient.SqlException (0x80131904): Cannot insert duplicate key row in object 'dbo.umbracoRelationType' with unique index 'IX_umbracoRelationType_name'. The duplicate key value is (Related Form).
The statement has been terminated.
at Microsoft.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
at Microsoft.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
at Microsoft.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
at Microsoft.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
at Microsoft.Data.SqlClient.SqlDataReader.TryConsumeMetaData()
at Microsoft.Data.SqlClient.SqlDataReader.get_MetaData()
at Microsoft.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString, Boolean isInternal, Boolean forDescribeParameterEncryption, Boolean shouldCacheForAlwaysEncrypted)
at Microsoft.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean isAsync, Int32 timeout, Task& task, Boolean asyncWrite, Boolean inRetry, SqlDataReader ds, Boolean describeParameterEncryptionRequest)
at Microsoft.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, TaskCompletionSource`1 completion, Int32 timeout, Task& task, Boolean& usedCache, Boolean asyncWrite, Boolean inRetry, String method)
at Microsoft.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method)
at Microsoft.Data.SqlClient.SqlCommand.ExecuteScalar()
at StackExchange.Profiling.Data.ProfiledDbCommand.ExecuteScalar() in C:\projects\dotnet\src\MiniProfiler.Shared\Data\ProfiledDbCommand.cs:line 322
at Umbraco.Cms.Infrastructure.Persistence.FaultHandling.FaultHandlingDbCommand.<ExecuteScalar>b__36_0()
at Umbraco.Cms.Infrastructure.Persistence.FaultHandling.FaultHandlingDbCommand.<>c__DisplayClass38_0`1.<Execute>b__0()
at Umbraco.Cms.Infrastructure.Persistence.FaultHandling.RetryPolicy.ExecuteAction[TResult](Func`1 func)
at Umbraco.Cms.Infrastructure.Persistence.FaultHandling.FaultHandlingDbCommand.Execute[T](Func`1 f)
at Umbraco.Cms.Infrastructure.Persistence.FaultHandling.FaultHandlingDbCommand.ExecuteScalar()
at NPoco.Database.<>c__DisplayClass297_0.<ExecuteScalarHelper>b__0()
at NPoco.Database.ExecutionHook[T](Func`1 action)
at NPoco.Database.ExecuteScalarHelper(DbCommand cmd)
at NPoco.Database.NPoco.IDatabaseHelpers.ExecuteScalarHelper(DbCommand cmd)
at NPoco.DatabaseTypes.SqlServerDatabaseType.ExecuteInsert[T](Database db, DbCommand cmd, String primaryKeyName, Boolean useOutputClause, T poco, Object[] args)
at NPoco.Database.InsertAsyncImp[T](PocoData pocoData, String tableName, String primaryKeyName, Boolean autoIncrement, T poco, Boolean sync)
at NPoco.AsyncHelper.RunSync[T](Task`1 task)
at NPoco.Database.Insert[T](String tableName, String primaryKeyName, Boolean autoIncrement, T poco)
at NPoco.Database.Insert[T](T poco)
at Umbraco.Cms.Infrastructure.Persistence.Repositories.Implement.RelationTypeRepository.PersistNewItem(IRelationType entity)
at Umbraco.Cms.Core.Cache.FullDataSetRepositoryCachePolicy`2.Create(TEntity entity, Action`1 persistNew)
at Umbraco.Cms.Infrastructure.Persistence.Repositories.Implement.EntityRepositoryBase`2.Save(TEntity entity)
at Umbraco.Cms.Core.Services.RelationService.Save(IRelationType relationType)
at Umbraco.Forms.Core.Migrations.V_12_2_0.AddRelationTypeBetweenContentAndForms.Migrate()
at Umbraco.Cms.Infrastructure.Migrations.MigrationBase.Run()
at Umbraco.Cms.Infrastructure.Migrations.MigrationPlanExecutor.RunMigration(Type migrationType, MigrationContext context)
at Umbraco.Cms.Infrastructure.Migrations.MigrationPlanExecutor.RunScopedMigration(Type migrationType, MigrationPlan plan)
at Umbraco.Cms.Infrastructure.Migrations.MigrationPlanExecutor.RunMigrationPlan(MigrationPlan plan, String fromState)
ClientConnectionId:caeddf39-523b-4ee3-b372-970d29192c72
Error Number:2601,State:1,Class:14
I added the Form Picker workaround to the page models, but strange enough I still get this even there is no relations for relation type umbForm
.
The log shows the following where the error occurs at Cannot insert duplicate key row in object 'dbo.umbracoRelationType' with unique index 'IX_umbracoRelationType_name'. The duplicate key value is (Related Form)
{"@t":"2023-12-20T13:44:50.6404777Z","@mt":"{StartMessage} [Timing {TimingId}]","StartMessage":"Starting unattended package migration for UmbracoForms","TimingId":"ecbe7c5","SourceContext":"Umbraco.Cms.Core.Logging.ProfilingLogger","ProcessId":30172,"ProcessName":"iisexpress","ThreadId":1,"ApplicationId":"e224b48e6c559816f174aac1467a3ec5843d9538","MachineName":"LAPTOP-BF","Log4NetLevel":"INFO "}
{"@t":"2023-12-20T13:44:50.6910236Z","@mt":"Starting '{MigrationName}'...","MigrationName":"UmbracoForms","SourceContext":"Umbraco.Cms.Infrastructure.Migrations.MigrationPlanExecutor","ProcessId":30172,"ProcessName":"iisexpress","ThreadId":1,"ApplicationId":"e224b48e6c559816f174aac1467a3ec5843d9538","MachineName":"LAPTOP-BF","Log4NetLevel":"INFO "}
{"@t":"2023-12-20T13:44:50.6910730Z","@mt":"At {OrigState}","OrigState":"{forms-init-state}","SourceContext":"Umbraco.Cms.Infrastructure.Migrations.MigrationPlanExecutor","ProcessId":30172,"ProcessName":"iisexpress","ThreadId":1,"ApplicationId":"e224b48e6c559816f174aac1467a3ec5843d9538","MachineName":"LAPTOP-BF","Log4NetLevel":"INFO "}
{"@t":"2023-12-20T13:44:50.6913755Z","@mt":"Execute {MigrationType}","MigrationType":"InstallUmbracoForms","SourceContext":"Umbraco.Cms.Infrastructure.Migrations.MigrationPlanExecutor","ProcessId":30172,"ProcessName":"iisexpress","ThreadId":1,"ApplicationId":"e224b48e6c559816f174aac1467a3ec5843d9538","MachineName":"LAPTOP-BF","Log4NetLevel":"INFO "}
{"@t":"2023-12-20T13:44:50.9815353Z","@mt":"At {OrigState}","OrigState":"7c7bc5ee-4c5b-42dc-9576-5ce6dfbddb8e","SourceContext":"Umbraco.Cms.Infrastructure.Migrations.MigrationPlanExecutor","ProcessId":30172,"ProcessName":"iisexpress","ThreadId":1,"ApplicationId":"e224b48e6c559816f174aac1467a3ec5843d9538","MachineName":"LAPTOP-BF","Log4NetLevel":"INFO "}
{"@t":"2023-12-20T13:44:50.9816099Z","@mt":"Execute {MigrationType}","MigrationType":"AddRecordCultureColumn","SourceContext":"Umbraco.Cms.Infrastructure.Migrations.MigrationPlanExecutor","ProcessId":30172,"ProcessName":"iisexpress","ThreadId":1,"ApplicationId":"e224b48e6c559816f174aac1467a3ec5843d9538","MachineName":"LAPTOP-BF","Log4NetLevel":"INFO "}
{"@t":"2023-12-20T13:44:50.9998518Z","@mt":"At {OrigState}","OrigState":"9f7e6fe6-bbd5-4b2b-8820-e9e0e36cc74c","SourceContext":"Umbraco.Cms.Infrastructure.Migrations.MigrationPlanExecutor","ProcessId":30172,"ProcessName":"iisexpress","ThreadId":1,"ApplicationId":"e224b48e6c559816f174aac1467a3ec5843d9538","MachineName":"LAPTOP-BF","Log4NetLevel":"INFO "}
{"@t":"2023-12-20T13:44:50.9999101Z","@mt":"Execute {MigrationType}","MigrationType":"AddRecordWorkflowAudit","SourceContext":"Umbraco.Cms.Infrastructure.Migrations.MigrationPlanExecutor","ProcessId":30172,"ProcessName":"iisexpress","ThreadId":1,"ApplicationId":"e224b48e6c559816f174aac1467a3ec5843d9538","MachineName":"LAPTOP-BF","Log4NetLevel":"INFO "}
{"@t":"2023-12-20T13:44:51.0049159Z","@mt":"At {OrigState}","OrigState":"1a8f0d04-9396-40a2-9423-39fc9ae3828f","SourceContext":"Umbraco.Cms.Infrastructure.Migrations.MigrationPlanExecutor","ProcessId":30172,"ProcessName":"iisexpress","ThreadId":1,"ApplicationId":"e224b48e6c559816f174aac1467a3ec5843d9538","MachineName":"LAPTOP-BF","Log4NetLevel":"INFO "}
{"@t":"2023-12-20T13:44:51.0049605Z","@mt":"Execute {MigrationType}","MigrationType":"AddRecordWorkflowAuditExecutionStage","SourceContext":"Umbraco.Cms.Infrastructure.Migrations.MigrationPlanExecutor","ProcessId":30172,"ProcessName":"iisexpress","ThreadId":1,"ApplicationId":"e224b48e6c559816f174aac1467a3ec5843d9538","MachineName":"LAPTOP-BF","Log4NetLevel":"INFO "}
{"@t":"2023-12-20T13:44:51.0231739Z","@mt":"At {OrigState}","OrigState":"6e692c5d-c670-4c34-af17-28d8dbf0dcd2","SourceContext":"Umbraco.Cms.Infrastructure.Migrations.MigrationPlanExecutor","ProcessId":30172,"ProcessName":"iisexpress","ThreadId":1,"ApplicationId":"e224b48e6c559816f174aac1467a3ec5843d9538","MachineName":"LAPTOP-BF","Log4NetLevel":"INFO "}
{"@t":"2023-12-20T13:44:51.0232316Z","@mt":"Execute {MigrationType}","MigrationType":"NoopMigration","SourceContext":"Umbraco.Cms.Infrastructure.Migrations.MigrationPlanExecutor","ProcessId":30172,"ProcessName":"iisexpress","ThreadId":1,"ApplicationId":"e224b48e6c559816f174aac1467a3ec5843d9538","MachineName":"LAPTOP-BF","Log4NetLevel":"INFO "}
{"@t":"2023-12-20T13:44:51.0246223Z","@mt":"At {OrigState}","OrigState":"5d84fee1-388c-4e5f-b98c-1e66947278f1","SourceContext":"Umbraco.Cms.Infrastructure.Migrations.MigrationPlanExecutor","ProcessId":30172,"ProcessName":"iisexpress","ThreadId":1,"ApplicationId":"e224b48e6c559816f174aac1467a3ec5843d9538","MachineName":"LAPTOP-BF","Log4NetLevel":"INFO "}
{"@t":"2023-12-20T13:44:51.0246657Z","@mt":"Execute {MigrationType}","MigrationType":"EnsureIndexOnFormFolderKey","SourceContext":"Umbraco.Cms.Infrastructure.Migrations.MigrationPlanExecutor","ProcessId":30172,"ProcessName":"iisexpress","ThreadId":1,"ApplicationId":"e224b48e6c559816f174aac1467a3ec5843d9538","MachineName":"LAPTOP-BF","Log4NetLevel":"INFO "}
{"@t":"2023-12-20T13:44:51.0408049Z","@mt":"At {OrigState}","OrigState":"22df962a-ae26-4bdd-b8fd-0513a9c636bf","SourceContext":"Umbraco.Cms.Infrastructure.Migrations.MigrationPlanExecutor","ProcessId":30172,"ProcessName":"iisexpress","ThreadId":1,"ApplicationId":"e224b48e6c559816f174aac1467a3ec5843d9538","MachineName":"LAPTOP-BF","Log4NetLevel":"INFO "}
{"@t":"2023-12-20T13:44:51.0408562Z","@mt":"Execute {MigrationType}","MigrationType":"AddRelationTypeBetweenContentAndForms","SourceContext":"Umbraco.Cms.Infrastructure.Migrations.MigrationPlanExecutor","ProcessId":30172,"ProcessName":"iisexpress","ThreadId":1,"ApplicationId":"e224b48e6c559816f174aac1467a3ec5843d9538","MachineName":"LAPTOP-BF","Log4NetLevel":"INFO "}
{"@t":"2023-12-20T13:44:51.1009245Z","@mt":"Exception ({InstanceId}).","@l":"Error","@x":"Microsoft.Data.SqlClient.SqlException (0x80131904): Cannot insert duplicate key row in object 'dbo.umbracoRelationType' with unique index 'IX_umbracoRelationType_name'. The duplicate key value is (Related Form).\r\nThe statement has been terminated.\r\n at Microsoft.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)\r\n at Microsoft.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)\r\n at Microsoft.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)\r\n at Microsoft.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)\r\n at Microsoft.Data.SqlClient.SqlDataReader.TryConsumeMetaData()\r\n at Microsoft.Data.SqlClient.SqlDataReader.get_MetaData()\r\n at Microsoft.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString, Boolean isInternal, Boolean forDescribeParameterEncryption, Boolean shouldCacheForAlwaysEncrypted)\r\n at Microsoft.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean isAsync, Int32 timeout, Task& task, Boolean asyncWrite, Boolean inRetry, SqlDataReader ds, Boolean describeParameterEncryptionRequest)\r\n at Microsoft.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, TaskCompletionSource`1 completion, Int32 timeout, Task& task, Boolean& usedCache, Boolean asyncWrite, Boolean inRetry, String method)\r\n at Microsoft.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method)\r\n at Microsoft.Data.SqlClient.SqlCommand.ExecuteScalar()\r\n at StackExchange.Profiling.Data.ProfiledDbCommand.ExecuteScalar() in C:\\projects\\dotnet\\src\\MiniProfiler.Shared\\Data\\ProfiledDbCommand.cs:line 322\r\n at Umbraco.Cms.Infrastructure.Persistence.FaultHandling.FaultHandlingDbCommand.<ExecuteScalar>b__36_0()\r\n at Umbraco.Cms.Infrastructure.Persistence.FaultHandling.FaultHandlingDbCommand.<>c__DisplayClass38_0`1.<Execute>b__0()\r\n at Umbraco.Cms.Infrastructure.Persistence.FaultHandling.RetryPolicy.ExecuteAction[TResult](Func`1 func)\r\n at Umbraco.Cms.Infrastructure.Persistence.FaultHandling.FaultHandlingDbCommand.Execute[T](Func`1 f)\r\n at Umbraco.Cms.Infrastructure.Persistence.FaultHandling.FaultHandlingDbCommand.ExecuteScalar()\r\n at NPoco.Database.<>c__DisplayClass297_0.<ExecuteScalarHelper>b__0()\r\n at NPoco.Database.ExecutionHook[T](Func`1 action)\r\n at NPoco.Database.ExecuteScalarHelper(DbCommand cmd)\r\n at NPoco.Database.NPoco.IDatabaseHelpers.ExecuteScalarHelper(DbCommand cmd)\r\n at NPoco.DatabaseTypes.SqlServerDatabaseType.ExecuteInsert[T](Database db, DbCommand cmd, String primaryKeyName, Boolean useOutputClause, T poco, Object[] args)\r\n at NPoco.Database.InsertAsyncImp[T](PocoData pocoData, String tableName, String primaryKeyName, Boolean autoIncrement, T poco, Boolean sync)\r\n at NPoco.AsyncHelper.RunSync[T](Task`1 task)\r\n at NPoco.Database.Insert[T](String tableName, String primaryKeyName, Boolean autoIncrement, T poco)\r\n at NPoco.Database.Insert[T](T poco)\r\n at Umbraco.Cms.Infrastructure.Persistence.Repositories.Implement.RelationTypeRepository.PersistNewItem(IRelationType entity)\r\n at Umbraco.Cms.Core.Cache.FullDataSetRepositoryCachePolicy`2.Create(TEntity entity, Action`1 persistNew)\r\n at Umbraco.Cms.Infrastructure.Persistence.Repositories.Implement.EntityRepositoryBase`2.Save(TEntity entity)\r\n at Umbraco.Cms.Core.Services.RelationService.Save(IRelationType relationType)\r\n at Umbraco.Forms.Core.Migrations.V_12_2_0.AddRelationTypeBetweenContentAndForms.Migrate()\r\n at Umbraco.Cms.Infrastructure.Migrations.MigrationBase.Run()\r\n at Umbraco.Cms.Infrastructure.Migrations.MigrationPlanExecutor.RunMigration(Type migrationType, MigrationContext context)\r\n at Umbraco.Cms.Infrastructure.Migrations.MigrationPlanExecutor.RunScopedMigration(Type migrationType, MigrationPlan plan)\r\n at Umbraco.Cms.Infrastructure.Migrations.MigrationPlanExecutor.RunMigrationPlan(MigrationPlan plan, String fromState)\r\nClientConnectionId:8a1bf9c6-aa89-441c-bda9-8eb4806ffd86\r\nError Number:2601,State:1,Class:14","InstanceId":"b9fa3c9c","SourceContext":"Umbraco.Cms.Infrastructure.Persistence.UmbracoDatabase","ProcessId":30172,"ProcessName":"iisexpress","ThreadId":1,"ApplicationId":"e224b48e6c559816f174aac1467a3ec5843d9538","MachineName":"LAPTOP-BF","Log4NetLevel":"ERROR"}
{"@t":"2023-12-20T13:44:51.1137512Z","@mt":"Plan failed at step {TargetState}","@l":"Error","TargetState":"c3e657f6-3ae7-4ee9-b442-01702a41de9a","SourceContext":"Umbraco.Cms.Infrastructure.Migrations.MigrationPlanExecutor","ProcessId":30172,"ProcessName":"iisexpress","ThreadId":1,"ApplicationId":"e224b48e6c559816f174aac1467a3ec5843d9538","MachineName":"LAPTOP-BF","Log4NetLevel":"ERROR"}
If I delete the migration and the umbForm
relation and restart application, I don't get this error and the relation it created again.
I guess Forms doesn't check if the relation already exists, before adding the relation with alias umbForm
?
Rebuilding the Forms index doesn't seem to add documents and I don't see any log errors.
The Forms table looks like this:
Actually I didn't have any forms records locally ... and submitted one and now I get the record in the index.
https://github.com/umbraco/Umbraco.Forms.Issues/assets/2919859/614be07f-50ef-4350-b270-73cc30de0c6d
and rebuilding the index also hide the loading indicator when finished.
But if I delete that record the index is also updated when 0 documents - but rebuilding index is just loading infinite.
https://github.com/umbraco/Umbraco.Forms.Issues/assets/2919859/6740991b-78bf-473c-bcd4-9835bcee7cd2
the logs only shows the index is being rebuilt and nothing further.
but when record was added and deleted it does.
@AndyButland I think there is two issues here
In this project we don't have any members and the index loading stops immediately after starting indexing.
I think there's a bottleneck somewhere when fetching data to UmbracoFormsRecordsIndex
?
https://github.com/umbraco/Umbraco.Forms.Issues/assets/2919859/85b9ef79-3fe7-4e4f-bbe1-afebca16e607
As of the next patch releases we'll have a fix in place to ensure the migration to create the relation type is idempotent. I'll close this issue as that's the part I've recognised as a problem that needs a fix, but there was quite a few further comments in here relating to migrations and indexing. I'm not sure if there are further issues being reported but I'd ask please if you think so, please can you create a new issue with replication steps? Thanks.
@AndyButland it was mainly the two issues in the bullets here I identified. https://github.com/umbraco/Umbraco.Forms.Issues/issues/1137#issuecomment-1864662313
If rebuilding UmbracoFormsRecordsIndex
without any forms entries, does it with latest changes stop the loading progress like it does for member index?
OK, thanks, I see it now. It seems we need to call the IIndex.IndexItems
method even if there's nothing to index. As without that the event handler that fires to indicate to the log and UI that the indexing is complete doesn't run. I'll update this for the next patch releases too.
No, but the errors look to me that you are missing some database migrations.
NodeId
is a new field added in 12.2 andCulture
was added in a minor before that. I'm not sure why your migrations haven't run. Maybe the logs will reveal something, or maybe you can compare to a clean install and manually apply the fields you are missing.
I got a similar issue with missing columns/migrations after upgrading Forms 12.1.1 -> 13.0 and Umbraco 12.2 -> 13.0. In the output I can se core v13 migrations running but no packagemigrations.
If I upgrade to Forms 12.2.1 and Umbraco 12.3.6 and after that upgrade to v13 everything works.
@AndyButland Hi! I just wanted to mention that I am still receiving the same error 'Index populating failed for populator "Umbraco.Forms.Examine.Indexes.FormsIndexPopulator"' even with the 13.0.1 release..
Maybe you can dig out some further information - e.g. full details of error from the logs, and replication steps - and create as a new issue please @JoellaAndersson? This issue ended up going "around the houses" a bit and the fixes applied weren't directly related to the error posted in the title.
So it will be easier to work with a new issue rather than re-opening this one.
I have upgraded an Umbraco project from 10.8.2 to 12.3.5 and upgraded Forms to latest version as well.
Everything seems to work, but I noticed log errros regarding forms index.
If I rebuild the index it seems to load infinite.
Umbraco 12.3.5 Forms 12.2.1
This item has been added to our backlog AB#36195