search

nkdAgility / azure-devops-migration-tools

Azure DevOps Migration Tools allow you to migrate Teams, Backlogs, Work Items, Tasks, Test Cases, and Plans & Suits from one Project to another in Azure DevOps / TFS both within the same Organisation, and between Organisations.
https://nkdagility.com/learn/azure-devops-migration-tools/
GNU Affero General Public License v3.0
500 stars 332 forks source link

Exception when converting to different organisation #717

Closed rdhaan1974 closed 3 years ago

rdhaan1974 commented 4 years ago

Describe your issue:

I'm using the Migration tool to migrate our project to a new organisation (we have a new owner for our organisation which also has an existing DevOps organisation, so its 'moving' project.

I've tested it with a testproject in our own organisation and made the configurations. This went well. Now I'm doing it in a test project in our new organisation and with the tasks I'm getting wierd errors. When restarting it wortks ok for the item, but the next one might fail. With almost 19000 to do, I have no desire to restart it every time.

Describe any Exceptions:

System.FormatException: The string 'Microsoft.TeamFoundation.WorkItemTracking.Common.ServerDefaultFieldValue' is not a valid AllXsd value. at System.Xml.Schema.XsdDateTime..ctor(String text, XsdDateTimeFlags kinds) at System.Xml.XmlConvert.ToDateTime(String s, XmlDateTimeSerializationMode dateTimeOption) at Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItemFieldData.ConvertFromXml(String value, Type type) at Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItemFieldData.TakeUpdateResult(XmlElement result, Int32 newRev) at Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItem.TakeUpdateResult(XmlElement result) at Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItemStore.SaveWorkItems(WorkItem[] workitems, Boolean bulk, SaveFlags saveFlags) at Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItem.Save(SaveFlags saveFlags) at MigrationTools.TfsExtensions.SaveToAzureDevOps(WorkItemData context) in D:\a\1\s\src\MigrationTools.Clients.AzureDevops.ObjectModel\TfsExtensions.cs:line 49 at VstsSyncMigrator.Engine.WorkItemMigrationContext.ReplayRevisions(List`1 revisionsToMigrate, WorkItemData sourceWorkItem, WorkItemData targetWorkItem, Int32 current) in D:\a\1\s\src\VstsSyncMigrator.Core\Execution\MigrationContext\WorkItemMigrationContext.cs:line 599

When migrating the parent link from the task to the (already migrated PBI) the GitRepositoryEnricher tries to update the item or link (not sure what exactly). One of the fields is expecting a date value, but gets the string 'Microsoft.TeamFoundation.WorkItemTracking.Common.ServerDefaultFieldValue' which I think comes from the server? But strange, next run the item is succesfully processed and then the next crashes.

Source Details

Target Details

I have completed the following:

Step 6 MAM debug.json.txt

migration.log

MrHinsh commented 4 years ago

I have added some additional logging code, but this is an error in the TFS core. Lets see if we can get closer to the error.

You may need to open the code in Visual Studio and debug to get a reall handle.

rdhaan1974 commented 4 years ago

Hello, I've run with the 11.6.44, but see no change in the logfile yet. Also, the GIT get worked, but trying to debug gives some errors. Still working on that... migration.log

rdhaan1974 commented 4 years ago

I've rerun this morning with 11.6.46 with the extra logging, but is still crashes. See the logfile: migration.log

I've tried debugging with Visual Studio 2019, but I can't get it to work (yet). A complete (re)build of the solution works however.

MrHinsh commented 4 years ago

Hopefully the next push will fix it.

rdhaan1974 commented 4 years ago

Hello,

The version 11.7.2 started laast night and runned perfect. All items are now in the new organisation test project. MigrationMAM.log

czechdude commented 3 years ago

I disagree, the migration does not run, because it requires newly ReflectedWorkItemId on the some weird Shared Steps WorkItemType. So I fixed it by editing the WIQLQueryBit by adding AND [System.WorkItemType] NOT IN ( 'Shared Steps','Shared Parameter' ) but still didnt work so I updated to 11.7.7 but there is some invalid field cast from Int to string in WorkItemData.cs and since 11.8.0 the configs are so messy that I broke down and cried.

MrHinsh commented 3 years ago

@czechdude there is no requirement for any modification on the Source. There has not been that need for at least all of v11, if not before if you did not set "UpdateSource".

Please note that the only supported version is the latest, which is currently v11.9.47!

hi2apurv commented 2 years ago

I am having the same isssue:

[20:30:52 DBG] TfsExtensions::SaveToAzureDevOps: ChangedBy: Microsoft.VisualStudio.Services.TFS, AuthorisedBy: Apurwa Kumar [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | ...FAILED to Save [20:30:52 INF] =============================================================== [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | Custom.ATPID (ATP ID) | [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | Custom.PlannedEndDate (Planned End Date) | null [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | Custom.PlannedStartDate (Planned Start Date) | null [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | Custom.ReflectedWorkItemId (ReflectedWorkItemId) | https://dev.azure.com/MetLife-US/PMACS2/_workitems/edit/390169 [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | Microsoft.VSTS.Common.AcceptanceCriteria (Acceptance Criteria) |

INT/QA Azure SQL server is added to resource group - "RG_11166_PMACS2_INT"
[20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | Microsoft.VSTS.Scheduling.FinishDate (Finish Date) | null [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | Microsoft.VSTS.Scheduling.StartDate (Start Date) | null [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | Microsoft.VSTS.Common.Risk (Risk) | [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | Microsoft.VSTS.Build.IntegrationBuild (Integration Build) | [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | Microsoft.VSTS.Common.ValueArea (Value Area) | Business [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | Microsoft.VSTS.Common.StackRank (Stack Rank) | null [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | Microsoft.VSTS.Common.Priority (Priority) | 2 [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | Microsoft.VSTS.Common.ClosedBy (Closed By) | [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | Microsoft.VSTS.Common.ClosedDate (Closed Date) | null [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | Microsoft.VSTS.Common.ResolvedReason (Resolved Reason) | [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | Microsoft.VSTS.Common.ResolvedBy (Resolved By) | [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | Microsoft.VSTS.Common.ResolvedDate (Resolved Date) | null [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | Microsoft.VSTS.Common.ActivatedBy (Activated By) | [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | Microsoft.VSTS.Common.ActivatedDate (Activated Date) | null [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | Microsoft.VSTS.Common.StateChangeDate (State Change Date) | 09/13/2022 20:30:55 [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | Microsoft.VSTS.Scheduling.StoryPoints (Story Points) | null [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | System.BoardLane (Board Lane) | [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | System.BoardColumnDone (Board Column Done) | null [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | System.BoardColumn (Board Column) | [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | System.Tags (Tags) | [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | System.RelatedLinkCount (Related Link Count) | 0 [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | System.History (History) | [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | System.Description (Description) |
Add development Azure SQL Server to development resource group - "RG_11166_PMACS2_INT".
SQL server needs to added from Azure portal using following steps :
1.Type or search SQL Database.
2.Under basic tab Subscription : Select Respective RG
3.Give Database name
4.Server , Create new
5.Compute + Storage - Provide as per Requirement.
6.Review + Create SQL database server.
[20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | System.CreatedBy (Created By) | Swatantra Tripathi [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | System.CreatedDate (Created Date) | 12/09/2021 03:05:39 [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | System.WorkItemType (Work Item Type) | User Story [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | System.AssignedTo (Assigned To) | [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | System.Reason (Reason) | New [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | System.ChangedBy (Changed By) | Microsoft.VisualStudio.Services.TFS [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | System.Rev (Rev) | 4 [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | System.Watermark (Watermark) | 153634 [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | System.AuthorizedDate (Authorized Date) | 09/13/2022 20:30:55 [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | System.State (State) | Funnel [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | System.Title (Title) | Add INT/QA Azure SQL Server [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | System.AuthorizedAs (Authorized As) | Apurwa Kumar [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | System.AreaId (Area ID) | 1803 [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | System.Id (ID) | 22765 [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | System.ChangedDate (Changed Date) | 01/22/2022 05:32:16 [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | System.RevisedDate (Revised Date) | 01/01/9999 00:00:00 [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | System.AreaPath (Area Path) | MLH TEST\Digital\Cloud Migration [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | System.NodeName (Node Name) | Cloud Migration [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | System.AttachedFileCount (Attached File Count) | 0 [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | System.HyperLinkCount (Hyperlink Count) | 0 [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | System.CommentCount (Comment Count) | null [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | System.RemoteLinkCount (Remote Link Count) | null [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | System.Parent (Parent) | null [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | System.TeamProject (Team Project) | MLH TEST [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | System.ExternalLinkCount (External Link Count) | 0 [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | System.IterationId (Iteration ID) | 1477 [20:30:52 INF] [ User Story][Complete: 1/726][sid:390169|Rev: 9][tid: null | System.IterationPath (Iteration Path) | MLH TEST [20:30:52 INF] =============================================================== [20:30:52 ERR] System.FormatException: The string 'Microsoft.TeamFoundation.WorkItemTracking.Common.ServerDefaultFieldValue' is not a valid AllXsd value. at System.Xml.Schema.XsdDateTime..ctor(String text, XsdDateTimeFlags kinds) at System.Xml.XmlConvert.ToDateTime(String s, XmlDateTimeSerializationMode dateTimeOption) at Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItemFieldData.ConvertFromXml(String value, Type type) at Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItemFieldData.TakeUpdateResult(XmlElement result, Int32 newRev) at Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItem.TakeUpdateResult(XmlElement result) at Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItemStore.SaveWorkItems(WorkItem[] workitems, Boolean bulk, SaveFlags saveFlags) at Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItem.Save(SaveFlags saveFlags) at MigrationTools.TfsExtensions.SaveToAzureDevOps(WorkItemData context) in D:\a\1\s\src\MigrationTools.Clients.AzureDevops.ObjectModel\TfsExtensions.cs:line 76 at VstsSyncMigrator.Engine.WorkItemMigrationContext.ReplayRevisions(List`1 revisionsToMigrate, WorkItemData sourceWorkItem, WorkItemData targetWorkItem) in D:\a\1\s\src\VstsSyncMigrator.Core\Execution\MigrationContext\WorkItemMigrationContext.cs:line 565 [20:30:52 INF] =============================================================== [20:30:52 DBG] TfsExtensions::SaveToAzureDevOps [20:30:52 DBG] TfsExtensions::SaveToAzureDevOps: ChangedBy: Microsoft.VisualStudio.Services.TFS, AuthorisedBy: Apurwa Kumar [20:30:52 ERR] System.FormatException: The string 'Microsoft.TeamFoundation.WorkItemTracking.Common.ServerDefaultFieldValue' is not a valid AllXsd value. at System.Xml.Schema.XsdDateTime..ctor(String text, XsdDateTimeFlags kinds) at System.Xml.XmlConvert.ToDateTime(String s, XmlDateTimeSerializationMode dateTimeOption) at Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItemFieldData.ConvertFromXml(String value, Type type) at Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItemFieldData.TakeUpdateResult(XmlElement result, Int32 newRev) at Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItem.TakeUpdateResult(XmlElement result) at Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItemStore.SaveWorkItems(WorkItem[] workitems, Boolean bulk, SaveFlags saveFlags) at Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItem.Save(SaveFlags saveFlags) at MigrationTools.TfsExtensions.SaveToAzureDevOps(WorkItemData context) in D:\a\1\s\src\MigrationTools.Clients.AzureDevops.ObjectModel\TfsExtensions.cs:line 76 at VstsSyncMigrator.Engine.WorkItemMigrationContext.d33.MoveNext() in D:\a\1\s\src\VstsSyncMigrator.Core\Execution\MigrationContext\WorkItemMigrationContext.cs:line 408 System.FormatException: The string 'Microsoft.TeamFoundation.WorkItemTracking.Common.ServerDefaultFieldValue' is not a valid AllXsd value. at System.Xml.Schema.XsdDateTime..ctor(String text, XsdDateTimeFlags kinds) at System.Xml.XmlConvert.ToDateTime(String s, XmlDateTimeSerializationMode dateTimeOption) at Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItemFieldData.ConvertFromXml(String value, Type type) at Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItemFieldData.TakeUpdateResult(XmlElement result, Int32 newRev) at Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItem.TakeUpdateResult(XmlElement result) at Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItemStore.SaveWorkItems(WorkItem[] workitems, Boolean bulk, SaveFlags saveFlags) at Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItem.Save(SaveFlags saveFlags) at MigrationTools.TfsExtensions.SaveToAzureDevOps(WorkItemData context) in D:\a\1\s\src\MigrationTools.Clients.AzureDevops.ObjectModel\TfsExtensions.cs:line 76 at VstsSyncMigrator.Engine.WorkItemMigrationContext.d33.MoveNext() in D:\a\1\s\src\VstsSyncMigrator.Core\Execution\MigrationContext\WorkItemMigrationContext.cs:line 408 [20:30:52 INF] DONE in 00:06:30.5854535 [20:30:52 FTL] Error while running WorkItemMigration System.AggregateException: One or more errors occurred. ---> System.FormatException: The string 'Microsoft.TeamFoundation.WorkItemTracking.Common.ServerDefaultFieldValue' is not a valid AllXsd value. at VstsSyncMigrator.Engine.WorkItemMigrationContext.d33.MoveNext() in D:\a\1\s\src\VstsSyncMigrator.Core\Execution\MigrationContext\WorkItemMigrationContext.cs:line 453 --- End of inner exception stack trace --- at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions) at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken) at VstsSyncMigrator.Engine.WorkItemMigrationContext.InternalExecute() in D:\a\1\s\src\VstsSyncMigrator.Core\Execution\MigrationContext\WorkItemMigrationContext.cs:line 155 at MigrationTools._EngineV1.Processors.MigrationProcessorBase.Execute() in D:\a\1\s\src\MigrationTools_EngineV1\Processors\MigrationProcessorBase.cs:line 47 ---> (Inner Exception #0) System.FormatException: The string 'Microsoft.TeamFoundation.WorkItemTracking.Common.ServerDefaultFieldValue' is not a valid AllXsd value. at VstsSyncMigrator.Engine.WorkItemMigrationContext.d33.MoveNext() in D:\a\1\s\src\VstsSyncMigrator.Core\Execution\MigrationContext\WorkItemMigrationContext.cs:line 453<--- [20:30:52 ERR] WorkItemMigration The Processor MigrationEngine entered the failed state...stopping run [20:30:52 INF] Application is shutting down... [20:30:52 DBG] Hosting stopping [20:30:52 DBG] Exiting with return code: 0 [20:30:52 INF] Terminating: Application forcebly closed.

hi2apurv commented 2 years ago

When i am going for a single work item or the complete Epic, it is working fine, but when i am using the path filter it is giving me the above error. Any help on this will be really appreciated. its critical

hi2apurv commented 2 years ago

@MrHinsh MrHinsh request you to please check.

hi2apurv commented 2 years ago

Source Process Template is Agile and the Target is Safe