nkdAgility / azure-devops-migration-tools

Azure DevOps Migration Tools allow you to migrate Teams, Backlogs, Tasks, Test Cases, and Plans & Suits from one Project to another in Azure DevOps / TFS both within the same Organisation, and between Organisations.
https://nkdagility.com/learn/azure-devops-migration-tools/
MIT License
490 stars 326 forks source link

[Bug] Sometimes the tool creates Empty workitems - no title, and set to new... no fields mapped #1961

Open MrHinsh opened 6 months ago

MrHinsh commented 6 months ago

I am trying to migrate around 1600 workitems and when the migration is completed, Its creating empty workitems. Why this happens and how can I compare which workitems these are to compare with source image

Originally posted by @madhuripawar2212 in https://github.com/nkdAgility/azure-devops-migration-tools/discussions/1955

We need someone with this issue to run a single specific work item with this issue in Visual Studio and try to diagnose this issue... we have been unable to replicate it.


HELP NEEDED - We are unable to replicate this! But it does happen!

Can someone with this error please run a specific work item that they find has this issue through the tool and do one of the following:

Option 1: Generate and attach Log - if you are non-technical this is your option. It might provide us with enough info to diagnose, but may only result in more asks.

  1. In the Config file change LogLevel to Debug.
  2. Find a work item ID that has this issue, and craft a query that only selects this work item.
  3. Delete any target copy of that work item.
  4. Then run the migration
  5. Comment here with the full Log attached

Option 2: Debug in Visual Studio - This option is best but requires some developer experience.

  1. Fork the Repo and Clone it to your local computer
  2. Open the Solution in Visual Studio (we use 2022) and do a Full Rebuild
  3. Edit the Launch Profile to point to your config.
    image
  4. Craft the query to only load the work item with the issue
  5. "Play" the Launch Profile that you created and investigate

Create a comment below with the results of your investigation!

madhuripawar2212 commented 6 months ago

I tried migrating less than 500 workitems at a time and it worked for me. When I try to migrate more than 1000, empty workitems are getting created

MrHinsh commented 6 months ago

Thats very odd @madhuripawar2212 as the number of items should be irrelevant to the issue.

madhuripawar2212 commented 6 months ago

How are you running the tool? - I am running the tool from local using Visual Studio code What is the specification of the computer? - What exactly this means? Is it behind a proxy? - No

MrHinsh commented 6 months ago

If it's working for batch sizes of 500 but not for 1000 then that would indicate a local computer issue.

We have run batches of 15000 with no issues and I'm interested in figuring out why this issue happens.

github-actions[bot] commented 5 months ago

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 10 days

github-actions[bot] commented 4 months ago

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 10 days

github-actions[bot] commented 3 months ago

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 10 days

FlexiGit commented 3 months ago

Hi MrHinsh,

I can reproduce the error in VS. What information do you need?

As far as I can see, the error arises when the TF library tries to load a revision of a work item with a type that doesn't exist anymore (The type of the work item has changed). And creating a mapping for the old type does not help.

At least I guess that's the cause. I'm not totally sure, if the Error in the logs and thus the exception does really produce the empty work items. Scratch that. I ran a query in the target project right before and right after the exception. And it very much indeed looks like that's the cause.

[17:36:08 ERR] [v15.0.4-0-local] Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItemTypeDeniedOrNotExistException: TF201077: The work item type  cannot be found. It may have been renamed or destroyed.
   at Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItem.get_Type()
   at Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItemFieldData.Microsoft.TeamFoundation.WorkItemTracking.Internals.IWorkItemOpenFieldDataHelper.SetLatestData(Dictionary`2 latestData)
   at Microsoft.TeamFoundation.WorkItemTracking.Internals.WorkItemHelper.LoadWorkItemFieldData(IRowSetCollectionHelper tables, IWorkItemOpenFieldDataHelper helper)
   at Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItem.LoadWorkItemFromRowSetInternal(Int32 rev, Nullable`1 asof, IWorkItemRowSets witem)
   at Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItem..ctor(WorkItemStore store, Int32 id, Int32 revision)
   at Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItemStore.GetWorkItem(Int32 id, Int32 revision)
   at MigrationTools.TfsExtensions.GetRevision(WorkItemData context, Int32 rev) in C:\GitRepos\azure-devops-migration-tools\src\MigrationTools.Clients.AzureDevops.ObjectModel\TfsExtensions.cs:line 35
   at VstsSyncMigrator.Engine.WorkItemMigrationContext.ReplayRevisions(List`1 revisionsToMigrate, WorkItemData sourceWorkItem, WorkItemData targetWorkItem) in C:\GitRepos\azure-devops-migration-tools\src\VstsSyncMigrator.Core\Execution\MigrationContext\WorkItemMigrationContext.cs:line 726
MrHinsh commented 3 months ago

It probably means that there is a casing mismatch for the mapping.... let me look and see....

FlexiGit commented 3 months ago

Could it help to recreate the missing work item type in the source project?

MrHinsh commented 3 months ago

All work item types from the source must exist or be mapped in the target.

At the moment there is certainly a casing issue with types that have "task" in the source and "Task" in the target.

github-actions[bot] commented 2 months ago

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 10 days