Best way to migrate large amount of issues

[YoavLax]

Hi,

Would like to hear what would be the most recommended way yo migrate a large amount of issues. I see there is the ability to split the export like explained here: https://github.com/solidify/jira-azuredevops-migrator/blob/master/docs/faq.md#15-how-to-limit-the-number-of-issues-to-be-exported-during-jira-export-pagination

Which indeed could be useful and save time on the export side, but how would this affect in the import? If for example i split in to 5 paginations, how would i proceed with the import? In parallel? Wouldn't this cause some data to get missed? Or on the import phase we should go one after the other?

Just looking for the best way to save some time during the migration and in addition to ensure all the data stays safe.

[Alexander-Hjelm]

In theory, the import should not be affected if you do one batch after the other and follow the guidelines here: https://github.com/solidify/jira-azuredevops-migrator/blob/master/docs/faq.md#scenario-2-multiple-projects

The links would be skipped on the first pass, but set up properly on the second pass, once both the source and target work items are present in the target project.

[YoavLax]

So just to verify i understand the process, the recommendation will be:

Export:

1. Set "download-options": 0
2. Force = true
3. Export each config into a separate workspaces in parallel

Import:

1. Import one workspace at a time
2. DON'T use force
3. After every import completes, copy the itemsJournal.txt from one workspace to the next, this will ensure mappings of the items and will continuously get updated on every import.

Did i get it right? Anything missed?

[Alexander-Hjelm]

That sounds about right!

[YoavLax]

Hey,

Following the process above, Do you possibly have an idea why i'm getting these errors in the import?

[I][15:09:11] Processing 5614/147165 - wi '1853211', jira 'TST-10220, rev 5' [E][15:09:11] Input string was not in a correct format. [W][15:09:11] ''*****-13729', rev 9' - not all changes were saved.

The only thing that this revision contains is a relation addition to an item that has already been imported and exists in the Azure DevOps. { "Author": "", "Time": "", "Index": 9, "Fields": [], "Links": [ { "Change": 0, "TargetOriginId": "****-4508", "TargetWiId": 0, "WiType": "System.LinkTypes.Hierarchy-Reverse" } ], "Attachments": [], "DevelopmentLink": null, "AttachmentReferences": false },

This causes many links to get skipped and then end result is that the items don't have the relations.

[Alexander-Hjelm]

Looks like maybe the itemsJournal file was corrupted? Check it and verify that there are no empty lines and that there are no other weird things going on.

[YoavLax]

I see this warning in the import log: [W][20:54:23] Detected unicode characters, removed. [W][20:54:23] Detected unicode characters, removed. [W][20:54:27] Detected unicode characters, removed.

Could be related? Though i don't see any corruption in the itemsjournal .

In addition, i checked if the item that it's trying to add in that revision appears inside the itemsjournal and it's indeed there.

[Alexander-Hjelm]

Should be fine. The unicode character warning comes from this PR: https://github.com/solidify/jira-azuredevops-migrator/pull/706, which solved another error related to characters that could not be properly parsed.

[YoavLax]

So any thoughts why this is happening? What could be the issue leading to this error? We see that most links are migrated properly, but we see many errors like described above which are failing to link.

Worth mentioning the itemsjournal is over 100K lines during the process run.

[Alexander-Hjelm]

Not sure straight away. Here is the line which is logging the above error:

Logger.Log(LogLevel.Info, $"Processing {importedItems + 1}/{revisionCount} - wi '{(wi.Id > 0 ? wi.Id.ToString() : "Initial revision")}', jira '{executionItem.OriginId}, rev {executionItem.Revision.Index}'.");

See: https://github.com/solidify/jira-azuredevops-migrator/blob/master/src/WorkItemMigrator/WorkItemImport/ImportCommandLine.cs#L131

I am not sure, but it seems like the expression (wi.Id > 0 ? wi.Id.ToString() : "Initial revision") results in an empty string? Any idea why that could be?

[YoavLax]

I'm sorry, the ID was mistakenly deleted. It does exist, like here: Processing 5614/147165 - wi '1853211', jira 'TST-10220, rev 5'

I think it's failing here: https://github.com/solidify/jira-azuredevops-migrator/blob/304e6e2f5692e6211ab6b40e154f0702fdc670c3/src/WorkItemMigrator/WorkItemImport/Agent.cs#L91 Which means for some reason the incomplete = true Which is probably happening here: https://github.com/solidify/jira-azuredevops-migrator/blob/304e6e2f5692e6211ab6b40e154f0702fdc670c3/src/WorkItemMigrator/WorkItemImport/Agent.cs#L643

Probably failure is here: https://github.com/solidify/jira-azuredevops-migrator/blob/304e6e2f5692e6211ab6b40e154f0702fdc670c3/src/WorkItemMigrator/WorkItemImport/Agent.cs#L651C55-L651C68

Which is related to the itemsjournal.txt :(

[YoavLax]

Seems like it's failing here: int sourceWiId = _context.Journal.GetMigratedId(link.SourceOriginId);

Cause if it was failing here: int targetWiId = _context.Journal.GetMigratedId(link.TargetOriginId);

It would have gotten into : if (link.TargetWiId == -1) { var errorLevel = Settings.IgnoreFailedLinks ? LogLevel.Warning : LogLevel.Error; Logger.Log(errorLevel, $"'{link}' - target work item for Jira '{link.TargetOriginId}'" + $" is not yet created in Azure DevOps/TFS. You can safely ignore this warning if" + $" this work item is scheduled for import later in your migration."); success = false; continue; }

[Alexander-Hjelm]

Ok! I see now that the Author and Datetime stamp of the issue has not been set, which indicates an internal failure in the Exporter. Did the JiraExport result in any obvious errors? Check the log maybe.

[YoavLax]

They have also been set, but removed so no internal data is published. Sorry for misleading.

[YoavLax]

Hi @Alexander-Hjelm ,

Updating we have managed to solve the issue by one small change.

In the export section, instead of setting "download-options": 0, we have set it to 7. Then, copied itemsjournal and Imported without force. This way it ensures that the migration passes including all its relations.

Thank you so much for your assistance!