Losing Lookup References Upon Moving Portal Records

[frick-brian]

This seemingly just happened with my most recent update, as I have been using Portal Records mover for a while now without any issue.

XRMTB: v2020.5.39 Portal Records Mover: v1.2020.4.19

So, if I move something, like say a Web Page, Page Template, or Web Template, the import fires without any errors or warnings. But when I go into CRM to verify, the content pages are not being associated properly to their parents, or the web pages do not have the website or template filled in - even if the objects already exist in the target system.

This is obviously causing all types of issues with our deployments. Wondering if you've seen/heard of this, and is there any way to fix it? After the most recent failure, we ended up pushing almost everything available in Portal Records MOver over, and it resolved most of the issues. Then, I had to patch one more webpage over (root and content), and both items were moved, but the content page was not associated to the parent. I had to re-move them, and on this subsequent run, the content was properly associated to the parent.

Just seems inconsistent at best - which is obviously frustrating. I will gladly provide more info or do some further test to help get to the bottom of this. Thanks for all of your hard work on this excellent product.

[MscrmTools]

Hi. Thanks for reporting. I will try to find time to look at this. In the mean time, can you check the logs to see if lookup association occurs since the tool is importing data without lookup then update lookups.

[frick-brian]

Thanks for the quick response and sorry for my delay. Here is a ZIP with a bunch of log files from 5/18 through 5/20. Some of these runs did have errors - but even runs with no errors and 'all green' text in the process window were causing the symptoms listed above. The wrong associations were happening on 5/19 and 5/20, as far as I know. Thanks - again please let me know if you need any further information and I will be glad to provide it.

PortalRecordsLogs.zip

[frick-brian]

@MscrmTools I just did a little log digging myself. So, as stated above, one of the two environments was migrated to perfectly fine, the other was not. I did both migrations without backing out of XRMTB or Portal REcords Mover, and did them within 30 seconds of each other upon the first successful completion.

So, I went into the log files for 5/19 at 5:12PM and 5:13PM (included in my zip). Here is the compare of the two log files. You can see that the 'failed env' did not execute the update to the M:M relationships - I wonder if this was the failure point? And if it is, why would it happen in one environment and not the other?

I will also say, after my latest updates to XRMTB, I am getting strange results in 'Retrieve Records' - in such that if I have to re-run a query, I do not get new results, or I do not see results that I know should be there (and verify by advanced find'ing them in CRM). The only fix is to kill the PRM instance, and then re-connect to PRM, re run the query, and all items show up without changing a single item in CRM. I do not know if this is related to the issue I am currently tracking, or if this is just some unrelated caching issue with XRMTB/PRM.

Thanks again!

[MscrmTools]

Point is: if one env is updated correctly and the other is not, then there should have been an outage when it did not work as expected. I don’t see how PRM could be the issue. Can you check portal solutions to see if both environnements are aligned ?

[frick-brian]

Is there a specific Portal solution you want the version# to? I have a whole host of them installed on both instances. Or, did you want me to grab something out of the D365 Admin Center rather than Adv Settings > Solutions?

[frick-brian]

The version from D365 Admin Center (Instances > Select Instance > Solutions) for the Customer Self Service portal is 9.1.9.0 in both environments. Both are upgrade eligible.

[MscrmTools]

I don’t really know what I am looking for... trying to understand what differences might exist between both envs

[frick-brian]

The environments are as close to complete clones to one another as is possible. They are my client UAT envrionments, to which we push Solutions and data (including portal records) from our QA environment.

What jumped out to me in those logs was there was no processing of the M:M relationships in the 'failed env' while the 'good env' had those events logged. I am not sure why that was the case, but it matches up with my theory that something silly happened on the 'failed env' - whether that is PRM related or not, I cannot be totally sure. AFAIK, there were no service disruptions to either environment - they are both GCC environments, so any disruption should have impacted both, I'd imagine.

I will keep looking into my logs and maybe find something else that could help us nail it down. Again, appreciate your efforts on the tooling, as well as the bugs!

[frick-brian]

Just wanted to show something I noticed while I was doing another PRM run today...

Initial Load of Items/Retrieve of Records

Good Deployment

After running PRM “transfer records”, come back in and ‘retrieve records’ – I go from 42 records to 41, and one has the Web Form blanked out.

When I kill PRM (leave XRMTB open), and then come back in and run the same query, I get 42 records, and the webform is populated – just like the first screenshot.

In my error scenario in this issue, I had migrated the second environment (FAIL ENV in screenshot) directly after doing the first, without killing PRM – which I have never had to do in the past.

[MscrmTools]

Thank you for the additional info! I will then check if there is some remaining cache data I should clear And I need to check if the connections are not inverted, because if you don’t get the same record, then it might be the target service that is used to retrieve records the second time

[MscrmTools]

I think I found the issue. Each time the connection was changed, classes that use an IOrganizationService were updated with the new service, which was generally the target service. I fixed it to update these classes only if they are not initialized with an IOrganizationService.

Could you test again with this updated assembly?

MscrmTools.PortalRecordsMover.zip

[frick-brian]

Sorry for the delay - other aspects of the project to work on between deployments, of course!

I will be doing some PRM work between tomorrow and Monday - and getting a jump start on it tonight. Probably a noob question, but how do I update the DLL for a tool in XRMTB? And how do I know if I am running the updated version through the UI (or code)? I tried to just drop it in my XRMTB folder, but I have no idea if it updated anything or not.

I am only doing one environment tonight, so I probably won't see the caching issue until downstream in my deployment process. Thanks again for keeping on this - hopefully I can help you resolve it.

[MscrmTools]

Go to settings, Paths tab, then click on storage link. You should find a Plugins folder. Paste the dll here and confirm file replacement

[frick-brian]

I figured it had to be something like that - thank you.

I will report back my findings NLT Monday. Thanks!

[frick-brian]

Hello @MscrmTools - looks like the new release in XRMTB did the trick - had you already deployed that DLL?

Man, did we have a fun time undoing all of the problems that cache issue caused! It corrupted our Sign In Page, which we had been bypassing for a while now, but we were doing some enhancements, and getting all kinds of ADX plugin errors when clicking sign in. After painstakingly comparing Site Settings as well as portal records, we decided to blow the entire portal away and re-push the files using PRM.

After that, sign in page worked a charm.

Thanks again for the attention and quick replies here. I will continue to let you know if we find any further bugs!

[MscrmTools]

Thank you for the feedback! This change is not yet deployed but I will do it tomorrow.