[CoE Starter Kit - BUG] MINOR Issue - Invalid character in field 'admin_environmentpurposetext'

[miroslav-harlas]

Does this bug already exist in our backlog?

• [X] I have checked and confirm this is a new bug.

Describe the issue

Hello, I have noticed that occasionally few Teams environments are not getting inventoried by our CoE starter Kit in our PROD tenant. After digging into Cloud Flows history, I have realized that there is an issue with special characters used on the MS Teams side, which prevents to get respective Environment record created in Dataverse.

Error happens at step "Upsert Environment" in the Cloud Flow "Admin | Sync Template v4 (Driver)" with message "Invalid character in field 'admin_environmentpurposetext': '\u000b', hexadecimal value 0x0B, is an invalid character."

I'm attaching screenshot from Cloud Flow as well as full Output Body with error details.

{ "error": { "code": "0x80040278", "message": "Invalid character in field 'admin_environmentpurposetext': '\u000b', hexadecimal value 0x0B, is an invalid character.", "@Microsoft.PowerApps.CDS.HelpLink": http://go.microsoft.com/fwlink/?LinkID=398563&error=Microsoft.Crm.CrmException%3a80040278&client=platform, "@Microsoft.PowerApps.CDS.InnerError.Message": "Invalid character in field 'admin_environmentpurposetext': '\u000b', hexadecimal value 0x0B, is an invalid character." } }

As far as MS Teams provisioning and setup is self-service in our company and as far as Power Platform Admins in our company have no chance to impact or resolve issues caused by presence of special characters on the MS Teams side, we would like to kindly ask you to get this situation resolved on the CoE Starter Kit side.

Expected Behavior

I'm not sure what coud be the best behavior but I could imagine of several possible:

1. Implement removal of unsupported special characters from string before new Dataverse record for Environment is created
2. Check for presence of special characters and if present, keep value of the field empty 'admin_environmentpurposetext'.

What solution are you experiencing the issue with?

Core

What solution version are you using?

4.24

What app or flow are you having the issue with?

Admin - Sync Template v4 (Driver)

What method are you using to get inventory and telemetry?

Cloud flows

Steps To Reproduce

Hard to say how our creative owners of MS Teams entered special characters like 0x0B (VT, Vertical Tab) into description of MS Teams. Probably by copying some text from Word document or similar. 🤔

Anything else?

No response

AB#3071

[Jenefer-Monroe]

Oh boy, thats an interesting one! Will put on the bug list to see if we can figure out how to catch these. Do you have a full list of the ones impacting you? I think it will have to be pretty hard coded replacement to remove them.

[miroslav-harlas]

Hello, Currently, I can see only this one special character in our tenant as mentioned above. However, I'm pretty sure that I saw Team environments with at least one other special character in the past, but we have executed a major clean up of Team environments and that got deleted.

I also intentionally used term "MINOR" in subject because this causes inventory miss for less than 0.1% of environments on our side, so no huge/major impact to us. 🙂

I'm also afraid how this could be fixed without causing negative impact on API calls and performance of the cloud flow. I found some RegEx Replace custom connector, not sure if this could be path forward. Or just simple "try-catch" construct and add second "upsert" operation into "catch" part but just with empty field 'admin_environmentpurposetext'? 🤔

[Jenefer-Monroe]

I'm hoping I can just do a replace expression directly in the current call to dataverse to say like ... replace '\u000b' with ''

[miroslav-harlas]

Hello, good idea with "replace" expression. It could be the most optimal for single character. I can try to modify Driver flow in our DEV env. to confirm this would work.

Actually this could be pretty tiny change. If I successfully test this in our DEV, could I contribute this change to CoE Starter Kit myself? However, I have never contributed and I'm not so familiar with how to do this. I found this page, but because there are many steps commented like "The steps to create your own copy of the source code will differ depending on your tools and software." and because I have no tools now and I need to start from green field, I'm not confident to contribute. Could be there some "How to contribute step-by-step" for Dev/DevOps newbies like me?

[Jenefer-Monroe]

We dont take contributions in any technical / merging kind of way, you just show me your expression and I apply it

But thank you! I will definately take your contribution manually 🤣

[tomas-andrews99]

we have the same error on quite a few environments, one of them is the following: { "error": { "code": "0x80040278", "message": "Invalid character in field 'admin_flowdescription': '\u000b', hexadecimal value 0x0B, is an invalid character.", "@Microsoft.PowerApps.CDS.HelpLink": "http://go.microsoft.com/fwlink/?LinkID=398563&error=Microsoft.Crm.CrmException%3a80040278&client=platform", "@Microsoft.PowerApps.CDS.InnerError.Message": "Invalid character in field 'admin_flowdescription': '\u000b', hexadecimal value 0x0B, is an invalid character." } }

The description contains this: The flow has the capability to handle any Dataverse unsupported characters and will replace them in schema name by an underscore “_” as the input.

[Jenefer-Monroe]

Thanks, and it sounds like this flow has what we need. Can you share?

[tomas-andrews99]

Certainly, what is it you need me to share?

[miroslav-harlas]

Hello all here. I'm sorry for coming back so late.

So far I have tested these two expressions, both with negative result:

1. Using simple Replace() replace(items('Apply_to_each_Environment')?['properties/description'],'\u000b','') This one fails, because not all Environments have description and Replace method fails on Null input for many environments then.

2. Using String() to convert nulls to string and then use Replace() replace(string(items('Apply_to_each_Environment')?['properties/description']),'\u000b','') Special character is not replaced. This ends with identical error on identical environment: "_Invalid character in field 'adminflowdescription': '\u000b', hexadecimal value 0x0B, is an invalid character." Fun fact: when using 'u000b' in Replace operation, it actually does not understand that it is a special character, it escapes \ with \\ , so when Replace is executed it is searching for \u000b and actually tries to replace exact string \000b instead of that special character. Not sure if that is clear. I hope I wrote it in understandable way.

Any other ideas what we can try to get rid of that annoying special character \u000b (0x0B) from environment description? Any different format for Replace method or use any other trick?

[Jenefer-Monroe]

@tomas-andrews99 the flow in question for you has the following description:

The flow has the capability to handle any Dataverse unsupported characters and will replace them in schema name by an underscore “_� as the input.

Thats exactly the caspability we are trying to find :) So people are failing try to deal with those special characters and dataverse, your flow in question seems to have a solution according to the description. Can you share that solution?