PowerApps-TestEngine is not working for PCF Controls Like command bar, Pivot and other controls.

[KoteshMCGit]

Recently I have started analyzing Power Apps Test engine to implement automation testing for one of my canvas App, which is having PCF controls.

About my test canvas APP Pls find the below screenshot of canvas app with pcf controls

New or Edit project screenshot

below of PCF controls are used to build the canvas App

1. CommandBar having 3 buttons (New Project, Edit Project and Submit Project) and need to click / select a particular button for unit test. 2.SearchBoxProjectName it is text box, used to read search string to filter project records to display in details grid.
2. PivotProjectStatus it is a Pivot control having different statuses of project. while clicking on specific button, the projects will be shown for respective status.
3. ProjectDetailsList it is grid control will show all project records for selected tab or statuses. having selection of records having link type column to show project details
   5. Data cards New or Edit project form is having all data cards to save new project or modify Project.

Describe the bug

I have tried of using select command by referring URL https://github.com/microsoft/PowerApps-TestEngine/blob/main/docs/PowerFX/Select.md. But we could not able select particular index wise / caption wise controls or buttons. And also data cards also not able find the controls to update or read the values

[A clear and concise description of what the bug is.]

find the error logs for more details

RUNNING TEST CASE: Automation Project commandbar Example

Executing Select function.

[Error]: Encountered an error. See the debug log for this test case for more information.

System.AggregateException: One or more errors occurred. (Exception has been thrown by the target of an invocation.) ---> System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.AggregateException: One or more errors occurred. (Can't change 'ExpandMenu''s type from Microsoft.PowerFx.Types.KnownRecordType to Microsoft.PowerFx.Types.KnownRecordType.) ---> System.NotSupportedException: Can't change 'ExpandMenu''s type from Microsoft.PowerFx.Types.KnownRecordType to Microsoft.PowerFx.Types.KnownRecordType. at Microsoft.PowerFx.RecalcEngine.UpdateVariable(String name, FormulaValue value) at Microsoft.PowerApps.TestEngine.PowerFx.PowerFxEngine.UpdatePowerFxModelAsync() in C:\Kotesh\GitHub\Playwright\PowerApps-TestEngine\src\Microsoft.PowerApps.TestEngine\PowerFx\PowerFxEngine.cs:line 152 at Microsoft.PowerApps.TestEngine.PowerFx.PowerFxEngine.b__13_1() in C:\Kotesh\GitHub\Playwright\PowerApps-TestEngine\src\Microsoft.PowerApps.TestEngine\PowerFx\PowerFxEngine.cs:line 51 at Microsoft.PowerApps.TestEngine.PowerFx.Functions.SelectTwoParamsFunction.SelectAsync(RecordValue obj, NumberValue rowOrColumn) in C:\Kotesh\GitHub\Playwright\PowerApps-TestEngine\src\Microsoft.PowerApps.TestEngine\PowerFx\Functions\Select\SelectTwoParamsFunction.cs:line 66 --- End of inner exception stack trace --- at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions) at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken) at System.Threading.Tasks.Task.Wait() at Microsoft.PowerApps.TestEngine.PowerFx.Functions.SelectTwoParamsFunction.Execute(RecordValue obj, NumberValue rowOrColumn) in C:\Kotesh\GitHub\Playwright\PowerApps-TestEngine\src\Microsoft.PowerApps.TestEngine\PowerFx\Functions\Select\SelectTwoParamsFunction.cs:line 32 --- End of inner exception stack trace --- at System.RuntimeMethodHandle.InvokeMethod(Object target, Span1& arguments, Signature sig, Boolean constructor, Boolean wrapExceptions) at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture) at Microsoft.PowerFx.ReflectionFunction.Invoke(IServiceProvider serviceProvider, FormulaValue[] args) at Microsoft.PowerFx.ReflectionFunction.<GetTexlFunction>b__11_1(IServiceProvider runtimeConfig, FormulaValue[] args) at Microsoft.PowerFx.CustomTexlFunction.Invoke(IServiceProvider serviceProvider, FormulaValue[] args) at Microsoft.PowerFx.EvalVisitor.Visit(CallNode node, EvalVisitorContext context) at Microsoft.PowerFx.ParsedExpression.EvalAsync(CancellationToken cancel, ReadOnlySymbolValues runtimeConfig) at Microsoft.PowerFx.CheckResultExtensions.EvalAsync(IExpressionEvaluator expr, CancellationToken cancel, RecordValue parameters) at Microsoft.PowerFx.RecalcEngine.EvalAsync(String expressionText, CancellationToken cancel, RecordValue parameters, ParserOptions options) --- End of inner exception stack trace --- at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions) at System.Threading.Tasks.Task1.GetResultCore(Boolean waitCompletionNotification) at System.Threading.Tasks.Task`1.get_Result() at Microsoft.PowerFx.RecalcEngine.Eval(String expressionText, RecordValue parameters, ParserOptions options) at Microsoft.PowerApps.TestEngine.PowerFx.PowerFxEngine.Execute(String testSteps) in C:\Kotesh\GitHub\Playwright\PowerApps-TestEngine\src\Microsoft.PowerApps.TestEngine\PowerFx\PowerFxEngine.cs:line 134 at Microsoft.PowerApps.TestEngine.PowerFx.PowerFxEngine.ExecuteWithRetryAsync(String testSteps) in C:\Kotesh\GitHub\Playwright\PowerApps-TestEngine\src\Microsoft.PowerApps.TestEngine\PowerFx\PowerFxEngine.cs:line 74 at Microsoft.PowerApps.TestEngine.SingleTestRunner.RunTestAsync(String testRunId, String testRunDirectory, TestSuiteDefinition testSuiteDefinition, BrowserConfiguration browserConfig, String domain, String queryParams, CultureInfo locale) in C:\Kotesh\GitHub\Playwright\PowerApps-TestEngine\src\Microsoft.PowerApps.TestEngine\SingleTestRunner.cs:line 161

To Reproduce

download and install creator kit from https://learn.microsoft.com/en-us/power-platform/guidance/creator-kit/overview to use PCF controls in canvas apps

1. Create a new canvas application with using of latest PCF controls
2. Create a command bar with more than 2 buttons or pivot controls
3. write a sample code or test code for each button wise
4. Test with respective .yaml file for newly created App

Expected Behavior: [What you thought would happen]
For PCF controls, we should be able to click / select a particular index wise buttons / controls

Data cards also should be able to recognize and read / write data.

for PCF Grid controls, we should be able to select a specific row to modify / submit project

Actual Behavior: [What actually happened. If applicable, add screenshots of error messages to help explain your problem.]

Power apps Testing commands like Select , set property are not able to identify index wise controls to perform respective action.

Additional context

[Add any other context about the problem here.]
[Add any helpful context on the .msapp like created from teams, contains custom pcf components, etc.]

[jt000]

Thanks for submitting, @KoteshMCGit. You're correct that Test Engine today does not support external code components (both in the Creator Kit & created by 3rd parties). I hope one day we will be able to satisfy this goal, but at the moment this is a known limitation. I'll make sure the documentation is updated to make the limitations for Test Engine clearer in the future.

[filcole]

@jt000 is this a necessary step to implement before model driven apps can be supported by Test-Engine?

[Grant-Archibald-MS]

@ficole with the new changes being developed support for code first PCF and Model Driven Applications are independent.