rebeckerspecialties / benchmarking-test-app

React Native test app for exploring benchmarking in AWS Device Farms
3 stars 0 forks source link

Research: Profiling AWS Device Cloud with XCode Instruments #2

Closed hmallen99 closed 3 weeks ago

hmallen99 commented 1 month ago

Problem

We want to profile real iOS devices in AWS device farm using XCode instruments. This is challenging, since we will need to run instruments from the command line and likely connect to a running XCode process from a test device. There are not any resources online of how to do this exactly.

Some possible ways to do this will be through Appium, which has support for XCode instruments, or by directly connecting an AWS mac to the device and running xctrace.

We will need to export the instruments trace for viewing and parse the trace to give the benchmark a pass/fail and upload to a GitHub page..

hmallen99 commented 1 month ago

XCode Instruments

Profiling options

xctrace is the command line equivalent of XCode instruments. The tool allows us to generate data about game performance, memory leaks, CPU profiling. I.e. all the data we need to profile for a game engine. The tricky part of this will be hooking everything up to AWS device cloud.

man page: https://keith.github.io/xcode-man-pages/xctrace.1.html.

Finding an attached device

xcrun xctrace list devices

Finding supported templates

xcrun xctrace list templates

Recording with xctrace

Attaching xctrace to a running process:

xcrun xctrace record --template 'Time Profiler' --device-name 'iPhone 15 Pro Max Simulator' --attach ReactTestApp

Launching a process with xctrace:

xcrun xctrace record --template 'Time Profiler' --device-name 'iPhone 15 Pro Max Simulator' --launch ios/benchmarking-test-app.xcworkspace

The generated profiles can be visualized in XCode instruments. We will also want to parse the data so that a pass/fail can be generated for the benchmark.

hmallen99 commented 1 month ago

AWS Device Farm

Custom Test Environment

https://docs.aws.amazon.com/devicefarm/latest/developerguide/custom-test-environments.html

Writing Performance Tests

https://developer.apple.com/documentation/xctest/performance_tests https://developer.apple.com/documentation/xcode/writing-and-running-performance-tests

React Native

https://www.callstack.com/blog/testing-your-react-native-apps