Moonshine Build for ZoneMinder App

[JoelProminic]

I reviewed this today with @JustinProminic for inclusion in Genesis Directory. He reiterated that we need to be able to open and build the full project in Moonshine.

I see that ZoneMinder is a XCode project, which is not supported in Moonshine currently. We should add a .genericproj file to open this in Moonshine. Can we handle the build with an Ant script?

Is CommandLineTools sufficient for the build, or do we need the full XPages? If it is the latter, this also not supported by Moonshine currently.

[rat-moonshine]

As its documented it is possible to build on Terminal. But the subject is broad enough, needs adequate investigation.

Some resource for a start:

https://developer.apple.com/library/archive/technotes/tn2339/_index.html https://betterprogramming.pub/xcode-command-line-tools-use-cases-6eb331eda99d

Beside the build/run we will also requires additional handling:

• Getting connected device information for device-debug/run (something like we do for device-debugging for AIR application)
• Retrieving certification information for device-installation. Currently the certificate manages automatically by Xcode by retrieving details from Apple Developer Centre based on the user authentication; This part could be messy since Xcode used to handle and manage authentication, certificate download and set

[rat-moonshine]

Okay, I able to do following steps using the command-line:

• Build project
• Launch simulator
• Install the .app in simulator
• Run the .app in simulator

However, there are multiple sub-steps included to successfully complete the above steps, and irony is parsing the output-values from those steps are painful. There is no easy way to cherry-pick based on our requirement, from those outputs.

I detailed the steps in a Wiki, currently keeping this in our private repository: https://github.com/prominic/SystemHealthAlerts/wiki/Build-And-Run-tvOS-Project-From-Command-Line

[JoelProminic]

We went through the command parsing in the meeting today.

For $scheme, I think this should be configured in the Ant script for the project.

We can use the id for destination like this:

% xcodebuild -scheme SystemHealthAlerts -showdestinations | grep "Apple TV 4K (at 1080p)" | sed 's/^.*id:\([^ ,}]*\)[ ,}].*$/\1/'
424D0461-9E7B-48F0-A752-4F47EEAF9413

xcodebuild clean build -scheme SystemHealthAlerts -project SystemHealthAlerts.xcodeproj -destination id=424D0461-9E7B-48F0-A752-4F47EEAF9413

For the simulator, we can use:

% xcrun simctl list devices available | grep 'Apple TV 4K (at 1080p)' | head -n 1 | sed 's/.*(\([^)]*\)) ([^)]*) *$/\1/'
424D0461-9E7B-48F0-A752-4F47EEAF9413

For the BUILD_DIR we can use:

% xcodebuild -project SystemHealthAlerts.xcodeproj -showBuildSettings | grep -m 1 "BUILD_DIR" | sed 's/^.*BUILD_DIR *= *\(.*\)$/\1/'
/Users/-snip-/Library/Developer/Xcode/DerivedData/SystemHealthAlerts-cvrvleilawaogaanelauhazgmwcv/Build/Products

[rat-moonshine]

Trying to parse a targeted destination, however returned many a information than one-line:

% xcodebuild -scheme ZoneMinder -showdestinations | grep "Apple TV 4K (2nd generation)" | sed "s/^.*id:\([^ ,}]*\)[ ,}].*$/\1/"
--- xcodebuild: WARNING: Using the first of multiple matching destinations:
{ platform:tvOS, id:dvtdevice-DVTiOSDevicePlaceholder-appletvos:placeholder, name:Any tvOS Device }
{ platform:tvOS Simulator, id:dvtdevice-DVTiOSDeviceSimulatorPlaceholder-appletvsimulator:placeholder, name:Any tvOS Simulator Device }
{ platform:tvOS Simulator, id:D348E5C4-0944-4ED5-9414-53092D089966, OS:16.0, name:Apple TV }
{ platform:tvOS Simulator, id:4D30B8A7-05A7-4AE0-88ED-119AFA20A2E2, OS:16.0, name:Apple TV 4K (2nd generation) }
{ platform:tvOS Simulator, id:F5A9A2BD-6682-4667-A437-957A90FB98A3, OS:16.0, name:Apple TV 4K (at 1080p) (2nd generation) }
4D30B8A7-05A7-4AE0-88ED-119AFA20A2E2

Furthermore, it could be a headache to look by a particular name-string, eg. I have "(2nd generation)" when @JoelProminic have not, and it seems the command return all items which contains 'Apple TV' value.

EDIT:

On a second look to this problem, it appears that we only needs the device UDID which is also returns by the xcrun command. Testing the xcrun command it appear to return the UDID in expected format, too:

% xcrun simctl list devices available | grep 'Apple TV 4K (at 1080p)' | head -n 1 | sed 's/.*(\([^)]*\)) ([^)]*) *$/\1/'

So I'm using this command only instead of finding destination which fits our basic needs.

[rat-moonshine]

I'm currently working in a basic Ant build script which is not cleaned-up yet, but can trigger with following base commands:

ant -DAPP_NAME=ZoneMinder -DAPP_ID=net.prominic.ZoneMinder

[rat-moonshine]

While more testing on Big Sur Xcode, I found it failed to open the project with following error:

While on Ventura I have Xcode 14 (where the project constructed) Big Sur has Xcode 12. Because of the version mismatch a generated .app also failed to start in simulator on Big Sur, left it a blank screen.

I changed the project format to Xcode 12.0-compatible. This also required me to adjust other requisite configuration in the project. I confirm I can now run the project on Big Sur Xcode 12 and also runs on simulator. I also confirm this statement valid in case of Ant build.

Somethings to note:

Testing on Xcode 12 simulators, I noticed some UI constraint problems in running app. However, all the constraints are working in expected ways while testing on Xcode 14 simulators. This basically affects to some sizing and alignment issues. I'm not sure if running on TV would make any difference or not, I haven't tested on TV from Xcode 12, yet.

Currently, I have set the recommended Xcode version to 14.

[JoelProminic]

@rat-moonshine, I did some quick cleanup for build.xml to make the script easier to use:

# You can run this with no additional parameters...
ant
# Which is equivalent to this:
ant -DAPP_NAME=ZoneMinder -DAPP_ID=net.prominic.ZoneMinder

# You can overide the simulator like this:
ant "-DSIMULATOR=Apple TV 4K"
# Or specify a simulator by ID like this:
ant -DtargetDeviceUDID=3933E5AC-724D-4F11-8E90-C1C89CADDC73

However, I found the simulator stopped starting in my tests this afternoon. I confirmed the device and path were correct, and the original version of the script also failed:

 [echo] Build directory: /Users/-snip-/Library/Developer/Xcode/DerivedData/ZoneMinder-elkkshsdeojowucgjjhplgxjjyju/Build/Products
 [echo] Starting target simulator..
 [echo] Simulator started with UDID: 3933E5AC-724D-4F11-8E90-C1C89CADDC73
 [echo] Installing app on simulator..
 [exec] An error was encountered processing the command (domain=com.apple.CoreSimulator.SimError, code=405):
 [exec] Unable to lookup in current state: Shutdown

Other updates:

• I added a .genericproj file and confirmed that I can open the project in Moonshine and run the Ant command (though it fails with the above error)
• I removed the svn-revision task
• I updated .gitignore
• I deleted ZoneMinder.xcodeproj/project.xcworkspace/xcuserdata and ZoneMinder.xcodeproj/xcuserdata, which had some generated user-specific data. Take a look to see if we need any other files in this directory.

As a reminder, we want to remove the dependency on ant-contrib.xml. I see you copied this script from another project, so it would be nice if you could cleanup the other tasks that are probably not needed so that they aren't a distraction later.

[JoelProminic]

I tested on Mojave as well, and I got this error even when I rolled back my changes:

     [echo] Build directory: 2023-01-10 14:50:52.147 xcodebuild[12044:1109572] Requested but did not find extension point with identifier Xcode.InterfaceBuilderBuildSupport.PlatformDefinition
     [echo] /Users/-user-/Library/Developer/Xcode/DerivedData/ZoneMinder-boczubcblswmlxaonmvmbgaawocl/Build/Products
     [echo] Starting target simulator..
     [echo] Simulator started with UDID: 01CD94AA-7201-4CDB-AABF-04301E0802FD
     [echo] Installing app on simulator..
     [exec] An error was encountered processing the command (domain=NSPOSIXErrorDomain, code=2):
     [exec] Failed to install the requested application
     [exec] An application bundle was not found at the provided path.
     [exec] Provide a valid path to the desired application bundle.

BUILD FAILED
/Users/-user-/Documents/workspace/ZoneMinder-tvOS/ZoneMinder/build.xml:271: exec returned: 2

I'm guessing I missed something simple, but I need to move on to other tasks for now.

[rat-moonshine]

I tested on Mojave as well, and I got this error even when I rolled back my changes

Incidentally, I was having same problem on Ventura tests, but I wasn't sure if this an issue with my system thus I wanted to test this with other systems.

Upon more investigation I noticed the BUILD_DIR parsing command isn't working in expected manner on more latest OS. However the command seems to be working well on Big Sur.

While on Big Sur the command outputs path only:

% xcodebuild -project ZoneMinder.xcodeproj -showBuildSettings | grep -m 1 "BUILD_DIR" | sed 's/^.*BUILD_DIR *= *\(.*\)$/\1/'
/Users/-snip-/Library/Developer/Xcode/DerivedData/ZoneMinder-dwflezkjpeypcfcpehpimzhndzta/Build/Products

Running the same command on more newer OS:

% xcodebuild -project ZoneMinder.xcodeproj -showBuildSettings | grep -m 1 "BUILD_DIR" | sed 's/^.*BUILD_DIR *= *\(.*\)$/\1/'                             
2023-01-11 12:36:26.332 xcodebuild[13761:144148] Requested but did not find extension point with identifier Xcode.InterfaceBuilderBuildSupport.PlatformDefinition
    BUILD_DIR = /Users/-snap-/Library/Developer/Xcode/DerivedData/ZoneMinder-dwflezkjpeypcfcpehpimzhndzta/Build/Products

Therefore when running the command in Ant build, it stores the full-string as path-value which is wrong and eventually fails during installation.

We need to re-master the command this way that it works accordingly in newer OS, too.

Separately I tried to get some idea why the Requested but did not find extension point with identifier warning was. Many people described that it requires Command Line Tools update. While trying to check whether the tools can update, I found I already has most latest updates on Command Line Tools. This solution didn't fixed the warning for me.

[rat-moonshine]

For some time I tried to parse-out the path-value from BUILD_DIR command at Ant side.

I tried not to use propertyregex since I'm not keen to keep dependency to ant-contrib at this moment. I tried to split/match the string using JavaScript, but for some reason none worked.

The stored output from BUILD_DIR command also a bit strange, it prints 2 [echo] when I'm trying to print, Eg:

<echo>${buildDirectory}</echo>
// output:
     [echo] Build directory:     BUILD_DIR = /Users/-snap-/Library/Developer/Xcode/DerivedData/ZoneMinder-dwflezkjpeypcfcpehpimzhndzta/Build/Products
     [echo] 2023-01-11 16:40:22.527 xcodebuild[10984:118247] Requested but did not find extension point with identifier Xcode.InterfaceBuilderBuildSupport.PlatformDefinition

[rat-moonshine]

I slim-down the Ant build-script covering our current needs now, ant-contrib dependency also removed.

[JoelProminic]

Even after the update, I'm still seeing the same behavior. Is this blocked because the wrong simulator (Apple TV 4K (3rd generation)) is started?

% ant -DtargetDeviceUDID=0EA314CC-8B79-4E77-ADCC-9F4DD854F580
...
     [exec] ** BUILD SUCCEEDED **
     [exec] 
     [echo] Build directory: /Users/-snip-/Library/Developer/Xcode/DerivedData/ZoneMinder-boczubcblswmlxaonmvmbgaawocl/Build/Products
     [echo] Starting target simulator..
     [echo] Simulator started with UDID: 0EA314CC-8B79-4E77-ADCC-9F4DD854F580
     [echo] Installing app on simulator..
     [exec] An error was encountered processing the command (domain=com.apple.CoreSimulator.SimError, code=405):
     [exec] Unable to lookup in current state: Shutdown

BUILD FAILED
/Users/-snip-snip-Documents/workspace/ZoneMinder-tvOS/ZoneMinder/build.xml:75: exec returned: 149

Total time: 16 seconds
% xcrun simctl list devices available
== Devices ==
...
-- tvOS 16.1 --
    Apple TV (25D7625C-0548-4A5E-B281-9283191E5760) (Shutdown) 
    Apple TV 4K (3rd generation) (01CD94AA-7201-4CDB-AABF-04301E0802FD) (Booted) 
    Apple TV 4K (3rd generation) (at 1080p) (0EA314CC-8B79-4E77-ADCC-9F4DD854F580) (Shutdown) 

[rat-moonshine]

No, it is clear that whichever simulator it could be, the run found a simulator:

[echo] Simulator started with UDID: 0EA314CC-8B79-4E77-ADCC-9F4DD854F580

Based on the output, the simulator didn't fully started and ready when the install command executed:

[exec] Unable to lookup in current state: Shutdown

We has 10 seconds delay between simulator boot-up and install command, as we discussed earlier. But the time may not sufficient enough in case a particular simulator runs for the first time. I see it returns us to the same old question how to somehow ensure the simulator is fully start.

[rat-moonshine]

You can now test the build-script in >Big-Sur @JoelProminic .

[JoelProminic]

I'm getting errors for javascript when I test this on Catalina:

     [exec] ** BUILD SUCCEEDED **
     [exec] 
     [echo] Build directory: /Users/-user-/Library/Developer/Xcode/DerivedData/ZoneMinder-elkkshsdeojowucgjjhplgxjjyju/Build/Products
     [echo] Starting target simulator..
     [echo] Simulator started with UDID: 424D0461-9E7B-48F0-A752-4F47EEAF9413

testSimulatorBoot:
     [echo]     Apple TV 4K (at 1080p) (424D0461-9E7B-48F0-A752-4F47EEAF9413) (Shutdown) 
     [echo] Note: Waiting for simulator boot. Re-checks in a moment.

BUILD FAILED
/Users/-user-/Documents/Flex/workspace/ZoneMinder-tvOS/ZoneMinder/build.xml:73: The following error occurred while executing this line:
/Users/-user-/Documents/Flex/workspace/ZoneMinder-tvOS/ZoneMinder/build.xml:90: Unable to create javax script engine for javascript

I did not get this error on Monterey, though (this machine is also less likely to have custom Ant configuration).

The simulator booted in both cases, but the application didn't load. in the simulator.

[JoelProminic]

I reran the test on Monterey, and it worked for all the available simulators. It looks like the first test failed because Ant timed out on the simulator booting (perhaps it was installing something). Note that the build was reported as successful:

restartTestSimulatorBoot:

testSimulatorBoot:
     [echo]     Apple TV 4K (3rd generation) (at 1080p) (0EA314CC-8B79-4E77-ADCC-9F4DD854F580) (Shutdown) 
     [echo] Note: Waiting for simulator boot. Re-checks in a moment.
     [echo] Error: Reaches maximum waiting period for the simulator to boot.

BUILD SUCCESSFUL
Total time: 1 minute 22 seconds

[rat-moonshine]

I need some help here @JoelProminic . I'm adding script to create a simulator in case no installer found installed. Apparently we wants to filter-out the UDID part only from the following output:

% xcrun simctl create "Test New tvOS Sim4" "Apple TV 4K (at 1080p)" | head -n 1 | sed 's/.*(\([^)]*\)) ([^)]*) *$/\1/'
No runtime specified, using 'tvOS 16.0 (16.0 - 20J373) - com.apple.CoreSimulator.SimRuntime.tvOS-16-0'
F94D21C4-9A41-43C4-8EFE-B161F7CF5A14

I tried to reuse the filtration part from other commands, but that seems not to work effectively, here.

[rat-moonshine]

Unfortunately, this still seems need a little help even after adding 2>&1 in the command:

% xcrun simctl create 'My Apple TV 4K' 'Apple TV 4K (at 1080p)'  2>&1  | sed 's/.*(\([^)]*\)) ([^)]*) *$/\1/'
No runtime specified, using 'tvOS 16.0 (16.0 - 20J373) - com.apple.CoreSimulator.SimRuntime.tvOS-16-0'
1A2CB29F-F48C-4A42-AC29-A1C5F699C69A

I see it still returns the No runtime specified... message with the output, and I failed to come-up to a proper sed command that returns the UDID/format only.

I see a UDID pattern regex somewhere although its written for Obj-C.

[JoelProminic]

I see the output from the create command is actually very simple, so you should just need to exclude the "runtime" line:

% xcrun simctl create "Test New tvOS Sim4" "Apple TV 4K (at 1080p)" 2>&1 | grep -i -v "No runtime specified"
339CC2C6-3CBF-4D3B-9F69-6E0C5C1945B1

Or you can parse a runtime for the create command like this:

% xcrun simctl list runtimes | grep tvOS | head -n 1 | rev | cut -d ' ' -f 1 | rev
com.apple.CoreSimulator.SimRuntime.tvOS-14-3
% xcrun simctl list runtimes | grep tvOS | head -n 1 | sed 's/^.* \([^ ]*\)$/\1/'
com.apple.CoreSimulator.SimRuntime.tvOS-14-3

[rat-moonshine]

Build script updated with following elements:

• Installation of new tvOS simulator in case targeted simulator not found
• An exit task if device UDID not found in any way