Post: How to use CocoaPods with Xcode CI Bots

[kylef]

See https://twitter.com/SebThiebaud/status/397499004331642880

[mtitolo]

There's also a lot of info here: https://groups.google.com/d/msg/cocoapods/eYL8QB3XjyQ/10nmCRN8YxoJ

[Dimillian]

So now we have at least 2 technique that works (and I have 1 complicated one). What is the best way to do this ? Should we write a blog post with all these techniques or prepare the gem we have discussed on the community chat?

[alloy]

@kylef That tweet is unavailable :/

[mtitolo]

Looks like tweets were removed, but here's a screenshot from Tweetbot. http://cl.ly/image/1y091n2l3n10

[amccarri]

I just finished setting up builds using Xcode bots this morning for our app which uses cocoa pods, however I will say that we are going to stick with Jenkins. So far my impression of bots is that they are meant for people who have never done CI before. They are EXTREMELY limited in capability (can’t monitor a build as it’s happening, build env can’t pass parameters to the build such as build numbers, can’t run scripts pre/post build etc, can’t administer a list of people allowed to download your test app, Xcode build list not auto-refreshed when new build is run, etc).

Looks like the only thing you can do is what you can add to your build scheme, and the only feature benefit is Xcode integration (which sucks so far, was not even able to create a bot in Xcode, had to use web page) so you can link from automated test runs to source code. Neither of these warrant moving from Jenkinsish/Testflight solutions.

All that is why we are NOT converting to bots, but if you want to use cocoa pods with bots here’s what I had to do:

1 - .go through the voodoo of setting up os x server and Xcode automation

• create home folder /var/teamsserver for _teamsserver user if it doesn’t already exist
• make sure to set proper ownerships on /var/teamsserver to _teamsserver account

2 - add .ssh with keys that can access your local pod repo to /var/teamsserver

3 - Set up new schemes specifically for Xcode bots

• edit scheme / build / pre-action add a bash script (or other script, your choice) that does something like if ~/.cocoapods/repos/localrepo does not exist, pod repo add your local repo if Pods folder does not exist pod install, else pod update
• share and check in the scheme

4 - Go http://yourserver/xcode and create the bot.

• create bots via xcode server web page… Xcode never let’s you get past authentication settings even when it shows them correct (this may be because we don’t allow HTTP urls on our github server, only ssh, so our url looks like git@ourserver.com:Team/App)

Fairly short list of steps, but OS X server docs were no help at all.

I’m really hoping this is just the starting point for bots, and that they will get a lot better in the future. Currently, even though jenkins is a much more complex CI server, I found myself pulling my hair out less setting up Jenkins than I did Xcode bots. Jenkins also supports something called a config matrix so you can run multiple build configs on a single checkout.

On Nov 7, 2013, at 5:05 PM, Michele notifications@github.com wrote:

Looks like tweets were removed, but here's a screenshot from Tweetbot. http://cl.ly/image/1y091n2l3n10

— Reply to this email directly or view it on GitHub.

[qnoid]

Thanks @amccarri for the detailed report! Much appreciated. What about server's ability to run your tests on multiple devices? Isn't that enough to justify it over Jenkins?

[blakewatters]

I also took the Xcode bots for a test spin. I went a slightly different direction than @amccarri in my setup:

Rather than creating a home directory at /var/teamsserver, I overrode the CP_HOME_DIR and CP_REPOS_DIR environment variables to point to a location inside the build path based off of PROJECT_ROOT (there are a bunch of environment variables configured by the build so there are options about where to put things).

If your project contains private pods (I have a private pod repository), then you wind up needing to do double configuration of your SSH access. This is because the Xcode bots walk you through setting up an RSA key to clone your code, but that configuration is never exposed to the build once it begins. So you have to configure secured Git access separately from secured Pod access. This also applies for public Pods if you pull from Github via SSH instead of via HTTPS.

Another irritating wrinkle is that you need to use a globally accessible Ruby installation for CocoaPods. If you are working off of the system Ruby and have done sudo installations of CocoaPods then it will work. But it’s pretty common (and often advisable) to see RVM or rbenv used to install parallel copies of Ruby. If you have these installed into your home directory and expect to use the same bits, then you are SOL. I went with an rbenv installation of Ruby into /usr/local/var/rbenv

I also have Bundler managing some additional dependencies necessary for my build, so I wound up having to install the bundler dependencies into a path under the build directory.

All of this required setting up pre build scripts on a Shared scheme, then pushing it remotely and testing, which was extremely slow and painful to work through.

The build output also did not update until the build had failed, so there’s a lot of dead time waiting. Then once it was all set up, the feedback mechanisms were limited and it felt very much like a bunch of trouble for very little payoff when compared to Travis CI Pro or Jenkins.

As to the ability to run tests on multiple devices:

I am not convinced that there's actually any benefit in this over running your test suite across multiple versions of the Simulator. I've done tons of automated testing and CI on iOS over the last couple of years and the types of issues that we uncover on manual on-device testing are problems that are related to concurrency, unexpected user behaviors, and performance. The concurrency and user behavior problems won't become visible just because you run your suite on a device instead of the simulator and performance issues are typically perceputal -- a view "feels sluggish" or laggy. These kinds of conditions aren't testable in an automated way. There is also a major performance penalty to running your tests on real hardware. They are going to run much slower as you have slower processors, less memory, and less I/O throughput. The slower they are to execute, the lower your iteration throughput and the less you can lean on CI. So what's the benefit over the simulator?

After giving CI bots a thorough tire-kicking, I ultimately went back to Jenkins and haven't looked back. My Jenkins setup is simpler and much more powerful/flexible. Hope this saves someone some wasted hours.

[amccarri]

Not unique to bots, you can do this with jenkins or any other CI tool that can run a command line build. From the xcodebuild man page:

xcodebuild -workspace MyWorkspace.xcworkspace -scheme MyScheme -destination 'platform=iOS Simulator,name=iPhone' -destination 'platform=iOS,name=My iPad' test

I created a blog post out of my original email that includes a sample script for my bots config, can find it here:

http://alexlikescode.blogspot.com

On Nov 10, 2013, at 4:14 AM, Markos Charatzas notifications@github.com wrote:

Thanks @amccarri for the detailed report! Much appreciated. What about server's ability to run your tests on multiple devices? Isn't that enough to justify it over Jenkins?

— Reply to this email directly or view it on GitHub.

[mtitolo]

@blakewatters thanks for the info! What kind of advantage do you see with the environment variables instead of the folder in /var/teamsserver? Do you think this approach would be more robust and possibly future-proof for updates?

@amccarri We already have instructions for other CI systems. This post will be specifically on Bots because so many people have asked about them. This isn't saying we recommend using Bots, but we should provide instructions on how to use cocoapods with them since Apple will likely push people to use them.

[amccarri]

@mtitolo sorry should have directed my comment, @qnoid was asking about bots' ability to run tests on multiple devices as an advantage over other solutions. I was just pointing out that running tests on multiple devices is not necessarily a feature of bots, but rather a feature of xcodebuild which can be used anywhere.

[amccarri]

Oh btw, here's my build pre-action script for using cocoa pods with bots:

cd $SRCROOT

if [ ! -e "$HOME/.cocoapods/repos/OurRepo" ]
then
    pod repo add OurRepo git@ourgithub:OurApps/CocoaPodSpecs.git
fi

if [ -e "Pods" ]
then
    pod update
else
    pod install
fi

[orta]

So is this nixed for the minute?

[mtitolo]

It still needs to happen. I haven't had the time to actually write the post and double check all of the solutions.

[swizzlr]

BUMP.

[mtitolo]

@swizzlr if you want to be useful, you could try one of the solutions listed above.

[swizzlr]

Thanks, I will!

[xanderdunn]

Thanks @amccarri, your experience really helped me.

Mac OS X 10.9.2, Xcode 5.1b5, CocoaPods 0.29.0

I created /var/teamsserver and set it's owner to _teamsserver.

I created a scheme pre-action script:

export PATH=/usr/local/opt/ruby/bin:/usr/local/bin:$PATH
brew update
brew upgrade
gem update --system
gem update cocoa pods
cd ${SRCROOT}
if [ -e "Pods" ]
then
pod update
else
pod install
fi

It's working flawlessly. As a two-step set-up procedure, I'd say this is really not a problem.

[swizzlr]

Alex, thank you for summarizing that. I'll see if I can try wrapping it into a cp plugin.

[mtitolo]

Starting to work on the blog post for this. Comments welcome - https://github.com/CocoaPods/blog.cocoapods.org/blob/master/_drafts/CocoaPods-Bots.markdown

This won't use any of the plugins (which may or may not work with Server anyway).

[egueiros]

That's great work everyone. I have all my builds compiling and my tests passing. I have, however, one error in my 'Analyze' phase. === ANALYZE TARGET CI_TestProject OF PROJECT CI_TestProject WITH CONFIGURATION Debug === ld: library not found for -lPods clang: error: linker command failed with exit code 1 (use -v to see invocation)

Everything else is great. Pre-script is run, pods are installed and linked. But Analyze seems to be the very first thing that runs, and I get that error. Any pointers? Thanks in advance!

[moayes]

@egueiros not sure if this applies to your case. Check Build Active Architecture Only. In my case Pods project had Build Active Architecture Only set to YES for Debug builds, but in the main project it was set to NO. Changing Pods project flag to NO fixed the problem.

[mtitolo]

And another blog post for reference: http://chris.cm/setting-up-xcode-bots-with-cocoapods/

[m1entus]

@egueiros I have the same issue, ld: library not found for -lPods

[alloy]

@moayes You should normally change it on your project to YES, not set it to NO on the Pods project.

[m1entus]

I hacked something but right now i'm getting error when building some of static lib, locally everything is working fine :/:

   The following build commands failed:
CompileC /Library/Server/Xcode/Data/BotRuns/Cache/2ab22af0-2cb0-4b18-ac23-4e5edb40d808/DerivedData/Build/Intermediates/Pods.build/Debug-iphonesimulator/Pods-SDWebImage.build/Objects-normal/i386/NSData+ImageContentType.o SDWebImage/SDWebImage/NSData+ImageContentType.m normal i386 objective-c com.apple.compilers.llvm.clang.1_0.compiler
(1 failure)

Additionaly if i uncheck find implicit dependencies on shared target i'm getting

ld: library not found for -lPods
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Update #1: Everything working on Debug (XCode), but when i changed target to release, it says that: ld: library not found for -lPods

[MatejBalantic]

@egueiros & @m1entus: I have the same problem: ld: library not found for -lPods

Did you find any solution?

[m1entus]

Nope :/

[m1entus]

@MatejBalantic Ok solved, problem was that i was using pre-build script which was propably called for every tharget in cocoapods, i debug it in Jenkins (lol) :) I solved it by adding additional static library target, my solution here: https://gist.github.com/m1entus/9660734

[obernal]

@m1entus sorry but your solution is unclear to me. I just recently started getting the same issue as @egueiros and @MatejBalantic. Could you maybe describe the steps in more detail? Thanks!

[m1entus]

@obernal read this http://chris.cm/setting-up-xcode-bots-with-cocoapods/ and after that see my gist

[thiagoperes]

Does XCode 6 solves this problem?

[mtitolo]

@thiagoperes no one has given a :+1: or :-1: to them yet.

[swizzlr]

Somewhat yes, haven't done it yet. I'm all in with Jenkins now because I hate myself, and I check in my pods to improve build times.

[Pintouch]

I tried with xCode6 pre-release and OS X Server 4.0, I still have the problem... I tried all workarounds existing for previous version but those don't work anymore, so I'm working on it cause I don't want to be constraint to have my pods folder and my workspace under Version Control. I let you know if I find a workaround.

EDIT

My BAD!! It's very easy on xCode 6, just adding this in the prescript of the bot (I'm using rvm) :

export PATH=/Users/username/.rvm/rubies/default/bin:$PATH
cd ./App_Folder

if [ -e "Pods" ]
then
pod update
else
pod install
fi

cd ..

Working like a charm.

[orta]

Yeah, I also expect it to be trivial in Xcode 6, been having issues with submodules on setting it up for one of my projects but don't think it should be too much effort. Would be nice to have a gem for CocoaPods caching though.

[clooth]

Any updates on this?

[kdawgwilk]

I'm also trying to get it running on Xcode 6 with server 4.0 but i keep getting

I have basically the same pre script as Pintouch above

[egueiros]

It seems the process got simpler. When creating a bot with XCode 6 we're now adding pre and post scripts to the bot itself, not in the project. The following worked for me on the bot's "pre" script:

export LC_ALL="en_US.UTF-8"
# Put the git repo name instead of “reponame” variable
cd reponame
# Remove the following line if there is no submodules in the project
# git submodule update --init --recursive
# If podfile is not in the root folder uncomment the following line
# and replace with the real folder name
# cd FolderName
pod install

Check out the full solution at http://papaanton.com/setting-up-xcode-6-and-apple-server-4-0-for-continues-integration-with-cocoapods/

Note: for those who had all their environment set up, like me, just scroll all the way to the final steps of setup

[kdawgwilk]

Here is my pre script:

export LC_ALL="en_US.UTF-8" export PATH=/Users/username/.rvm/rubies/default/bin:$PATH pod repo update podRepoName --verbose cd reponame pod install

I had to add line:

export PATH=/Users/username/.rvm/rubies/default/bin:$PATH

because the server couldn't find my cocoa pod gem and in the trigger log I was getting:

pod: command not found

Now I just have an issue with private-pods that don't want to install

[egueiros]

I know this is probably not the right thread...but since we deal with pre-scritps I decided to ask. Has anyone have had success running post-script with Xcode6 and Server 3.2.1? The post scripts in the Bot are being fired after the Analyze, before the Build/Archiving. And the post-script in the project Schema doesn't seem to be triggered at all.

[hshah2811]

Not sure if this thread is closed or not. Anyone here has done successfully integration of COCOAPODS with OS X server 4.0? We are facing issue of ssh keys as user is different when bots are integrated. It is using _xcsbuildd user instead of _teamserver. Where & how to replace ssh in OS X to access git repos ?

[kdawgwilk]

@hshah2811 It took me forever to figure out this simple fix

Use this command:

sudo -u _xcsbuildd /bin/bash

you can login under Xcode build user, than you need to add ssh key, as usual, ssh the repo from console, and verify the key.

[ricky-hufei]

For OSX Server 4.0, the directory for provision profiles will be changed to /Library/Developer/XcodeServer/ProvisioningProfiles

[trembb01]

Has anyone used a private cocoapod and got it to work with their bot? My private cocoapod repo requires authentication, so pod install attempts to retrieve the repo with the _xcsbuildd username.

I looked into git config to try to set a global username for a specific host, but it still uses _xcsbuildd.

Any ideas?

[hshah2811]

@trembb01 Try with passphrase free ssh key. When it asks to set ssh key "Enter passphrase (empty for no passphrase):", simply press enter.

This way when bot tries to access your private cocoapods repo, it doesn't ask for authentication.

[joeblau]

@trembb01 I just followed @kdawgwilk instructions and it works.

sudo -u _xcsbuildd /bin/bash
ssh-keygen -b 4096
<enter>
<enter>
<enter>
cat /var/_xcsbuildd/.ssh/id_rsa.pub | pbcopy
``

Paste your public key from the Xcode Server Build Daemon into your GitHub/BitBucket approved SSH keys and your build will work.  I'm going to write a blog post on how to get this whole thing working based on all of the feedback from this thread.

[hons82]

Currently we use this script

export LC_ALL="en_US.UTF-8"

if [ ! -e "$HOME/.cocoapods/repos/reponame" ]
then
    pod repo add reponame https://username:password@path/to/private/Specs.git
fi

cd ${XCS_SOURCE_DIR}/Projectname_without_dot_git

git submodule update --init --recursive

if [ -e "Pods" ]
then
    pod update
else
    pod install
fi

Problems:

• we use a private Spec repo so we need to set username and password of a service user there which I think can't be avoided
• I need to specify for every bot the Projectname_without_dot_gitfor which I'd need a variable to make it independent

[AliSoftware]

1) Since you need to specify source 'https://path/to/private/Specs.git' in your Podfile, that line with pod repo add is quite useless, as CocoaPods will clone this Spec repo for you if not present already.

2) pod update will do the same as pod install if the Pods folder and Podfile.lock are not present, so I don't get why you need that test to run either one or the other

3) Using pod update is probably not a good idea as it will update your pods, bumping all their versions (regardless of the versions to which they were locked to the last time you wrote and tested your code and commited it) to the latest, which is generally only done on demand, when you're ready to upgrade all your dependencies.

[joeblau]

@hons82 Why don't you use keys instead of username password?