It’s been a couple years since I looked at the options for Mobile Continuous Integration (CI) for iOS apps. Last time I did this, I decided on BuddyBuild, who has since been bought by Apple. Unfortunately BuddyBuild is no longer accepting new customers, and they discontinued free plans and Android support in March of 2018. I hope this means that Apple will offer their own hosted CI solution soon, and with any luck it will go as well as the acquisition of TestFlight did. There was no news at this year’s WWDC, so for now we must find something else.
Candidates
While I’ve built complicated CI setups on Jenkins in the past, I don’t think that’s a good use of most people’s time these days. I’m also not interested in Xcode Server because it had a lot of stability problems when my team used it. It’s possible that our Mac Mini just wasn’t up to the task, but even if that was the issue, its native feature set is pretty basic. If you do want to roll your own Xcode Server or Fastlane setup, MacStadium seems to be a popular hosting provider.
The options I found are similar to two years ago. Buddybuild is gone, but Visual Studio App Center looks interesting. The GreenhouseCI domain now forwards to Nevercode, so I assume they changed their name or there was some kind of acquisition. That leaves us with the following candidates.
Features
The current hosted Mobile CI options come in two main flavors. There are “full feature” providers that try to automatically configure everything for you, as well as manage code signing and the various mobile-specific deatils. Then there are services that basically give you a Mac VM and some job infrastructure. The latter tend to be cheaper than a full time Mac hosting provider, and they also have some nice features that are generally useful for CI builds, like custom VM images and workflows.
The features I care about are as follows.
- VCS support: cloning repos, starting builds on push, updating build status 1
- Deployment: both ad-hoc and to AppStore Connect
- Extensibility: some way to run code review tools like Danger.
- Crash reporting: web access to crash reports is nice, and most services offer other integrations, so you can get notification in your chat software or have tickets opened automatically for new crashes.
- Basic analytics
Based on my reading of the documentation, these are the features that are supported.
| Service | VCS | Deployment | Extensibility | Crash Reporting | Analytics | Other |
|---|---|---|---|---|---|---|
| Bitrise | Bitbucket, GitHub, Gitlab | yes | Step Library | no | no | no |
| App Center | Bitbucket, GitHub, VSTS | yes | build lifecycle scripts | yes | yes | real device testing, push notifications |
| Nevercode | Bitbucket, GitHub, Gitlab | 3rd party only | build lifecycle scripts / Fastlane | no | no | real device testing 2 |
| CircleCI | Bitbucket, GitHub | Fastlane | full control | no | no | |
| Travis CI | GitHub | Fastlane | full control | no | no |
Pricing
Pricing strategies vary quite a bit. They’re usually based on some combination of number of concurrent builds, build runtime and number of total builds for the month. Some services offer an open source discount, which is usually free. Low usage plans range from $40 - $100, and some providers offer a modest discount for paying yearly instead of monthly.
These are the prices at the time of this writing. Most plans scale up beyond the entry-level tier by charging for more concurrent builds, and features like real device testing and push notifications are usually an additional charge.
| Service | Free Tier | Lowest Paid Tier | Discounts |
|---|---|---|---|
| Bitrise | 10 minutes per build / 200 builds per month | $40 / 45 minutes / unlimited builds | Student, Contributor |
| App Center | 30 minutes per build / 240 minutes per month | $40 / 60 minutes / unlimited | |
| Nevercode | n/a | $99 / 2 apps / 2 concurrent builds / 45 minutes per build / unlimited builds per month | Case by case |
| CircleCI | n/a | $39 / 2 concurrent builds / 500 minutes per month | Open Source |
| Travis CI | n/a | $69 / unlimited | Open Source |
Conclusion
The only candidate that provides the same level of functionality that BuddyBuild did is Visual Studio App Center. I assume that some of their iOS functionality came from their acquisition of HockeyApp, which I’ve used in the past and been happy with. Being a Microsoft offering, I think we shouldn’t have to worry about it going away anytime soon. All that said, only 240 build minutes per month on the free tier is low. Once you account for dependency setup, launching the simulator for tests and possibly doing an archive build, it’s not hard to have an iOS build get into the 5-10 minute range with even a minimal number of tests. While I can’t recommend something I haven’t used yet, App Center would be on the top of the list if I was doing this evaluation for a client.
Nevercode doesn’t have a free tier, and their entry level is more than double the cost of other options. They also don’t have some features that I’m interested in.
Travis and Circle only offer a free tier for open source, and they’re both more work to setup and maintain than I want to deal with. Travis also doesn’t support Bitbucket. While I haven’t used Circle’s macOS platform, I have used it for Ruby and Elixir projects both at an employer and personally. Their 2.0 platform works well and has good documentation, so I would seriously consider it if I needed that level of control.
Finally there is Bitrise. It doesn’t have all the features I’d like, but it has the ones that are most important to me - VCS support, deployment and extensibility. With 200 builds per month, the free tier should cover the needs of small or personal projects. I also like that they give a discount for contributing to their platform. I’ll be trying Bitrise out, and I’ll capture my experience there in an upcoming post.
- I use Bitbucket for private repos. ↩
- via AWS Device Farm ↩