How Kristal.AI built the iOS app CI/CD pipeline and saved up to 2 hours of developers’ time daily
July 7, 2021
Kristal.AI is a digital wealth platform that aims to provide private banking solutions through transparent access to investment and services, which are traditionally reserved for the Ultra-high net worth. We leverage technology, algorithms, and human expertise to help clients with portfolio allocations and optimization. We are licensed and operational in Singapore, Hong Kong, and India.At the center of Kristal.AI, sits the 35 person engineering team. While small in number, for building and managing our digital wealth management platform. We are the tech-savvy ones and love to try different things to improve the performance, scalability, and reliability of the platform.Amongst other things, the team is driven by this simple motto:"Some things are meant to be done by machines."- Vineeth Narasimhan, Co-Founder & CTO, Kristal.AI.Here’s the journey of how we built our iOS app CI/CD pipeline using Github Actions.
The Problem Statement
Before we discovered the magic of GitHub Actions, we used to manually attach the QA iOS device and give the build via Xcode. This made sense to our use case at that time since it saved time, rather than going through the DevOps route.Then with the pandemic and WFH situation, we were forced to rethink this approach as we could no longer give the build manually.We tried a number of things. While all of them worked fine in the short term, none of them emerged as a viable permanent solution.Below is a non-exhaustive list of everything we tried and why it failed.Our non-tech readers can gladly skip to the next section, where we talk about what actually worked and how it benefited us.1. We started giving builds via TestFlight. Why it failed: It took a lot of time for a build to upload and be available to QA, thereby eating up time even for small changes.2. We then tried out giving Ad hoc builds for our internal testing which saved us some time. Why it failed: The developer had to manually upload the .ipa file to our S3 bucket via Jenkins which was taking up the developer’s time.3. Then we moved to Fastlane deployments. This really saved the manual upload of the build to S3 time every time. Why it failed: It was still going via the developer’s system. It would take around 40 minutes for Fastlane to Build and Upload to S3 automatically. That meant the developer was blocked off work during these 40 minutes. We gave 2–3 builds daily, which meant about 2 hours of our developer’s dev time daily. Also, our app build was now fully dependant on that developer’s system. What if the developer is on leave or his/her system crashes?4. We then thought of installing a Mac Mini in our office premises and install this Fastlane workflow on that machine and use Jenkins to trigger builds. Why it failed: It was difficult to monitor the Mac Mini remotely and the internet might not always be stable 24/7. It would also be very difficult to debug the Mac Mini system remotely in case of any issues in the setup.5. AWS recently launched their macOS machines in cloud Why it failed: They were too costly to be used for the build and would not have justified our budget.After trying out multiple solutions, we finally decided on Github Actions. It was a much more integrated solution and best suited for our use case. It was also the solution that best suited our budget. Now all our staging and prod builds automatically go via Github Actions.In a later section, we get into the details of how we implemented this development. But first let's take a look at how this benefited the platform, the team, and the business overall:
It freed up the developer’s time that was getting wasted due to the system's dependency. We ended up saving around 1.5–2 hrs of our developer time daily
Now our developers can focus on building great apps rather than worrying about giving out builds
Saving developers’ time eventually saves business cost
We eliminated the dependency on a single developer machine
We were also able to give out builds much faster as the Github ecosystem is very robust and the chances of failure are fewer
Now to the tech-y part of the article. The Implementation. Here’s how the team implemented this development, broken down into 3 Jobs:
In a Nutshell :
Daily Cron job is triggered at 10:00 AM IST (4:30 AM UTC) to run this workflow
Workflow is divided into 3 jobs
macOS machine is only used for building IOS App and generating its files
Ubuntu machines are used for other things like uploading files to AWS and BrowserStack
The workflow can also be triggered manually on-demand basis
Workflow Steps Overview :
Job 1: To generate IOS build for ad-hoc distribution via URL
Job 2: Upload the app files to S3
Job 3: Upload app files to BrowserStack for Automation Testing
Job 4 (Optional): Upload build to TestFlight if it is prod environment
Workflow In-DepthSummary: This workflow is aimed at providing IOS build for testing on a daily purpose. The jobs are run synchronously.No of jobs in the workflow : 3No of machines used: 2 (macOS and Ubuntu)Job 1 Steps (Build) :
Send message in Slack that Build has started
git clone from the staging branch in the freshly booted macOS machine
Check if the correct Xcode version is used
Configure keychain. Distribution certificates need to be in the OS keychain for Code Signing. So as we get a fresh machine always, we need to create a new keychain and upload certificates there
Certificates and profiles are in git in encrypted form. So they need to be decrypted form to be able to upload to the keychain. So we decrypt the certificates and profiles in this step
After decrypting the certificates, upload them to the newly created keychain in Step 4
Xcode needs to find profiles in a certain path. So profiles are uploaded to this path. These profiles are basically provisioning profiles that tell the Xcode what certificates to use for Code Signing and which devices are whitelisted for the app to be installed.
Install NodeJS so that the npm command can be used
Install NPM packages. Make sure to have a package-lock.json for correct NPM packages to be installed
Install Pods. These are IOS Libraries. Make sure to have a Podfile.lock file to install the correct library version.
Build and archive the app. The app is converted into binary format
Export the archive file into IPA files which can be distributed to users for testing.
Send message in Slack that Build has completed
Upload app files in storage so that they can be retrieved later across jobs.
Job 2 Steps (Uploading to S3) :
Send message in Slack that Build is uploading to S3
Download the app files from the shared storage from Job 1
Configure AWS credentials
Copy app files from ubuntu machine to S3
Send message in Slack that Build is ready for testing
Job 3 Steps (Uploading to BrowserStack) :
Upload app files from the Ubuntu machine to BrowserStack so that whenever Automation tests are run, the BrowserStack always has the latest app
Perform Unit and UI testing of the whole app
Prepare Testing Report and publish it in Slack
Job 4 Steps (Uploading to TestFlight) :
Upload the app build to TestFlight if is a Prod Workflow
CI/CD is a must in today’s world which really enhances your tech stack. It really saves a lot of boring, repetitive tasks that can be easily done by machines and also is very cost-effective.
This website/application including the ‘KristalFlex’ platform is owned, operated and maintained by Globalise Inc., a company incorporated in United States of America (the “Company”). The information placed on the platform is for informational purposes only and does not constitute as an offer to sell or buy a security. The Company reserves the right to make modifications and alterations to the content available on the platform. The Company is neither registered as an investment advisor nor as a broker dealer. The KristalFlex platform offers users access to investments in global securities through multiple stock exchanges and such other services as may be provided either directly or through its affiliates and partners. Price and availability of products and services offered on the platform are subject to change without prior notice. The Company will not be liable for any lack of availability of products and services you may order through the platform.
Do note that investment in securities market are subject to market risks. The valuation of securities may increase or decrease depending on various factors affecting the securities market. The services being provided by through the KristalFlex platform is not professional investment advice. All the recommended list of offshore funds, portfolios etc. being provided through the platform shall be/are in the form of generic research reports on offshore securities and shall be accessible by all authorised / subscribed users of these services at the same time.
The platform and the services thereunder are provided on an “as is” basis. The Company makes no warranties or representations, express or implied, on products offered through the platform. It accepts no liability for any damages or losses, however caused, in connection with the use of, or on the reliance of its product or related services. The Company is not responsible for any technical failure or malfunctioning of the software or delays of any kind and is also not responsible for non-receipt of any e-mails. You shall bear all responsibility of keeping the password secure and shall be solely responsible for the loss or misuse of the password.
Any descriptions of, references to, or links to other products, publications or services does not constitute an endorsement, authorization, sponsorship by or affiliation with the Company or its affiliates unless expressly stated otherwise. Any such information and/or products have not necessarily been reviewed by the Company and are provided or maintained by third parties over whom the Company exercises no control. Certain hyperlinks or referenced websites on the platform, if any, are for your convenience and forwards you to third parties’ websites. The Company will, under no circumstance, be responsible for the content, the accuracy of the information, and/or quality of products or services provided by or advertised on these third-party websites. The Company does not endorse in anyway any advertisers/ contents of advertisers on its webpages or the KristalFlex platform.
We are licensed in Singapore, Hong Kong and India. Kristal Advisors (SG) Pte. Ltd. presently operates under the CMS license by the Monetary Authority of Singapore (MAS). Kristal Advisors (HK) Ltd is licensed and regulated by the Securities and Futures Commission (SFC) to carry out Type 4 and Type 9 regulated activities and is not involved in the discretionary management of any collective investment scheme. Kristal Advisors Private Ltd. presently operates as a Registered Investment Advisor under the jurisdiction of the Securities and Exchange Board of India (SEBI).
This is offered only to Accredited and Institutional Investors as defined under the Securities and Futures Act, Chapter 289 of Singapore (“Act”), which broadly comprises of regulated financial Institutions, large corporates, high net worth individuals and sophisticated investors.
An Accredited Investor is an individual
Whose net personal assets exceed in value SGD 2 million (or it’s equivalent in a foreign currency) with value of his/her primary residence capped at SGD 1 million, or
Whose financial assets (net of any related liabilities) exceed in value SGD 1 million (or it’s equivalent in a foreign currency), or
Whose income in the preceding 12 months is not less than SGD 300,000 (or it’s equivalent in a foreign currency)
I agree to opt-in as Accredited Investor and will submit required documentation to confirm the same.