Interview with Donal O'Brien - We measure developer build times so that we can measure improvements and regressions
In this interview, we talk with Donal O'Brien from the core clients team at SoundCloud. He shares how they leveraged modularization, Tuist, and tools like Sourcery to overcome the challenges they faced while scaling the app. Moreover, he touches on some present challenges like developer awareness and the maintenance of the tools around the project.
Interviewee: Donal O'Brien
This week we are interviewing Donal. He’s currently part of the core clients team at SoundCloud where his role is to provide support for their engineers to be productive in their day to day work. The motivation for building Tuist came from the challenges SoundCloud was facing so we are pleased to have Donal sharing more about them in his interview.
Teams are typically structured in a cross disciplinary fashion with a well defined focus on some particular part of the business, e.g. listeners, creators, ads, growth etc. There are typically a mix of engineers from different disciplines, e.g. there might be two iOS, two Android and two backend on a single team. Of course, most teams also have an engineering manager, product manager and design / UX representative.
There are currently 16 iOS engineers working on features and two dedicated to infrastructure / tooling, i.e. the Core Clients engineers.
The project was originally a monolith, which was gradually migrated to a modularised setup over a few years. We still have quite a lot of code in the main app but most developers no longer need to touch this part on a frequent basis.
There are currently 27 modules / subprojects, which are dedicated to some business concerns in particular (e.g. Ads or Discovery). Most of these modules actually expose more than one framework. Some of these additional frameworks are used for testing, e.g. in order to share mocks with other parts of the codebase and some are used to run “example apps” that enable developers to work on features without the compile times of the main app.
Typically new features are developed using clean architecture such as VIPER.
We use some code generation techniques such as Sourcery for mock and test data generation. There’s a central collection of user scripts that we commit to the app’s repo for all sorts of repetitive tasks such as generating feature flag code and running common commands.
Before Tuist there were some concerns around the length of time it would take to generate a new module. In certain circumstances this could take up to a few days. Also, with developers relying on relatively verbose and sometimes outdated documentation to create the frameworks, inevitable inconsistencies in project structure and settings emerged.
Mainly CocoaPods but also some Carthage.
We don’t rely too heavily on any one external dependency but there are a few, which were introduced at the beginning and managed to spread throughout the codebase (i.e. an early version of ReactiveCocoa or the Specta unit test framework). However, it’s highly discouraged to use these frameworks currently for some well documented reasons. In that way we no longer rely on them but it’s hard to get rid of them!
Generally I think a thorough cost / benefit analysis should be carried out before committing to any third party dependency that might lead to negative business impact in the future such as the emergence of app instability or lower developer productivity.
There’s a strong emphasis on unit testing, with a much lower number of UI tests.
Yes, but only for legacy reasons. Developers are now encouraged to use XCTest directly, with the addition of some in-house conveniences on top.
We currently have over 16,000 unit tests. These are split roughly half and half between the modules and the main app. We have far fewer UI tests.
Some tooling around code and framework generation using Sourcery and Tuist’s scaffold command are some things that come to mind.
Deciding upon the correct hierarchical framework structure can be a tough call. Should you go with a single base layer to enable communication between frameworks or should there be multiple shared frameworks?
A clean build can take up to seven or eight minutes. There are constant ideas and initiatives to monitor and improve build times. Firstly we measure developer build times so that we can measure improvements and regressions. However, we want to improve on this further. We intend to continue modularising the app by creating new modules and extracting functionality from the main app into other existing ones. We intend to implement tooling that can relieve some of the burden on developers during modularisation. We’re also interested in the potential of build artefact caching and having the ability to dynamically link frameworks in development but statically for release. We’ve also identified some optimisations that could be possible around some of the build phases of the app that we intend to tackle in the future.
tuist generate 😃. Aside from that I also like the scaffold feature. We use that for templating new frameworks, which speeds things up quite a bit. I use tuist edit all the time to get code completion and all the other goodies that come with the IDE.
Yes, our entire setup is done using the ProjectDescriptionHelpers. In some cases, the manifest for a framework is a single line of code due to the use of ProjectDescriptionHelpers.
