Hyper Modular Swift Packages

Learn how to make SwiftPM scale to 100s of targets in one Package.swift file.

Hyper Modular Swift Packages
Photo by Paul Teysen / Unsplash

Swift Package Manager (SwiftPM) has been around now for a while, and it's achieved a pretty healthy adoption among Swift developers. It's goal is enable developers to put their code into a nice box, called a package, and eventually use that code in an application, say for iOS or macOS. Packages can also be dependencies of other packages though, and they can be used to make executables for other supported platforms.

So, your code gets packaged up so that it can be used in a variety of ways. Which is great. Inside the package, which is just a Package.swift file, the author describes the code in terms of products and targets. A product is the thing Β which gets added into an app (at which point we call it a library), or it is depended on by another package. So, the product it's the end result of what is usable from the package. They are not much more than just a collection of target. And a target, is just a bunch of Swift files.

Typically, the Package.swift looks something like this:

// From https://github.com/apple/example-package-deckofplayingcards/blob/main/Package.swift

// swift-tools-version:5.3
import PackageDescription

let package = Package(
  name: "DeckOfPlayingCards",
  products: [
    .library(name: "DeckOfPlayingCards", targets: ["DeckOfPlayingCards"]),
  ],
  dependencies: [
    .package(name: "PlayingCard", url: "https://github.com/apple/example-package-playingcard.git", from: "3.0.0"),
  ],
  targets: [
    .target(
      name: "DeckOfPlayingCards",
      dependencies: [
        .byName(name: "PlayingCard")
      ]),
    .testTarget(
      name: "DeckOfPlayingCardsTests",
      dependencies: [
        .target(name: "DeckOfPlayingCards")
      ]),
  ]
)

This is pretty easy stuff, we create an instance of Package and the initializer allows us to declare everything about the Package up front. Beyond this, SwiftPM doesn't really guide developers much further. For example, how many targets should you create? Should every target have tests? How can I make this maintainable? How can this scale?


Lets assume that we've got a huge codebase with lots of source files, and we want to breakup the codebase into logical groups so that we can better maintain and work with it. We're going to modularise the codebase. SwiftPM to the rescue!

Depending on how granular the logical groupings are, this could mean you have many modules. If we take the example above, and add 50 targets, each with their own test target, and say, 25 products - that is a lot of typing, and a lot of repeated Strings which cannot autocomplete. The way that the instance of the Package is constructed just does not lend itself to scaling beyond a few targets.


But, it's Swift code right, so surely there are alternatives. The first thing to note, is that the package doesn't need to be a constant defined all in one go. We can make it a variable, and append new targets and products as we go. So lets get started,

// swift-tools-version: 5.7
import PackageDescription

var package = Package(name: "DeckOfPlayingCards")

Next, we are going to define constant String values for all of the modules which we are going to create.

// MARK: - 🧸 Module Names
let DeckOfPlayingCards = "DeckOfPlayingCards"

This doesn't look like much, but now, when we need to reference the DeckOfPlayingCards module, we can use this constant, and Xcode will auto-complete it for us. No more typos or weakly defined strings.

We can also add an extension on String, for some handy helpers.

// MARK: - πŸͺ„ Package Helpers

extension String {
    var dependency: Target.Dependency {
        Target.Dependency.target(name: self)
    }
    var tests: String { "\(self)Tests" }
}
We're going to build up some code which can be included at the bottom of Package.swift files

Even with just this, our humble deck of cards Package is much improved,

package.dependencies = [
  .package(name: "PlayingCard", url: "https://github.com/apple/example-package-playingcard.git", from: "3.0.0"),
]

// MARK: - 🧸 Module Names
let DeckOfPlayingCards = "DeckOfPlayingCards"

package.products = [
  .library(
    name: DeckOfPlayingCards, 
    targets: [DeckOfPlayingCards]
  ),
]

package.targets = [
  .target(
    name: DeckOfPlayingCards, 
    dependencies: [
      .byName(name: "PlayingCard")
    ]),
  .testTarget(
    name: DeckOfPlayingCards.tests,
    dependencies: [
      DeckOfPlayingCards.dependency
    ]
  )
]

We can also improve how we handle and reference package dependencies.

Dependencies are often shown at the top of a Package.swift file, because the initialiser has the order of products, dependencies and then targets. When written in a top-to-bottom fashion as it usually is, implies that the dependencies are more important than the targets. This order is clearly intentional from the SwiftPM creators, as they have optimized for the consumer of a Package. In this scenario, the products of the package are the most important, followed which additional packages it depends on, the actual targets are just an implementation detail.

However, if you're making use of SwiftPM for a local package for your application, this logic is reversed somewhat. The dependencies are the implementation detail, and the targets are the most important, some of which might be package products.

With that in mind, lets move dependencies towards the bottom of the file.

// etc

// MARK: - 🧸 Module Names
let DeckOfPlayingCards = "DeckOfPlayingCards"

package.products = [
  .library(
    name: DeckOfPlayingCards, 
    targets: [DeckOfPlayingCards]
  ),
]

package.targets = [
  .target(
    name: DeckOfPlayingCards, 
    dependencies: [
      .playingCard
    ]),
  .testTarget(
    name: DeckOfPlayingCards.tests,
    dependencies: [
      DeckOfPlayingCards.dependency
    ]
  )
]

// MARK: - πŸ‘œ Dependencies

package.dependencies = [
  .package(name: "PlayingCard", url: "https://github.com/apple/example-package-playingcard.git", from: "3.0.0"),
]

extension Target.Dependencies {
  static let playingCard: Self = .product(
    name: "PlayingCard", package: "PlayingCard"
  )
}
3rd party dependencies are an implementation detail. Define static constants for their products which we depend on.

Looking at this further, there are still many issues.

  1. It's just far too much typing, so much boiler-plate and ceremony.
  2. Test targets are "equal" in relevance to regular targets, just defined in the same array of targets. But, we know that test targets will always have a dependent source code target. Furthermore, we are already assuming some conventions that test targets will always be named <TargetName>Tests.
  3. The important information, is really the name of the target, and other targets which it depends on (not really shown in this example). But this information gets lost in the mess of boiler plate.

Ideally, we want to declare only the specifics of each module, with minimal boilerplate, and keep the names prominent. Something like this maybe..

DeckOfPlayingCards.add(
  to: package, 
  dependencies: [.playingCard]
)

But even this doesn't quite fulfil our true needs because this example is a bit too simplistic to highlight everything. In real projects, packages will have many targets. Some of them will have cross-cutting functionality, such as Utilities or UIComponents. If you follow the excellent PointFree, then you might want modules for UserDefaultsClient, FileManagerClient etc. Some of these cross-cutting modules might need to be dependencies of every other module. They might also have common 3rd party dependencies.

Other modules in your package have a single focus, typically to provide a specific feature, UserProfileFeature or NewsFeedFeature. And these kinds of modules will also have a common set of dependencies.

Furthermore, perhaps, in addition to unit tests, you have snapshot tests or PACT tests. Each of these would have some common dependencies for the testing infrastructure, and should follow a naming convention. Yet more boilerplate in Package.swift.


Hopefully, by now it's clear that we need a datatype to describe the attributes of these modules. We can define one in our package helpers.

struct Module {
  var name: String
  var dependsOn: [String]
  var with: [Target.Dependency]
  var defaultWith: [Target.Dependency]
  
  var createUnitTests: Bool
  var unitTestsDependsOn: [String]
  var unitTestsWith: [Target.Dependency]
  
  var createSnapshotTests: Bool
  var snapshotTestsDependsOn: [String]

  var swiftSettings: [SwiftSetting]
  var plugins: [Target.PluginUsage]
  
  // there is actually more to this, but hopefully you get the idea
}
A model for a module.

We also need to be able to add a Module value to a Package value.

extension Package {
  func add(module: Module) {
    targets.append(
      .target(
        name: module.name,
        dependencies: module.dependencies,
      )
    )
 
    // there is actually more to this too, but you get the idea
  }
}

The next thing we need, is a succinct way to "add" a new module, if we have to create a new value, and add it to the package, this will detract from what the module actually is. Lets create a convenience builder

struct Module {
  // etc
  
  // 1
  typealias Builder = (inout Self) -> Self
  
  // 2
  static func builder(
    withDefault defaults: Module
  ) -> (Builder?) -> Module {
    { builder in
      
      // 3
      var module = Self(
        name: "TO BE REPLACED",
        defaultWith: defaults.defaultWith,
        swiftSettings: defaults.swiftSettings,
        plugins: defaults.plugins
      )
      
      // 4
      builder?(&module)
      
      // 5
      return module.merged(with: defaults)
    }
  }
  
  private func merged(with other: Self) -> Self {
    var copy = self
    // merge & union dependencies with other
    return copy
  }
}

Okay, so that seems like a lot, lets unpack it. Line 1 defines a Builder which is a function which receives a mutable Module and returns a Module. This is where we're going to customise each module.

Line 2, defines a function which receives a Module value, which we use to instantiate a mutable Module, line 3. This is then passed to the builder block, line 4, and finally we perform a merge with defaults to ensure that the dependencies are consistent, line 5.

Next step, we can create a concrete instance of a "builder",

// MARK: - πŸ”‘ Builders
let πŸ“¦ = Module.builder(
    withDefaults: .init(
        name: "Basic Module",
        dependsOn: [
            Utilities
        ],
        defaultWith: [
            .playingCard,
        ],
        unitTestsDependsOn: [
        	// any default test utilities?
        ],
        plugins: [ .swiftLint ]
    )
)

That's right, I've used the package emoji, πŸ“¦ to represent a function which makes a "basic module". And a more feature rich module could be a 🎁. It can be invoked like this,

package.add(module: πŸ“¦ { 
    $0.name = DeckOfPlayingCards
    $0.createSnapshotTests = false
  }
}

This isn't great though obviously, the important information here, is DeckOfPlayingCards which gets a bit lost. Lets create an operator to do away with all of the boilerplate.

infix operator <+
extension String {

    /// Adds a module to the package
    static func <+ (lhs: String, rhs: Module) {
        var module = rhs
        module.name = lhs
        package.add(module: module)
    }
}

With this in place, we can write the following,

DeckOfPlayingCards <+ πŸ“¦ {
  $0.createSnapshotTests = false
}

These three lines are doing a lot of heavy lifting. Because the operator closes over the previously defined package value, under the hood, we're adding the appropriate targets & products. The target will be called "DeckOfPlayingCards", the tests will be "DeckOfPlayingCardsTests". Dependencies on our own targets, or 3rd party packages, swift settings, and package plugins can all be set, either through the default πŸ“¦, or overriden inside the builder closure.

At this point, we're could stop. With our package helpers in place at the bottom of Package.swift we can define new module names, and add them using the <+ πŸ“¦ { } shortcut.

However, over time, as your project gains new functionality, more modules will get added, each being a directory on disc. Xcode faithfully reproduces this list in alphabetical order too. After 10 or so modules, it'll become clear that having a flat directory of modules does not scale very well. We need to be able to group some modules together, and place them in another directory. For example, all of those cross-cutting modules like Utilities , UIComponents or UserDefaultsClient could go into a Shared directory.

To get this to work, we will need to make use of SwiftPM's support for file paths, instead of assuming targets are in Sources/. We can extend the Module type to give it an optional group name.

struct Module {
  // etc
  
  var group: String?
  
  func group(by newGroup: String) -> Self {
    var copy = self
    if let group {
      copy.group = "\(newGroup)/\(group)
    } else {
      copy.group = newGroup
    }
    return copy
  }
}

Similarly, when we add the module to the target, we need to check the group.

extension Package {
  func add(module: Module) {
    // etc
    let path = "\(module.group ?? "")/Sources/\(module.name)"
    targets.append(
      .target(
        name: module.name,
        dependencies: module.dependencies,
        path: path,
        resources: module.resouces,
        swiftSettings: module.swiftSettings,
        plugins: module.plugins
      )
    )
    // etc
  }
}

Lastly, we need to update our operators so that we can include the concept of groups. To do this, we can take inspiration from SwiftUI, and make a GroupBuilder

protocol ModuleGroupConvertible {
  func makeGroup() -> [Module]
}
extension Module: ModuleGroupConvertible {
  func makeGroup() -> [Module] { [self] }
}
@resultBuilder
struct GroupBuilder {
  static func buildBlock() -> [Module] { [] }
  static func buildBlock(
    _ modules: ModuleGroupConvertible...
  ) -> [Module] {
    modules.flatMap { $0.makeGroup() }
  }
}
struct ModuleGroup: ModuleGroupConvertible {
  var name: String
  var modules: [Module]
  init(_ name: String, @GroupBuilder builder: () -> [Module]) {
    self.name = name
    self.modules = builder()
  }
  func makeGroup() -> [Module] {
    modules.map { $0.group(by: name) }
  }
}

This is a bit of plumbing in our package helpers, which we can use to update our operators to use @GroupBuilder values.

infix operator <<&
extension String {

    /// Nest module groups
    static func <<& (
      groupName: String, 
      @GroupBuilder builder: () -> [Module]
    ) -> ModuleGroup {
        ModuleGroup(groupName, builder: builder)
    }
}

infix operator <&
extension String {

    /// Collect modules together under a logical directory
    static func <& (
      groupName: String, 
      @GroupBuilder builder: () -> [Module]
    ) {
        let modules = ModuleGroup(
          groupName, 
          builder: builder
        ).makeGroup()
        for module in modules {
            package.add(module: module)
        }
    }
}

With these building blocks in place, it is now possible to describe our hyper-modularized package in terms of its modules and their logical grouping. Consider the following expansion of this package,

"CardGames" <& { // top-level directory
  "Poker" <<& { // nested directory inside "CardGames"
    PokerRules <+ πŸ“¦ {
      $0.createSnapshotTests = false
    }
    TexasHoldEm <+ 🎁 {
      $0.dependsOn = [
        PokerRules
      ]
    }
    // Add other modules for other poker variants
  }
   
  // Expand with other card game families etc
}

"Shared" <& {
  DeckOfPlayingCards <+ πŸ“¦ {
    $0.createSnapshotTests = false
  }
}

Try out the techniques described in this post by copying the package helpers into you own Package.swift file. This gist should be added at the bottom of your file. At the top, you should redefine your package value as a variable with just a name.

Swift Package Manager DSL
Swift Package Manager DSL. GitHub Gist: instantly share code, notes, and snippets.