Tag: source

Sapling: Solving Branching Challenges for Meta’s Monorepo

Sapling is a scalable, user-friendly, and open-source source control system that powers Meta’s monorepo. At the GitMerge 2024 conference, we discussed the complexities of designing and implementing branching workflows for large monorepos. These workflows involve challenging tradeoffs between scalability and the developer experience.

Following the conference, we designed, implemented, and open sourced our monorepo branching solution within Sapling. While the code is already open source, this article shares valuable learnings on:

– How we resolved scalability and developer experience tradeoffs in the design and implementation.
– The problems this solution addresses.
– Feedback we received from other developers at Meta.

The key technical insight is that two workflows — non-mergeable full-repo branching and mergeable directory branching — solved all branching-related problems for the wide and diverse set of products built at Meta. We hope Sapling’s open-source code and the insights shared here will benefit the wider industry and open source communities.

—

### How Source Control Is Handled at Meta

At Meta, engineering teams work within a large monorepo with a single main branch. This approach enables:

– Unified dependency management
– Large-scale refactoring
– Easier collaboration
– Code reuse across projects

However, managing multiple versions of code within this setup presents challenges.

In multi-repo environments, teams typically rely on repository branches to manage different versions. Source control tools like cherry-pick and merge help manage differences between versions effectively.

In contrast, in a monorepo, repository branches are less effective. Branches affect the whole repository, so creating a branch means unrelated projects and dependencies remain frozen and quickly become stale. We refer to this approach as *full-repo branching.*

For workflows that do not require merging back to the main branch — such as product releases where the branch ceases to exist after release — full-repo branching works well. Sapling supports this workflow with the `sl bookmark` family of commands.

However, for product development workflows that require merging back to the main branch, full-repo branching is not scalable. This is because full-repo merges create merge commits with multiple parents, producing a wide (high branching factor) and non-linear commit graph.

In large monorepos, this non-linear history causes performance problems for operations like `sl log` and `sl blame`. Maintaining a mostly linear commit graph, where most commits have a single parent, is crucial for keeping these operations fast for all monorepo users, including those not using branches.

The core limitation is that full-repo branches are all-or-nothing. You cannot create a branch limited to the part of the codebase you own. If you need to patch legacy code or maintain a custom variant for a project, branching forks the entire repository, which is inefficient and cumbersome.

A common workaround was for teams to copy their code into multiple directories. However, this causes loss of standard developer tooling for branch management, resulting in duplicated effort and error-prone manual patching between directories.

—

### Directory Branching: Sapling’s Monorepo Branching Solution

To address these challenges, we introduced a new set of source control tools in Sapling to implement *directory branching*. This approach bridges the gap between multiple repository branches and maintaining code copies in separate directories.

With directory branching, you can treat directories within the monorepo much like traditional repository branches. You:

– Create branches by copying the code
– Maintain the code by cherry-picking and merging changes between directories
– View history for each directory, including all copies and merges

Crucially, while directory branches support merging between directories, at the repository commit graph level they appear as linear commits. This resolves scalability challenges associated with repo-level merge commits while still providing merging workflows at the directory level.

—

### How Directory Branching Is Implemented in Sapling

Directory branching in Sapling relies on a series of operations centered around the `sl subtree` command.

– To branch a directory, you use `sl subtree copy` to copy a directory or file — either at the current version or from any historical revision — to a new location in the repository.
– Sapling records metadata in the commit to track the source directory, source revision, and copy relationship, allowing complete recovery of the file histories in the new branch.
– If the code you want to branch is not yet in the monorepo, you can use `sl subtree import` to create a directory branch from an external repository.

Once you have a directory branch, you can use:

– `sl subtree graft` and
– `sl subtree merge`

to cherry-pick or merge changes between directory branches. These commands use the stored metadata to reconstruct directory relationships and perform three-way merges scoped to specific directory content.

The merge algorithm locates the common ancestor of the two directory branches (using the copy metadata) and applies a standard three-way merge as done for traditional repository merges, but limited to the directory content.

—

### Build System and Developer Tooling Integration

One major advantage of directory branching is that the latest versions of all directory branches are visible simultaneously. This means continuous integration (CI) can test against multiple branches with a single checkout, and teams can be confident no hidden old branches are unexpectedly still in use.

At Meta, we use Buck2 as our build system. When a component depends on another component using directory branching, we use Buck config modifiers (i.e., running `buck build` with the `-m` flag) to select the branch being built.

A downside of directory branching is that code searches may return multiple hits across branches. While it is important to recognize the searched-for code appears in multiple places, this can complicate browsing if results from multiple branches intermix. Advanced code search systems capable of ranking results can mitigate this issue.

—

### User Feedback on Directory Branching

Directory branching has been widely adopted within Meta by diverse engineering teams to help manage multiple versions of code effectively.

Some teams even combine full-repo branching and directory branching by freezing most of the monorepo on an old commit for stability, while using directory branching to merge changes for specific projects.

We identified three common reasons teams choose directory branching:

1. **Reducing CI costs or avoiding disruption:** Teams separate development and production versions of code, enabling better control over when changes deploy to production.

2. **Collaborative experimental changes:** Large groups collaborate over months on changes that might disrupt production. Directory branching offers better scalability than handling many stacked diffs to simulate a branch.

3. **Migrating from Git:** During migration, teams need equivalents of Git branches within the monorepo to complete consolidation. Directory branching provides this functionality without needing to consolidate all Git branches pre-migration.

Despite these exceptions, having a single version of code remains the monorepo’s default assumption. However, when one of the above scenarios applies, directory branching offers extensive branching workflows without sacrificing monorepo benefits.

—

### Future Work with Directory Branching

We are exploring leveraging directory branching to improve integration of Git repositories into the Sapling monorepo. Specifically, we plan a lightweight repository migration mechanism:

– Instead of committing all Git repository commits irreversibly into the monorepo history, we create a *soft link* to the external Git repository.
– Sapling can then load Git history on demand, lowering the barrier to entry for Git repositories joining the monorepo.
– This approach facilitates preliminary integrations without committing to migrating full history upfront.

This functionality will be available as an option with the `sl subtree import` command when working with external Git repositories.

Stay tuned — we will publish a dedicated article once we have more insights to share on this exciting advancement.

—

### Acknowledgements

Many contributors across Meta’s Source Control, Developer Experience, and Open Source teams helped design and implement directory branching in Sapling. We would like to thank:

Chris Cooper, George Giorgidze, Mark Juggurnauth-Thomas, Jon Janzen, Pingchuan Liu, Muir Manders, Mark Mendoza, Jun Wu, and Zhaolong Zhu.

—

To learn more about Meta Open Source, please visit our [website](https://opensource.fb.com).
https://engineering.fb.com/2025/10/16/developer-tools/branching-in-a-sapling-monorepo/