- Buck2, our new large scale open source build system is now available on GitHub.
- Buck2 is an extensible and powerful build system written in Rust and designed to make your build experience faster and more efficient.
- In our internal testing at Meta, we found that Buck2 completed builds twice as fast as Buck1.
Buck2, Meta’s open-source large-scale construction system, is now publicly available through the Buck2 websiteand the Buck2 GitHub repository . While it shares some similarities with other build systems (like Buck1 and Bazel), Buck2 is a rewrite from the ground up. Buck2 features full separation of core and language-specific rules with increased concurrency, integration with remote execution and virtual filesystems, and redesigned console output. All of these changes aim to help engineers and developers spend less time waiting and more time iterating their code.
Thousands of developers at Meta are already using Buck2 and performing millions of builds every day, completing builds twice as fast as Buck1. Our own internal analysis showed that engineers were able to produce significantly more code when their builds were run by Buck2, and we hope the wider industry will benefit as well.
Why rebuild Buck?
Build systems stand between a programmer and the execution of his code. As such, anything we can do to make the experience faster or more productive directly impacts how effective a developer can be. The goal of Buck2 was to keep what we liked about Buck1 (the core concepts and workflows) and take inspiration from post-Buck1 innovations (including Bazel , Adapton And Shake ) and focus on speed and enabling new ones Experiences.
Buck2’s design is based on the following principles:
- The core build system has no language-specific rules. Separating the rules from the core means the rules are easier to change and understand. The core of Buck2 is written in Rust and its language rules (e.g. how to build C++) are written in Starlark . This separation is in contrast to Buck1 (where all rules are written in the core) and Bazel (where C++/Java is written in the core).
- The build system is powered by a single incremental dependency graph , avoiding any stages (unlike Buck1 or Bazel). This decision eliminates many types of bugs and increases concurrency.
- The Rules API is designed to include advanced features for performance , along with dynamic (or monadic) dependency features for expressiveness. At the same time, these functions are carefully constrained to ensure that other properties (such as fast queries or hermetics) are not compromised.
- The open source version is almost identical to our internal version . The only parts that have been swapped out are the toolchains (pointing to our compilers’ internal copies) and remote execution (pointing to our internal servers). — Both have open-source alternatives. In addition, we publish all rules exactly as they are used internally. Additionally, we’ve split some of the logical components into separate boxes (e.g. Starlark , Superconsole , Allocative , Pavilion ) so they can be used outside of Buck2.
- Buck2 is written to integrate with remote execution , with the ability to run actions on remote computers. We use the same API as Bazel and have tested remote execution with Bauscheune and EngFlow . Although not required (nor really expected for people starting with the open source version), we are able to efficiently compute recursive digests and efficiently send them to remote execution.
- Buck2 is written to integrate with virtual file systems , not fully checking out the entire repository, but rather fetching it on-demand when the files are accessed. In particular, we support Sapling-based file systems . For a good integration we pay attention to file notifications (with guard ) and request both files and file digests without direct file operations. The benefit is that we can create virtual filesystems as quickly as a full checkout, but with the benefits of a much faster checkout and much lower disk usage.
The key takeaway from all of these improvements is that we designed Buck2 to be fast. In real use, Buck2 is significantly faster than Buck1, depending on the build. If no changes are made to the source code, Buck2 is available almost immediately on subsequent builds. When there is a lot of work to be done, Buck2 starts running faster and has greater concurrency. This increase in speed is a result of many of the factors above as well as due care and attention.
The user view
For end-users, Buck2 works much the same as Buck1 (which, to a first approximation, is quite similar to Bazel). A user defines goals in a BOCK file:
rust_binary(
name = “my_binary”,
srcs = (“main.rs”),
deps = (“:my_library”),
)
A user can then build with buck2-build //:my_binary . The value main.rs is a source file and :my library is a dependency BOCK file defined in it . It’s worth noting that Buck2 is mostly compatible with Buck1’s BOCK files.
Seksuaalisen terveyden haasteet voivat vaikuttaa moniin miehiin jossain elämänsä vaiheessa. On tärkeää ymmärtää, että apua on saatavilla, ja vaihtoehtoisia hoitomuotoja kannattaa harkita. Monilla on kysymyksiä, kuten lääkkeiden hinnat suomi, ja niihin voidaan löytää vastauksia luotettavista lähteistä. Esimerkiksi, verkkosivustolla https://pillerit-suomi.com/ voi olla hyödyllistä tietoa ja resursseja koskien hoitoja ja niiden kustannuksia.
In addition to the increase in speed, there are two other significant differences compared to Buck1 that are visible to the user.
Firstly, the console edition has been redesigned in addition to the super console library that we developed specifically for Buck2. The console shows a few more details and feels much more comfortable:
Second, there is a persistent daemon that maintains a single dependency graph. If you change a BOCKWhen you create a file, dependency, or source file, we invalidate those things in the dependency graph, and then request the output artifacts from the command line. There are several distinct dependency diagrams in Buck1, leading to phases such as goal diagram construction, action diagram construction, and then action diagram execution. There are also some operations that are not performed on the chart. When certain things change in Buck1, entire diagrams are discarded and not just the minimal parts are invalidated. With a single dependency graph, Buck2 is simpler, avoids more redundant work, and avoids explicit stages.„A la carte system bauen“ ).
The rule author’s view
While the user model closely follows Buck1, the approach to rules is completely different. In Buck for example there are many rules rust_binary used above. While a rule in Buck1 was a Java class built into Buck1, a rule in Buck2 is completely decoupled. Buck2 also comes with a “prelude” of rules that implement most of the Buck1 rules.
The Buck1 rules were tweaked over time, had numerous performance improvements and powerful features like graph traversal, but these rules were also expected to obey many complex invariants — sometimes violating those rules . For Buck2, the rules API is entirely in Starlark, which forced us to abstract these functions as generically reusable APIs to make them safe, expressive, and powerful — a difficult balance. We will address two such examples.
OCaml dependencies
The dependency structure of the OCaml library is difficult to express in Buck1. An OCaml library consists of a set of OCaml files. These need to be compiled in dependency order — so if A.ml uses B.ml you need to compile B.ml first. Bazel requires the A.ml dependency on B.ml to be explicitly written to the BOCK file. Both Buck1 and Buck2 leave this internal dependency implicit and run the tool ocamldep to infer it, which requires less maintenance when the structure changes. Buck1 ran ocamldepjust after parsing the BOCK file, which wasn’t really allowed, and dependencies weren’t tracked. So if you changed the imports too much, Buck1 would cause spurious compilation errors. Buck2 allows us to use the new dynamic_output primitive , which allows you to run a command, read the file’s output, and then wire the rest of the diagram — injecting the correct dependencies between the .ml files automatically.
C++ link dependencies
Consider the C++ linking model: to build a library, you typically need to link its build output along with the transitive closure of the build output of its dependencies. If you simply duplicate the set of dependencies at each level as you move up the diagram, you get this: An 2 )memory usage. In Buck1, many rules had custom code to capture this pattern, which relied on the ability to share Java values in memory and represent the dependencies in place within the rule structure (since there was no verified dependency graph). There are much stronger abstraction boundaries in Buck2, so such reuse needs to be made more explicit. Therefore, we introduced transitive sets (tsets) to capture this pattern of sets representing a transitive closure. By making tsets more abstract, we were also able to associate the tsets directly with the underlying dependency graph, meaning that this representation is both memory and computationally efficient.
Try Buck2 now
We look forward to people trying Buck2 and would appreciate any feedback ( GitHub issues are the best way). We assume that Buck2 will be most interesting for medium-sized multilingual projects. Visit the Buck2 First Home for more information.