Platform agnostic build environments

By Patrick Ferris on 2020-12-29 11:27:59 +00:00

Table of Contents

Since it's inception, Docker has taken the developer world by storm. Whether it's mimicking larger complex system interactions like applications talking to servers with docker-compose, CI testing with simple dockerfiles or development environments for websites there are many use cases for docker. Docker relies on something called BuildKit. For OCluster this was too unreliable and not very flexible.

OBuilder provides a high-level abstraction for build environments by providing an infrastructure centered on two key components: a snapshotting filesystem and an execution environment.

A simplified diagram

OBuilder configuration diagram

In this simplified diagram there are two contexts, a current working directory of the user and OBuilder's context consisting of a sandbox and a store. In reality things are a little more complex than this, but it helps introduce the key concepts. OBuilder's typical execution is:

  1. After initial setup (creating the various store directories) the OBuilder context receives a spec file describing the job. Each operation (run, from etc.) is executed within the sandbox. Some of these have little actual effect like env which sets some environment variables. run and copy however have to actually be executed.
  2. For these operations, upon success, the state of the environment is stored by taking a snapshot and naming it the hash which is generated from the spec file and current operation.
  3. Sometimes you may be executing operations that have been done for. If this is the case then the result directory of the store should contain the associated hash and instead of replaying this, the state of the environment can be restored.