Intro to Orb development for CircleCI

Reasoning

The issue we’ve faced at work is that we’ve had multiple projects, living in multiple repos, each with their own CircleCI config file. While this might work (ish), it’s no ideal for several reasons:

  1. It’s hard to maintain. If we want to change something in the config, we have to do it in multiple places.

  2. It’s hard to keep in sync. If we want to add a new job, we have to do it in multiple places.

  3. It scratches me wrong. as a programmer, I don’t like to repeat myself. I like to keep things DRY.

Approach

So to combat all of the above, i’ve looked into a few options from CircleCI. The first one I looked at was Dynamic Configs [link needed]. This is a good approach for some, but it felt a bit overcomplicated for us, and it didn’t feel like tool for the job. it’s not about having configs that dynamically change according to the project, it’s about having a set of reusable jobs that we can use across multiple projects.

So naturally the next step was authoring an Orb. If you’ve ever used CircleCI for iOS development, or used it to have Slack alert - you’ve used Orbs before. Orbs let you setup reusable jobs, commands, and executors that you can use across multiple projects. They’re a great way to keep your config DRY, and to keep your config in sync.

The main attracting points for us were:

  1. We could have a single source of truth for our jobs. If we wanted to change a job, we could do it in one place.
  2. We could have our executors setup in once place. this means that when we needed to update our Xcode version, or the MacOS version, we only needed to do it once.
  3. There’s a dev release before a prod release. this means that we don’t have to break everything everywhere at once; we can develop and test and only deploy when we’re ready and sure of ourselves.

Getting started

Firstly, I am going to assume you’ve got all the right permissions for the repo / project that your’e working on. i’d suggest you start off using the orb template that CircleCI provides. You can find it here. This will give you a good starting point for your orb. This will give you the basic file structure so we can keep working on it; now let’s go over the files and what they do, and why you might care about them:

@orb.yml

This is the “front page” of your orb, this is where you have the description of the orb, and it holds the description of the orb, the home and source URLs (where does this orb live? ), and perhaps most importantly - this is where you will define any orbs that your orb depends on (if any). So for our usage as an iOS app, we had to use the macos orb, and the slack orb which we used for notifications.

Commands

This folder is where you put all of your, well, commands. Commands are the building blocks of workflow, these should be concise, and encapsulated. We will go into how to write a command in a bit.

Executors

This is where you can define what you jobs will run on. Executors are the environment that your jobs will run in. This is where you can define the image, the environment variables, and the resources that your job will have access to. For our example, this is also where you define your mac version, and your xcode version, which are very important.

Includes

This is where you can any scripts you want to be bundled into the orb. These are BASH scripts that you can use in your jobs, and having them here means that you can keep your jobs clean and concise, and you have the ability to write tests for them (although I never did, because I’m a bad person).

Jobs

These are the meat and potatoes of your orb - these are the things that will be visible to the config calling the orb, this is where you will define what parameters you need, what executors you will use, and what commands you will run.

Examples

Here you should put what an example usage of your orb would look like. This is a good place to show off what your orb can do, and how it can be used.

Writing an executor

Like we mentioned, executors are the machines and environments that your jobs will run on. Here’s an example of an executor that we used in our orb:

description: >
  Default mac executor

macos:
  xcode: "16.0.0"
  resource_class: macos.m1.medium.gen1

shell: /bin/bash --login -o pipefail

environment:
  FASTLANE_XCODEBUILD_SETTINGS_TIMEOUT: "120"
  FASTLANE_XCODEBUILD_SETTINGS_RETRIES: "5"

So we have the description, which is pretty self explanatory. We then have the macos key, which is where we define the xcode version that we want to use, and the resource class that we want to use. We then have the shell key, which is where we define the shell that we want to use. Finally, we have the environment key, which is where we define any environment variables that we want to use. This is where we define the FASTLANE_XCODEBUILD_SETTINGS_TIMEOUT and FASTLANE_XCODEBUILD_SETTINGS_RETRIES variables, which are used by fastlane.

Writing a command

Like we mentioned above, commands are the building blocks of jobs. They can have as many or as few steps as you’d like, but they should be concise, and encapsulated. Here’s an example of a command that we used in our orb:

description: "Execute fastlane based on the lane name"

parameters:
  fastlane_type:
    type: string

steps:
  - run:
      name: Executing fastlane << parameters.fastlane_type >>
      command: |
        bundle exec fastlane << parameters.fastlane_type >>

This command is pretty simple, it takes a fastlane_type parameter, and then runs the fastlane command with that parameter. As you can see it’s quite reusable, this can be called from where ver and doesn’t depend on anything or any project.

Writing a job

Now that you’ve written a command, you can write a job. Jobs are the things that will be visible to the config calling the orb, and if your commands relies on other things happening before it, or after it, this is where you’ll define that flow. Here’s an example of a job that we used in our orb:

description: >
  Execute any lane

parameters:
  fastlane:
    type: string

executor: mac_executor

steps:
  - load_workspace
  - restore_cache_gems
  - fastlane_execute_lane:
      fastlane_type: <<parameters.fastlane>>

This job is a bit more complex, it takes a fastlane parameter, and then runs a series of commands before running the fastlane command. This is where you can define the flow of your jobs, and how they should be run. Lets explain a bit of what’s going on here:

1. The description is to explain to whatever is calling this job what the does the job do. pretty self explanatory.
2. The parameters are the things that you can pass to the job. In this case, we're passing a `fastlane` parameter, which is a string.
3. This needs to run on a mac machine, so we use a mac executer that we've defined in the `executors` folder.
4. We then run a series of commands, `load_workspace`, `restore_cache_gems`, and then `fastlane_execute_lane` which is the command we defined above. To call a job, you just need to use it's file name. so we saved the above job as `fastlane_execute_lane`, so we call it as `fastlane_execute_lane`.

Orb development

Congratulations! You’ve written your first orb, with a command, a job, and an executor. Now you’re ready to use it’s dev version and try it out. if you’ve used the template provided above, you’ll notice that this orb project has a .circleci folder, which holds a config. we’re not going go over how things work in that config, but what you need to know for now is - if you push your branch, a workflow will be triggered that will create a dev version of your build. This will happen in the publish job, and when it’s done you’ll be able to see the hash of your new dev orb ( which will be available for 90 days), or you can just use @dev:alpha to use the latest dev version of your orb. Now you can import it in whatever config that you want to use it in, for example:


version: 2.1

orbs:
our-orb: our-org/orb-name@dev:alpha

// The rest of your config...

Now you’ve got your orb in your config, you can start using it’s jobs in your workflows. Here’s an example of how you might use the job we defined above:

fastlane:
  jobs:
    - checkout_code
    - our-orb/fastlane:
        fastlane: "test"
        requires:
          - checkout_code

So by doing this, we’ve defined a workflow that will run the checkout_code job, and then run the fastlane job that we defined in our orb. This is a very simple example, but you can see how you can start to build up your workflows using the jobs that you’ve defined in your orb. You can also pass those variables through the pipeline, you can have them passed in as ENV variables, or you can hard code them, it’s up to you.

Once your’e done, if you merge your orb into master / main, and create a tag with a sem-ver version, your orb will be published and available for use in any config that you want to use it in.

Conclusion

This was a very high level overview of how to write an orb, and how to use it in your config. There’s a lot more that you can do with orbs, and a lot more that you can do with CircleCI. I’ve really enjoyed working with them, and the CircleCI documentation is nice and descriptive, but can be a tad overwhelming. I hope you’ve enjoyed this brief post, and I’d love to hear if you’ve got questions or comments. I’m always looking to learn more, and to help others learn more too.