I recently began centralizing my CI/CD pipelines in a Github repository named “PipelineTemplates”. For years, I have copied/pasted and slightly modified pipelines, workflows, and configurations between repositories. These pipelines handled tasks like linting and formatting my Python and Go code on PR open to the main branch, building/compiling code, creating Github tags and releases, deploying artifacts, and most of them were performing the same larger role, but in ways that were tailored to each individual repository.

The Problems

The practice of copy/pasting pipelines worked for a while, especially for smaller, more repeatable and predictable patterns, but over time became difficult for me to maintain. I’ve used tools like Renovate and Dependabot to help to keep my pipeline references up to date and secure, but it meant merging many of the same PRs as Github Actions and CLI tooling versions change upstream. When I would create a new repository, I would have to go back to the last repository I remembered having the pipeline(s) I wanted to use, copied updated everything, and tweaked them specifically for the new repository. My pipelines became more like a genetic lineage than a cohesive strategy; each new repository got a pipeline with some of its DNA from the parent pipeline, and then passed any new modifications onto the next iteration.

One of the biggest challenges I had with this pattern was staying on top of updates for components like Github Actions. Every time the actions/checkout Action would change, for example, I would have to go back to every repository’s pipeline that used that Action and bump the version. As I approached, and eventually exceeded, 100 code repositories, it became increasingly obvious that this pattern would not work forever. And to make matters worse, Github Actions isn’t the only CI platform I use! I self-host Concourse CI, Woodpecker CI, and Jenkins, I have tinkered with Dagger and Taskfile pipelines, and I have used Gitlab CI for code repositories I host on Gitlab.

The problems I wanted to solve were:

  • Code duplication from copying between repositories
  • Central creation and maintenance of pipelines, workflows, and versioned components
  • Keeping dependencies and pipeline components up to date
  • Consolidating functionality like linting and formatting, builds, releases, and deployments

The Solution

I started a plan to centralize all of my pipelines in 1 single repository, with the idea being I could freely move the repository around and not get tied to any 1 platform. I would store all of my pipelines (Github Actions and reusable workflows, Gitlab CI components, Concourse pipelines, etc) in this repository, and create “calling” pipeline stubs in consuming repositories to make use of the centralized pipelines. If I ever moved the centralized repository to another code forge, for example from Github to Gitlab, I could simply update the caller pipelines with the new location.

I created my PipelineTemplates repository to store my pipelines, and a PipelineTemplates-Test repository to prototype and test them before using them in my “real” repositories. I started with a go-build Action and used the PipelineTemplates-Test repository’s test-build-go-flat.yml pipeline to test it. I implemented component versioning using bump-my-version and created some pipelines and scripts to check the pipeline component paths for changes on PR merge to main, and bump any version files for changed components. The pipeline uses git history and Conventional Commit messages to determine the version bump type (major, minor, or patch), and a script to handle the bumps.

These initial pipelines helped me work out the repository’s structure, and taught me some things about wiring workflows and pipelines together. For instance, Github is very strict about how you structure pipelines; everything must go under either .github/actions/ (for Github Action components) or .github/workflows/ (for reusable workflows that encapsulate repeatable logic that other repositories consume). For Actions, you cannot call scripts from outside the .github/actions/<action-name>/ path, which put a dent in my plan of sharing the same scripts across multiple CI platforms. I set out with a plan to write purpose-built scripts and write the pipelines that call them as simple “wrappers” that pass inputs into the script to determine behavior, so I could easily move CI platforms without significantly altering the “business logic” of a given pipeline.

Having worked with Concourse CI and Azure DevOps, this was a shift in my understanding of how I should build the PipelineTemplates repository. While I could still force each Action to clone the whole PipelineTemplates repository to make a script outside the Action’s path accessible, that strategy introduces moving parts that make breakage more likely, and so I decided to embrace each platform’s philosophies and treat PipelineTemplates as a monorepo.

Info

As of 2026-07-18, I am still evaluating Dagger as the solution to this problem. The appeal of Dagger is that you “write 1 pipeline and run it anywhere.” The Dagger modules I write would do the same exact thing whether I ran them locally from the CLI, in a Github Action, a Concourse pipeline, or a Woodpecker CI pipeline. All of the important logic is encapsulated in the Dagger module, and the pipeline that calls it can be simplified to just passing inputs into the module.

I have not fully embraced this, but as I build out this repository the concept becomes more appealing, and I will most likely end up converting at least some of the pipelines to Dagger modules.

For each pipeline/workflow/component I created, I would create a “caller” pipeline in the PipelineTemplates-Test repository to test it. The test repository also serves as a set of example pipelines I can copy/paste into my “real” repositories to add the pipeline’s functionality. For example, this blog uses a pipeline named hugo-main.yml to call the PipelineTemplates repository’s hugo-site-main.yml workflow. The centralized pipeline orchestrates version bumping for versioned Hugo sites, building the site and uploading the static files as a pipeline artifact, creating a tag and release on Github with the static site assets, and deploying to 1 or more targets such as Github Pages, Cloudflare Pages, or Netlify. After much trial and error, I now have a configurable workflow that repeats these same steps for any Hugo repository I call it from. It supports versioned and unversioned sites, building from a Hugo site stored at the repository root or in a subdirectory, and it supports the various ways I might deploy a site to another platform. If the platform watches a branch for changes to publish, the workflow supports “publishing” the site to a branch, i.e. github-pages, or if the platform supports uploading a raw archive, the pipeline will upload one of the release assets. For example, Cloudflare Pages supports uploading a .zip archive of your site with the Wrangler CLI, and that’s exactly what the hugo-publish-cloudflare-pages.yml workflow does.

Keeping all of my pipelines in 1 place also makes it easier to document them. For instance, the Hugo site pipeline I described above is documented in docs/ci-cd/github-actions/workflows/hugo-site-main.

The Results

While I still have a lot of work to do to centralize all of the different pipelines I regularly use, the work I’ve done has already improved my process. I maintain a number of personal sites that are Hugo blogs, and I now have workflows to handle all of the different structures and publishing targets I use. If I ever want to publish to a new platform, or add steps (like security scanning or Lychee link checking), it is easier to add a single step and have every other Hugo repository benefit from it. Updating pipeline component and CLI tool versions is much easier with the PipelineTemplates’s Renovate workflow, which runs every day at 3am and creates pull requests to bump versions to a new release. Every repository that calls a pipeline in the central repository benefits from up-to-date versions, and as syntax changes, I only have to update in 1 place (and maybe make a slight tweak here and there to the calling pipelines).

This repository has taken the maintenance burden of keeping pipelines and versions up to date across many repositories, and turned it into a convenient one-stop-shop for all of my pipeline needs. If I ever want to use another platform, like Gitlab or Codeberg, I can easily isolate the pipelines for those forges into their own directories in the central repository, and they benefit from the automated Renovate scanning, too. The separate PipelineTemplates-Test repository helps me to mock and test new pipelines in a non-commital way, allowing me to make all of the mistakes I need to in a throwaway sandbox environment before converting real repositories to one of the centralized pipeline. And it makes documenting everything much easier, as whenever I make changes to the pipeline, I can just go update the pipeline’s documentation in the central repository in the same commit.

Because each pipeline component is versioned, I can use or SHA hashes to lock a calling repository to a specific version of the component, ensuring repdroducible and (ideally) idempotent runs each time the central pipeline is called. I can rest easy knowing “this ran last time, so it will run this time,” more or less.

Example

The PipelineTemplates-Test repository serves as both my testing environment/sandbox, and as a set of examples for calling pipelines stored in the central PipelineTemplates repository. For the purposes of this blog, I will discuss my Renovate workflow to demonstrate how I develop, test, and use these pipelines.

In the central PipelineTemplates repository, I have a .github/workflows/renovate.yml workflow definition. This pipeline runs on a schedule and accepts inputs from a calling repository or manual dispatch to control how the Renovate pipeline runs. It expects a path to a renovate.json file, but also supports repositories that do not provide this file. When a repository calls this pipeline and does not provide a renovate.json, the default configuration in PipelineTemplates/config/renovate/default.json is used.

When I want to use this pipeline in one of my other repositories, I create a workflow in .github/workflows/renovate.yml, and copy the test-renovate.yml pipeline from the PipelineTemplates-Test repository. If the consuming repository has a renovate.json file, I pass it to the central pipeline using the config-file input. When the pipeline runs in the consuming repository, it uses the logic from the central workflow in PipelineTemplates.

The calling repository may not look much simpler than the central pipeline, but the benefit is that I can just copy and paste this and make very minimal modifications to add the same Renovate functionality to new repositories. An example caller pipeline from a consuming repository might look like:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64

---
name: Run Renovate

on:
  schedule:
    ## Runs every day at 3am
    - cron: "0 3 * * *"
  ## Manual trigger
  workflow_dispatch:
    inputs:
      ## Control whether the run is informational/debug, or a "live
      #  execution that will open PRs and update a dashboard issue.
      mode:
        description: "Renovate mode"
        required: false
        default: "lookup"
        type: choice
        options:
          - extract
          - lookup
          - run
      log-level:
        description: "Renovate log level"
        required: false
        default: "info"
        type: choice
        options:
          - info
          - debug
          - trace

## The caller pipeline has to forward the permissions the
#  central pipeline needs.
permissions:
  contents: read
  pull-requests: write
  issues: write
  actions: write

jobs:
  renovate:
    ## Tells Github to use the renovate.yml from the central repository
    uses: redjax/pipelinetemplates/.github/workflows/renovate.yml@main
    with:
      mode: ${{ github.event_name == 'schedule' && 'run' || inputs.mode }}
      log-level: ${{ inputs.log-level || 'info' }}
      repository: ${{ github.repository }}
      ## If the consuming repository provides a renovate.json, pass it here
      config-file: renovate.json
      ## If no renovate.json file exists in the consuming repository and this
      #  is 'optional', the central renovate.json config will be used
      require-config: optional
      ## When 'true', the consuming repository running this pipeline will discover
      #  and monitor/interact with all of your other repositories on that platform.
      #  I almost never set this to 'true'
      autodiscover: false
      runner-image: ubuntu-latest
      renovate-author-email: "1111111+your-username@users.noreply.github.com"
    ## You must set these secrets in the consuming repository's secrets. The documentation
    #  for this pipeline describes the permissions each of these tokens need, and how to
    #  create them on Github
    secrets:
      renovate-token: ${{ secrets.RENOVATE_TOKEN }}
      gh-api-token: ${{ secrets.GH_API_TOKEN }}

This blog uses the central Renovate pipeline, here is an example of what a Renovate pipeline run looks like.

Future Plans

I still have a lot of disparate pipelines to bring into the fold in this centralized repository. I need to create centralized workflows and Actions for formatting, linting, testing, building, and deploying a variety of apps in languages I use (Python, Go, Bash, Powershell, Astro, etc). As I create new pipelines, I find more efficient ways of doing things, and I run into plenty of scenarios where the goal I set out with is impossible or infeasible. This process helps to refine the centralized pipeline into a repeatable, assembly-line style process, and forces me to write more generic and capable of pipelines, rather than customizing each workflow specifically to the repository it’s running in. It forces me to be more diligent and disciplined about how I structure the consuming repositories. This rigidity is also freeing, in that I no longer have to think about when I should start automating a repository and how I should do it. If I am writing a new Hugo site, I can add the calling pipeline very early on in the process and let the repository benefit from the workflow I’ve ironed out for building Hugo websites, and get right back to work on building the actual site.

I need to test moving the repository around code forges, and documenting the process; a big goal with this project was to keep all of my pipelines in a single place that I could easily backup or move to other platforms, with minimal interruption in the calling repositories. Part of this test will be converting an existing workflow from one platform to another, i.e. recreating the Hugo site workflows for Github into a Gitlab-friendly version, in case I ever move my code there.

I eventually want to spend time consolidating some of the logic embedded in the pipeliens into scripts or small apps I can re-use across pipeline technologies, which will make it easier to move between platforms.