SMS Blog

Cost-Effective CI/CD: Setting Up GitHub Actions Runners on AWS Spot Instances

Steve Youngblood/27 March 2025/

Introduction

When it comes to building, testing, and deploying infrastructure as code (IaC), using self-hosted runners for GitHub Actions is a great choice if you’re dealing with sensitive data or have strict security requirements. I chose GitHub AWS self-hosted runners, Philips Labs runner module, because they give me full control over my infrastructure, unlike public runners, which might not meet stringent security needs.

To save costs, I run these self-hosted runners on AWS Spot Instances, which provide access to unused AWS capacity at a lower price. Spot instances are ideal for non-critical workflows because they can be interrupted when AWS reclaims the capacity. The GitHub AWS runner module creates all the infrastructure needed for self-hosted, auto-scaling runners (outside of VPC and NACL requirements). It handles scaling up and down using AWS Lambda functions, ensuring that the number of runners is minimized when no workflows are active, which helps further reduce costs.

In this blog, I’ll show you how I used Terraform to set up AWS self-hosted runners on AWS Spot Instances, ensuring both security and cost efficiency for non-critical workloads like CI/CD pipelines. Spot instances allow you to leverage unused AWS capacity at a fraction of the cost of on-demand instances, making them ideal for jobs that can handle occasional interruptions. We’ll walk through the differences between spot and on-demand instances, explain how the GitHub AWS module automates the deployment of self-hosted runners, and outline the prerequisites, AWS resources required, and Terraform code needed to set it all up.

What is a Spot Instance?

AWS Spot Instances provide access to spare EC2 capacity at a lower price than on-demand instances, but they can be interrupted by AWS when the capacity is no longer available. This makes spot instances perfect for non-critical, fault-tolerant workloads like GitHub Actions runners, which do not run critical workflows.

  • Spot Instances: Lower cost, but can be interrupted. Ideal for short-term, flexible workloads.
  • On-Demand Instances: More expensive, but reliable, with no risk of interruption. Suitable for critical workloads that must always be running.

By using Spot Instances for self-hosted runners, you can significantly reduce operational costs for your CI/CD pipelines.

How Does the AWS Runner Module Work?

The GitHub App triggers an API Gateway in your AWS account via a webhook when a GitHub Actions workflow is initiated. The API Gateway forwards the event to a Lambda function that processes the webhook and sends a message to an SQS queue. This queue then notifies a scale-up Lambda, which registers a self-hosted runner with your GitHub organization or repository and requests a Spot Instance to run the job.

Once the Spot Instance is launched, the runner retrieves the required binaries from an S3 bucket and starts processing the workflow. A scale-down Lambda runs every 5 minutes, checking for idle runners. If a runner is found to be idle, it is automatically shut down, and the runner is deregistered from the GitHub organization or repository to avoid unnecessary costs.

image 6

Pre-Requisites

Before setting up AWS self-hosted runners on AWS Spot Instances, make sure you have the following tools and resources in place:

  1. AWS Account: You’ll need access to an AWS account to create and manage the necessary resources.
  2. AWS Vault: This tool helps securely manage and access your AWS credentials, ensuring a secure connection to AWS.
  3. GitHub Account: Required to access your repositories and set up GitHub Actions.
  4. GitHub Repository: You’ll need a repository with GitHub Actions enabled, where the self-hosted runner will be registered.
  5. Terraform: Terraform is essential for automating the infrastructure setup. We’ll be using local Terraform applies to deploy the resources needed for this setup.

AWS Resource Requirements

Deploying and managing self-hosted runners on AWS Spot Instances requires the following AWS resources, which will be automatically provisioned using Terraform:

  1. API Gateway: This is the entry point for GitHub to communicate with AWS, triggering the scale-up of self-hosted runners.
  2. EC2 Spot Instances: The runners will run on cost-efficient EC2 Spot Instances.
  3. IAM Role and Instance Profile: These provide the necessary permissions for the EC2 instances to interact with AWS services, ensuring smooth operation.
  4. Security Group: Defines the inbound and outbound traffic rules for the EC2 instances, ensuring secure communication.
  5. AWS Lambda Functions: Lambda functions are used to manage the lifecycle of the runners, including scaling up and down.
  6. SQS: Acts as a message broker, triggering the Lambda functions responsible for scaling up the runners.
  7. S3 Bucket: Used for storing runner binaries, artifacts, and other data needed during the workflows.
  8. CloudWatch Logs: Captures logs from the EC2 instances, making it easier to monitor runner activity and troubleshoot any issues that arise.
  9. AWS Secrets Manager: Stores sensitive data such as the GitHub App credentials (App ID, private key, and webhook secret) securely.

Terraform Code

Here’s a quick look at what this Terraform configuration does:

It sets up AWS self-hosted runners using Spot Instances to reduce operational costs. Sensitive GitHub App credentials, like the App ID, private key, and webhook secret, are securely retrieved from AWS Secrets Manager. The runners are deployed in a specific VPC with designated subnets for proper networking.

The configuration uses ARM64-based instance types (such as t4g.large and c6g.large, powered by AWS Graviton processors) to further optimize cost and performance. A maximum spot price is specified to keep costs under control. In addition, the setup includes Lambda functions to sync runner binaries, manage the lifecycle of the runners, and ensure runners are automatically scaled up or down. SSM access is enabled for remote management, and error logging is turned on to make troubleshooting easier.

terraform {
  required_version = ">= 1.0.0"
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 5.0"
    }
  }
}

# Retrieve the secret values from AWS Secrets Manager
data "aws_secretsmanager_secret_version" "github_app_secrets" {
  secret_id = "arn:aws:secretsmanager:us-east-1:123456789012:secret:github-app-secrets-abc123"
}

# Parse the secret's JSON string into individual variables
locals {
  github_app_secrets = jsondecode(data.aws_secretsmanager_secret_version.github_app_secrets.secret_string)
}

# AWS Runner configuration
module "github-runner" {
  source  = "github-aws-runner/terraform-aws-github-runner/modules/runners"
  version = "6.4.0"

  # Required AWS region
  aws_region = "us-east-1"

  # Required VPC and Subnets
  vpc_id     = "vpc-0abcd1234efgh5678"
  subnet_ids = ["subnet-123abc456def78901", "subnet-234bcd567efg89012", "subnet-345cde678fgh90123"]

  # Required inputs stored in Secrets Manager
  github_app = {
    github_app_id  = local.github_app_secrets["app_id"]
    github_app_key = local.github_app_secrets["private_key"]
    webhook_secret = local.github_app_secrets["webhook_secret"]
  }

  # Required to create Spot Instance requests
  create_service_linked_role_spot = true
  instance_target_capacity_type   = "spot"

  # Ensure all instance types have ARM64 architecture (ie. AWS Graviton processors)
  instance_types          = ["t4g.large", "c6g.large"]
  instance_max_spot_price = ".15"

  # Grab zip files via lambda_download, will automatically get the ARM64 build
  webhook_lambda_zip                = "../lambdas-download/webhook.zip"
  runner_binaries_syncer_lambda_zip = "../lambdas-download/runner-binaries-syncer.zip"
  runners_lambda_zip                = "../lambdas-download/runners.zip"

  # Enable access to the runners via SSM
  enable_ssm_on_runners = true

  # Enable error logging
  log_level = "error"
}

Conclusion

Setting up GitHub AWS self-hosted runners on AWS Spot Instances is a powerful way to optimize your CI/CD pipelines, balancing security, scalability, and cost efficiency. By leveraging Spot Instances, you can take advantage of AWS’s spare capacity at a reduced price, which is perfect for non-critical workloads like running GitHub Actions.

In this blog, we walked through the process of deploying these runners using Terraform, allowing you to automate and manage the necessary infrastructure with ease. We highlighted the importance of using tools like AWS Vault for managing credentials securely and outlined the essential AWS resources—such as API Gateway, S3 Buckets, and Lambda functions—that power this setup. We also explored how the AWS runner module automates the scaling of runners based on workload demands, ensuring that no excess resources are used when workflows are idle.

With the use of AWS Secrets Manager, we ensure that sensitive GitHub App credentials are stored and retrieved securely, allowing your setup to comply with best practices for data security. Furthermore, by choosing ARM64-based instances and setting a max spot price, you optimize both performance and cost savings.

Hopefully, you find this information helpful in deploying a flexible, cost-efficient, and secure solution for running GitHub Actions in your AWS environment. This setup is designed to meet the needs of modern DevOps workflows and streamline your CI/CD processes.

References

Leave a Comment





Posted in