Local Caching in AWS CodeBuild

What is AWS CodeBuild

AWS CodeBuild is a completely overseen persistent incorporation administration. You simply indicate the area of your source code, pick your construct settings, and CodeBuild will run manufacture contents for accumulating, testing, and bundling your code. There are no servers for provisioning and scaling purposes, or software for installation, configuration, and operation.

 

Build Caching in CodeBuild

A cache can store reusable bits of your build environment and use them over various build. By using a cache, you can save more time in the project build setup.

AWS CodeBuild now enables you to store antiques locally on the build with little cost as well as quicker reuse. You could just utilize Amazon S3 as the cache to store your ancient rarities between builds. Despite the fact that S3 caching permits the reuse of relics, it accompanies a network cost and inactivity because of the related file transfer.

Now AWS CodeBuild supports Local caching, which makes it feasible for you to continue intermediate build artifacts locally on the build host so that they are accessible for reuse resulting in build runs.

 

Two types of caching

1) Amazon S3 Caching

It stores the cache in an Amazon S3 bucket that is accessible over different build hosts. This is a decent choice for small intermediate build vestige that is more costly to build than to download. This isn’t the best alternative for enormous build artifacts since they can set aside a long time to transfer over your network, which is depended on the build performance.

2) Local Caching 

Local caching stores a cache locally on a host that is accessible to the build host only. This is a perfect and alternative option for build artifacts since the cache is immediately available on the build host. This implies that build performance is not impacted by network transfer time. Also, do note that if you choose to build, performance is not impacted by network transfer time. While choosing local caching, you have to choose one or more of the following cache modes.

Local caching increases build performance for:

    Projects with a large, monolithic source code repository.

    Projects that create and reuse many intermediate build artifacts.

    Projects that build large Docker images.

    Projects with many source dependencies.

 

How to use local caching

  1. Open AWS CodeBuild console at https://console.aws.amazon.com/codesuite/codebuild/home.
  1. Select Create project.
  1. In Project configuration, enter a name and description for the build project.
  1. In Source, for Source provider, select the source code provider type. We use an AWS CodeCommit repository name in this example.
  1. For Environment image, choose the Managed image or Custom image, as appropriate. For environment type, choose Linux or Windows Server. Specify a runtime, runtime version, and service role in relation to your project.
  1. Configure the build spec file for your project.
  1. In Artifacts, expand Additional Configuration. For Cache type, choose Local, as shown here.

 

 

Local caching supports the below mentioned caching modes:

a) Source cache mode caches Git metadata for primary and secondary sources. After the cache has been created, subsequent builds pull just the change between commits. This mode is a decent decision for projects with a perfect working directory and a source that is a huge Git repository. If you are choosing this option and your project does not use a Git repository (GitHub, GitHub Enterprise, or Bitbucket), the option is ignored. 

 

b) Docker layer cache mode caches subsist Docker layers. This mode is a perfect selection for projects that build or pull large Docker images. It can avoid the performance matter caused by pulling large Docker images down from the network. 

Note:

  • Docker layer cache can be used only in the Linux environment. 
  • The privileged flag should be set so that your project has the appropriate Docker permissions.
  • The security implication should be considered before you use a Docker layer cache.

 

c) Custom cache mode caches directories that specify under the build spec file. This mode is also a perfect choice if your build structure is not suitable for one of the other two local cache modes. If you use a custom cache:

*Only the directories can be specified for caching rather than specifying individual files.

*Symlinks are used for referencing cached directories.

*Cached directories are merged to your build before it downloads its project sources. Cached items will be overridden if a source item has an identical name. Directories are defined using cache paths in the build spec file.

 

To use source cache mode

In the configuration for build project, under Artifacts, expand the Additional Configuration. Choose Local for Cache type, choose Local. and select Source cache, as per the below image.

 

 

 

To use Docker layer cache mode

Under the build project configuration, in Artifacts, enlarge Additional Configuration. For Cache type, click Local. Select Docker layer cache, as shown here.

In Privileged, choose the ‘Enable this flag if you want to build Docker images or want your builds to get elevated privileges’. This allows elevated privileges to the Docker process running on the build host.

 

 

 

To use custom cache mode

Under your build spec file, need to add the cache path as mentioned below.

Under the build project configuration, in Artifacts, expand Additional Configuration. For Cache type, choose Local. Select Custom cache, as shown here.

YAML
version: 0.2
phases:
 pre_build:
 commands:
 - echo "Enter pre_build commands"
 build:
 commands:
 - echo "Enter build commands"
 
cache:
 paths:
 - '/root/.m2/**/*'
 - '/root/.npm/**/*'
 - 'build/**/*'

 

This article clearly explains in detail on how to enable and configure the cache setting for AWS CodeBuild. These things can save significant build time and improves flexibility by avoiding external network connections to an artifact repository as well.

 

Leave a Reply