Guide: Scan secrets

• Guide: Scan secrets
  • Overview
  • Key files
  • Configuration checklist
  • Testing
  • Removing sensitive data

Overview

Scanning a repository for hard-coded secrets is a crucial security practice. “Hard-coded secrets” pertain to sensitive data such as passwords, API keys and encryption keys that are embedded directly into the code. This practice is strongly discouraged as it may lead to security incidents.

Gitleaks is a powerful open-source tool designed to identify hard-coded secrets and other sensitive information in Git repositories. It works by scanning the commit history and the working directory for sensitive data that should not be there.

Key files

• scan-secrets.sh: A shell script that scans the codebase for hard-coded secrets
• gitleaks.toml: A configuration file for the secret scanner
• .gitleaksignore: A list of fingerprints to ignore by the secret scanner
• scan-secrets/action.yaml: GitHub action to run the scripts as part of the CI/CD pipeline
• pre-commit.yaml: Run the secret scanner as a pre-commit git hook

Configuration checklist

• Add custom secret patterns to the configuration file to align with your project’s specific requirements
• Create a secret scan baseline for your repository by adding false-positive fingerprints to the ignore list
• Ensure that the GitHub action, which incorporates Gitleaks, forms part of your GitHub CI/CD workflow. It is designed to run a full scan as a part of the pipeline, providing additional protection against hard-coded secrets that might have been included prior to the rule additions or by bypassing the scanner
• Further details on this topic can be found in the decision record as well as in the NHSE Software Engineering Quality Framework where a usage of an alternative tool is shown

Testing

You can execute and test the secret scanning across all commits locally on a developer’s workstation using the following command

ALL_FILES=true ./scripts/githooks/scan-secrets.sh

Removing sensitive data

Here are some tools that can help in removing sensitive data, such as passwords or API keys, from the Git history

• rtyley/bfg-repo-cleaner
• newren/git-filter-repo

For additional guidance, please refer also to the official GitHub documentation.