ADR-002

Scan repository for hard-coded secrets

Known Issues / Todo

⛅ This page is draft and is subject to rapid change, and may not be fully accurate or complete

Date 05/06/2023

Status RFC

Deciders Engineering

Significance Construction techniques

Owners Dan Stefaniuk, Jon Pearce, Tamara Goldschmidt, Tim Rickwood


Date	`05/06/2023`
Status	`RFC`
Deciders	`Engineering`
Significance	`Construction techniques`
Owners	`Dan Stefaniuk, Jon Pearce, Tamara Goldschmidt, Tim Rickwood`

Context
Decision
- Assumptions
- Drivers
- Options
- Outcome
- Rationale
Consequences
Compliance
Notes
Actions
Tags

Context

To safeguard sensitive details like passwords, API keys etc. from being incorporated into code repositories, it is imperative that we employ secret scanning of the code. This safeguarding process should be conducted in two key areas. Firstly, on the developer’s machine, we utilise a git pre-commit hook to halt the inclusion of any secrets within the committed code. Secondly, as a safety net, a similar scan should be integrated into the CI/CD pipeline. Should a secret be detected within this pipeline, it is crucial that the pipeline serves as a gate to fail the build, subsequently blocking any related pull requests.

Decision

Assumptions

There is already a well-known and fit-for-purpose tool git-secrets in use that was selected as the outcome of a decision made around 4 years ago. The purpose of this document is to review that decision.

Drivers

Within NHS England, we are observing an adoption of the gitleaks tool, which is an alternative to git-secrets.

Options

There are three options presented in this decision record.

Git-secrets
- Repository metadata
  - Contributions
    - Number of contributors: 28
    - Number of commits: 110
    - Commit dates / frequency: last commit more than a half a year ago, very low frequency
    - Number of Stars & Forks: 11.1k & 1.1k
  - Implementation technologies: Shell script
  - Licence: Apache-2.0
- Features
  - Scan whole history
  - Scan single commit
  - Predefined set of rules: A very limited number of rules
  - Definition of custom rules
  - Definition of custom exclusions patterns
  - Entropy detection
  - Pre-backed Docker image
- Pros
  - A well-known tool that has been around for a while
- Cons
  - Rules and exclusion patterns are not easy to manage as no comments or metadata are allowed in the definition
  - No pre-backed Docker image
  - Activity of the repository has dropped (last commit a while ago)
Trufflehog
- Repository metadata
  - Contributions
    - Number of contributors: 69
    - Number of commits: 2050
    - Commit dates / frequency: last commit today, high frequency
    - Number of Stars & Forks: 11.3k & 1.3k
  - Implementation technologies: Go language
  - Licence: AGPL-3.0
- Features
  - Scan whole history
  - Scan single commit
  - Predefined set of rules
  - Definition of custom rules
  - Definition of custom exclusions patterns: Only whole files
  - Entropy detection
  - Pre-backed Docker image
- Pros
  - Entropy detection
  - Fast to scan the whole history
- Cons
  - AGPL-3.0 licence comes with conditions
Gitleaks
- Repository metadata
  - Contributions
    - Number of contributors: 135
    - Number of commits: 929
    - Commit dates / frequency: last commit three days ago, medium frequency
    - Number of Stars & Forks: 13k & 1.2k
  - Implementation technologies: Go language
  - Licence: MIT
- Features
  - Scan whole history
  - Scan single commit
  - Predefined set of rules
  - Definition of custom rules
  - Definition of custom exclusions patterns
  - Entropy detection: Set against a rule
  - Pre-backed Docker image
- Pros
  - Ease of managing rules and exclusion patterns as the configuration file uses the toml format
  - Entropy detection at a rule level
  - Fast to scan the whole history
- Cons
  - No full entropy detection as an option

Outcome

The decision is to support Option 3 and endorse the usage of the gitleaks tool. This decision is reversible, and the state of secret scan tooling will be monitored by means of the NHS England Tech Radar.

Rationale

This decision was made with the understanding that the chosen tool must support the NHS England Coding in the Open initiative/policy and also be compatible with usage in private repositories.

Consequences

As a result of this decision, any new repository created from the repository template should contain a secret scanning implementation based on gitleaks provided as a GitHub Action.

Compliance

Compliance will be checked by the GitHub Scanning Tool.

Notes

This is an addition to the GitHub Secret Scanning feature that should be considered to be turned on for any public repository within the NHS England GitHub subscription.

Actions

Update the NHS England Software Engineering Quality Framework accordingly