How to Shift-Left Better with Git Hooks

Introduction

The philosophy of "shifting left" in software development is transforming the way we approach error and resolution. By moving the focus of error detection to earlier stages in the development cycle, teams can address issues when they are more accessible and less expensive to fix. Integral to this shift-left approach are Git hooks, powerful tools that allow us to enforce quality control right from the code-commit stage.

Understanding the Shift-Left Approach

The shift-left approach and Git hooks are transformative forces in software development, revolutionizing how we detect errors and maintain code quality. This article offers an in-depth look into these two interconnected strategies. Here's a summarized overview:

• The shift-left approach signifies a paradigm shift in error detection, advocating for early-stage detection and resolution during the software development lifecycle (SDLC).
• Git hooks provide a mechanism for automating workflows, ensuring quality control, and promoting consistency in a project's codebase.

Decoding the Shift-Left Approach

The shift-left approach brings this process closer to the development stage rather than the traditional method of addressing errors towards the end of the SDLC. The strategy is simple: catch and fix mistakes at the earliest stages, making them more accessible and less costly.

This approach aligns well with Agile and DevOps principles, promoting iterative development, frequent feedback, a development environment, and increased collaboration. The result is a smooth, streamlined development process with reduced chances of issues propagating downstream.

Git Hooks: An In-depth Overview

Git hooks are essential for automating and customizing the Git workflow, ensuring the quality of version control systems and consistency. These hooks reside in the .git/hooks directory of each Git repository and are executed before or after events like commit, push, and receive.

Developers can validate and modify commit messages and content with client-side hooks to enforce coding standards. On the other hand, server-side hooks enable policy enforcement and additional checks before accepting changes in the project's official history. By leveraging these hooks, teams can streamline their development process, maintain code quality, and implement and enforce project-specific security guidelines.

Installation and Initial Setup of Git Hooks for Shift-Left Approach

Git hooks are instrumental tools for facilitating automation and enforcing standards in your development tool and process. These hooks often utilize scripting languages such as bash, underscoring the importance of having at least a basic understanding of these languages. With that knowledge in hand, we can then focus on how to install and configure these pre-commit hooks initially, setting the stage for more advanced uses down the line.

Establishing Executable Scripts in the .git/hooks Directory

Setting up Git hooks begins with the creation of executable scripts inside the .git/hooks directory of your repository. The file's name for each script should correspond to the hook you're implementing, such as pre-commit, pre-push, etc. These scripts can be written in any scripting language (as long as the appropriate interpreter is available on the system) and executed each time the repository, corresponding repository, or Git event happens.

For instance, to create a pre-commit hook, you would make a new file named 'pre-commit' within the .git/hooks directory. For example, the file pre commit' could be a bash script running specific checks each time a commit is prepared to a remote repository.

Here's a basic example of a pre-commit hook script that prevents commits with 'TODO' comments:

#!/bin/sh

# Redirect output to stderr.
exec 1>&2

# Check for 'TODO' comments in staged files.
if git diff --cached --name-only -z | xargs -0 grep -i 'TODO'; then
  echo 'Commit contains TODO comments. Please remove them and try again.'
  exit 1
fi

Remember to make the script executable:

chmod +x .git/hooks/pre-commit

Surpassing Local Hooks Limitations with Shared Hooks

Git hooks, though powerful, are local to each repository and do not get cloned along with it. This limits their effectiveness in enforcing rules across a team. A way around this is to use shared hooks. These hooks are stored in a shared directory and linked to the repo using Git's core.hooksPath configuration variable.

To configure shared hooks, you can use the following command:

# Setting up shared hooks
git config --global core.hooksPath /path/to/your/shared/hooks

This command instructs Git to look for hooks in files in the specified directory instead of the default files in the default .git/hooks directory.

Client-Side vs. Server-Side Hooks

Git hooks are broadly categorized into client-side and server-side hooks. While client-side git hooks are designed to validate the content of Git commit commands and commit messages and enforce coding standards before committing changes, server-side git hooks validate access rights and enforce custom rules before accepting pushed changes.

Here's a look at some key client-side and server-side hooks:

Client-side hooks:

pre-commit: Check the entire post-commit below. For example, verify that the code doesn't contain debug statements before a commit.

#!/bin/sh

if git diff --cached | grep -q 'console.log'
then
  echo "Code contains console.log statement. Please remove them before committing."
  exit 1
fi

prepare-commit-msg: Allows certain points of the commits default message to be edited before the commit message author sees it. This hook could prepend a specific pattern to the commit message.

#!/bin/sh

COMMIT_MSG_FILE=$1

echo "TASK-1234: $(cat ${COMMIT_MSG_FILE})" > ${COMMIT_MSG_FILE}

commit-msg: Checks the final commit message. For example, it can test or enforce a commit message standard.

#!/bin/sh

COMMIT_MSG_FILE=$1
COMMIT_MSG=$(cat $COMMIT_MSG_FILE)

if [[ ! ${COMMIT_MSG} =~ ^TASK-[0-9]+:\ .+ ]]
then
  echo "Commit message does not follow the standard (TASK-XXXX: <message>)."
  exit 1
fi

pre-rebase: Can be used to prevent rebasing of files in certain branches.

#!/bin/sh

if [ $(git rev-parse --abbrev-ref HEAD) == "main" ]; then
  echo "You can't rebase the main branch!"
  exit 1
fi

Server-side hooks:

pre-receive: Invoked when updates are received on the server side.

#!/bin/sh

while read oldrev newrev refname
do
  if [[ $refname = "refs/heads/main" ]]; then
    echo "Direct push to the main branch is not allowed."
    exit 1
  fi 
done

update: Invoked for each ref to be updated.

#!/bin/sh

refname="$1"

if [[ $refname = "refs/heads/main" ]]; then
  echo "Direct updates to the main branch are not allowed."
  exit 1
fi

post-receive: Invoked after updates are accepted on the server side.

#!/bin/sh

echo "Push was successful. Notifying stakeholders..."

# Your notification logic here

Git Hooks and Shift-Left Approach: A Dynamic Duo

Acting as the skeletal structure of a robust shift-left strategy, Git hooks are tools that developers can effectively use to automate code reviews, standardize coding practices, and perform tests at the point of committing code.

Git Hooks: The Heart of a Shift-Left Strategy

Being a memorable part of a resilient shift-left strategy, Git pre-commit hooks are crucial in ensuring high-quality code and barricading bugs from infiltrating subsequent stages of the development lifecycle. These pre-commit hooks are scripts that Git triggers before or after events like a git commit command, git push call, or receive. Integrating these hooks into a shift-left strategy enables developers to spot and rectify issues early in the development trajectory, thus significantly conserving both time and resources.

Automating Code Reviews, Standardizing Coding Practices, and Enhancing Testing with Git Hooks

Git hooks are a powerful medium to automate several components of the development process, including code reviews, the standardization of coding practices, package name, configuration, and testing.

For example, the pre-commit hook can review code automatically before each commit. This code review can be done by deploying static code analysis tools like ESLint for JavaScript or Pylint for Python. This ensures that the committed code aligns with the team's coding norms.

To demonstrate, here's a pre-commit hook script configured for ESLint:

#!/bin/sh 

# pre-commit hook enforcing code standards 
ESLINT="node_modules/.bin/eslint" 

git diff --cached --name-only --diff-filter=d | xargs $ESLINT 

if [ $? -ne 0 ]; then 
echo "ESLint checks failed, fix them before committing." 
exit 1 
fi

Git “pre-” hooks can also be leveraged to automate testing procedures before commits. Unit tests or integration tests can be performed before every commit using the pre-commit hook command, or before every push using the pre-push hook. This empowers developers to detect and mend any failing tests early in the development lifecycle.

To illustrate, here's a pre-push hook script that runs unit tests using Jest:

#!/bin/sh

# pre-push hook running unit tests
jest

if [ $? -ne 0 ]; then
echo "Unit tests failed, fix them before pushing."
exit 1
fi

Embracing Example Scripts for ESLint and Jest Integration

Incorporating ESLint and Jest into your pre-commit and pre-push hooks adds a new dimension of automation and validation to your development workflow. While ESLint helps the developer uphold coding standards, Jest ensures all unit tests get a green signal before passing the code further down the pipeline.

The scripts are practical illustrations of how ESLint and Jest can merge with Git hooks to become an integral part of a shift-left merge strategy. They aid in automated code analysis, standardization of coding practices, and unit testing, thus ensuring high-quality code.

Best Practices for Implementing Git Hooks in a Shift-Left Strategy

Git hooks are powerful tools for automating tasks in your development process, but they must be used thoughtfully and efficiently to prevent disruption to the developer's workflow. The following are some best practices to consider when using Git hooks in a shift-left strategy.
**
Use Hooks To Reduce Repetitive Tasks**

Let's use the example of how to reduce repetive tasks using a pre-commit hook to scan for accidentally committed secrets using TruffleHog, a Python tool that searches for high entropy strings, which are typically secrets like API keys:

First, install TruffleHog globally (or within a virtual environment):
pip install truffleHog

Then create a .git/hooks/pre-commit file and add the following content:

#!/bin/sh

# Get list of staged files
STAGED_FILES=$(git diff --cached --name-only --diff-filter=ACM)

# Check each file for secrets with TruffleHog
for FILE in $STAGED_FILES
do
  truffleHog --regex --entropy=False $FILE
  if [ $? -ne 0 ]
  then
    echo "TruffleHog found potential secrets in staged file '$FILE':"
    echo "Please remove any sensitive data and commit again."
    exit 1 # prevent commit
  fi
done

# If no secrets were found, allow the commit
exit 0

The script runs TruffleHog on each staged file, checking for potential secrets. If TruffleHog detects any, it blocks the commit and prompts the developer to remove the sensitive data. While TruffleHog is a great open source solution we highly recommend GitGuardian as a more extensive solution for finding secrets in your repos.

Optimization to Ensure Rapid Hook Execution

One of the crucial aspects to consider when creating your Git pre-commit hooks is speed. A slow hook execution can potentially disrupt the developer's workflow, and therefore, it's essential to optimize your scripts to run only the necessary checks. Long-running operations like launching a full test suite or extensive linter might not be suitable for hooks that run on each commit like pre-commit install after-commit.

Here's an example file of a simple and optimized pre-commit hook that only checks staged files for any Python syntax errors:

#!/bin/sh

# Check for python syntax errors in staged files
git diff --cached --name-only | \

grep '\.py$' | \

xargs -I {} python -m py_compile {}

if [ $? -ne 0 ]; then
  echo "Python syntax check failed, fix errors before committing."
  exit 1
fi

Keeping Hooks Simple and Understandable

Git hooks should be kept simple and readable. A hook with complex logic can be difficult to understand, maintain, and modify in the future. If a check requires complex logic, consider creating a separate script or tool that the hook can call. This will keep the hook code clean and straightforward.

Here's an example of keeping things simple where a pre-push hook calls an external script for running unit tests:

#!/bin/sh

# pre-push hook running unit tests via external script

./scripts/run_unit_tests.sh

if [ $? -ne 0 ]; then
  echo "Unit tests failed, fix them before pushing."
  exit 1
fi

Providing Informative Rejection Explanations

An essential aspect of creating a useful Git commit hook is making sure it provides informative feedback. If a new commit hook rejects a commit message or a push, it should explain why clearly. Developers should understand what caused the failure and how they can rectify it.

For instance, consider a pre-commit hook that checks for TODO comments in the code and rejects the commit if it finds any:

#!/bin/sh

# pre-commit hook checking for 'TODO' comments
if git diff --cached -G'TODO'; then
  echo "Commit rejected. Found 'TODO' comments in your changes:"
  git diff --cached -G'TODO'
  exit 1
fi

In this example, if the hook finds any 'TODO' comments in the staged file changes, it prints out those changes to help the developer quickly locate and address the issue.

It's important to remember that hooks can be bypassed using the --no-verify option. When designing your shift-left strategy, you should consider this and possibly incorporate checks at later stages of your Software Development Life Cycle (SDLC) to handle cases where critical hooks may be bypassed.

Conclusion

I particularly like Git hooks because they help developers adopt a security culture in a straightforward and effective manner. With Git hooks in place, the development team can focus more on writing secure code and addressing security concerns earlier in the development lifecycle. It is a proactive approach that minimises the risk of security breaches and also saves time.

Git hooks not only facilitate the integration of security practices but also foster a security-conscious culture within the development teams.