Enhancing AppSec through Fuzzing in CI/CD Pipelines

In this blog, we are going to take a closer look at the concept of Fuzzing, using Go, and how to integrate it into your CI/CD pipeline. As a quick primer, Fuzzing is an automated testing technique that involves feeding random, unexpected, or invalid data to a program or API to uncover bugs and vulnerabilities. The core idea is to expose the program to inputs that developers may not have anticipated, thereby revealing flaws such as crashes, memory leaks, and security vulnerabilities.

Fuzzing in Go

Fuzzing capabilities are directly baked into the Go programming language (as of Go 1.18) itself. They offer a streamlined way to add fuzz testing into your development workflows. Leveraging Go's native tools and idioms, they make it easy and accessible for Go developers to adopt fuzzing techniques without a steep learning curve.

Integrating it into CI/CD Pipelines

Integrating fuzzing into CI/CD pipeline ensures that your code is continuously tested for unexpected inputs which could lead to vulnerabilities. Here is how you can integrate it into GitHub and GitLab:

Using GitHub Actions

Create a .github/workflows/go-fuzz.yml file in your repository:

name: fuzz

on:
  push:
    branches:
      - main
  pull_request:

jobs:
  fuzz:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v2

      - name: Set up Go
        uses: actions/setup-go@v2
        with:
          go-version: <version>

      - name: Install dependencies
        run: go mod tidy

      - name: Run Fuzzing
        run: go test -fuzz=<fuzz-function> -fuzztime=60s

Using GitLab CI/CD

Add the following in .gitlab-ci.yml in your repository:

stages:
  - fuzz

fuzz:
  stage: fuzz
  image: golang:<version>
  script:
    - go mod tidy
    - go test -fuzz=<fuzz-function> -fuzztime=60s

How Fuzzing Helps

Let’s take a look at how fuzzing can help you to identify security issues. As an example, we can use fuzzing to uncover an existing vulnerability (GO-2022-0211) within the net/url package. Specifically, the url.Parse function in this package accepts URLs with malformed hosts, allowing arbitrary suffixes in the Host field. These suffixes don’t appear in either Hostname() or Port(), potentially leading to authorization bypasses in certain application versions, as illustrated in the following examples:

Example Code

package main

import (
    "log"
    "net/http"
    "net/url"
)

func handleUserURL(input string) {
    parsedURL, err := url.Parse(input)
    if err != nil {
        log.Printf("Failed to parse URL: %v", err)
        return
    }
    log.Printf("Parsed URL: Host=%s, Hostname=%s, Port=%s", parsedURL.Host, parsedURL.Hostname(), parsedURL.Port())
}

func main() {
    http.HandleFunc("/example", func(w http.ResponseWriter, r *http.Request) {
        input := r.URL.Query().Get("input")
        handleUserURL(input)
    })
    http.ListenAndServe(":8080", nil)
}

Fuzz Test Code

package main

import (
    "net/url"
    "testing"
)

func FuzzURLParse(f *testing.F) {
    f.Add("javascript://alert(1)")

    f.Fuzz(func(t *testing.T, input string) {
        parsedURL, err := url.Parse(input)
        if err != nil {
            t.Logf("Failed to parse URL: %v", err)
            return
        }
        host := parsedURL.Hostname()
        port := parsedURL.Port()

        switch parsedURL.Host {
        case host + ":" + port:
            return
        case "[" + host + "]" + port:
            return
        case host:
            fallthrough
        case "[" + host + "]":
            if port != "" {
                panic("unexpected Port()")
            }
            return
        }
        t.Logf("Parsed URL: Host=%s, Hostname=%s, Port=%s", parsedURL.Host, parsedURL.Hostname(), parsedURL.Port())
    })
}

Running the Fuzzer

To run the fuzzer, simply use the following command:

go test -fuzz=FuzzURLParse -fuzztime=60s

Fuzzing for APIs

In a previous blog we demonstrated how continuous fuzzing can increase the overall robustness and quality of your code. We also integrated ParseRequestURI logic and a related fuzzing test into the test code. The result was the following unicode error:

What Happened?

The fuzz test showed that a database operation failed because the output of ParseRequestURI being unicode which is fed by the fuzzer.

// ParseRequestURI parses a raw url into a [URL] structure. It assumes that
// url was received in an HTTP request, so the url is interpreted
// only as an absolute URI or an absolute path.
// The string url is assumed not to have a #fragment suffix.
// (Web browsers strip #fragment before sending the URL to a web server.)
func ParseRequestURI(rawURL string) (*URL, error) {
    url, err := parse(rawURL, true)
    if err != nil {
        return nil, &Error{"parse", rawURL, err}
    }
    return url, nil
}

Further analysis shows that ParseRequestURI only checks for a certain range of ASCII code. Unlike Python’s urllib.parse library, which has unicode checks and states that input validation is required while using it, the Go documentation does not mention anything similar. It only says that It assumes that url was received in an HTTP request, This may lead to situations where developers use ParseRequestURI thinking that it performs the required checks for validating the URLs.

Conclusion

Integrating fuzzing into your CI/CD pipelines can significantly enhance application security by continuously testing for unexpected inputs that could lead to vulnerabilities. For applications written in Go, its native fuzzing capabilities can be easily adopted for this technique. This proactive approach helps identify and mitigate potential security issues early in the development process, ensuring more robust and secure applications.

Enhancing AppSec through Fuzzing in CI/CD Pipelines

Get A Demo