Using Go Flags in Tests

• go
• testing

Many developers are familiar with the flag package from the Go standard library. This package provides an easy method for adding command line options and argument parsing for Go programs. What may not be as well known is that flag can be used in Go unit tests as well.

The go test Command

To understand how flag might be used in go tests, first we need to understand how the go test command works. When you run go test it compiles all the files ending in _test.go into a test binary and then runs that binary to execute the tests. Since the go test binary is simply a compiled go program, it can process command line arguments like any other program – although with some caveats, as we will see.

Parsing Flags

The standard way of using the flag package in a program is to define one or more flag variables and the call the flag.Parse() command in the main() function of the program.

For example:

package main

import (
    "flag"
    "fmt"
)

func Greet(loc string) string {
    return fmt.Sprintf("Hello, %s!", loc)
}

func main() {

    var loc string

    flag.StringVar(&loc, "location", "World", "Name of location to greet")
    flag.Parse()

    fmt.Println(Greet(loc))
}

Which produces the following output, depending on the command line arguments passed:

$ go run hello.go
Hello, World!

$ go run hello.go -location 'San Francisco'
Hello, San Francisco!

There is nothing explicitly requiring us to call flag.Parse() from main(), we simply need a place to call Parse() once (and only once) before any of the flag variables are referenced. main() (or a function called by main() early in the execution phase) is an easy choice to achieve this.

Parsing Test Flags

The above approach does not directly work for Go tests, as we typically don’t write main() functions for unit tests, we simply write test functions and let the tooling handle things from there. Where do we call flag.Parse from, then?

Well it turns out, the test binary generated by go test is already using the flag package internally and calls flag.Parse() during normal operation. All we need to do is to define our flag variables as globals so they are known before running flag.Parse().

The easiest way is to define our global variable and immediately set it to the return value from flag.String, flag.Boolean or the like. For example:

package main

import (
    "flag"
    "testing"
)

var loc = flag.String("location", "World", "Name of location to greet")

func TestGreet(t *testing.T) {

    res := Greet(*loc)

    if res != "Hello, San Francisco!" {
        t.Errorf("String mismatch on test")
    }
}

Notice that loc is now a *string not a string in the previous example, so we need to use the dereferenced form, *loc, when calling Greet().

Now when we run our tests, we can see that the test will only pass when “San Francisco” is passed as an argument:

$ go test
--- FAIL: TestGreet (0.00s)
    hello_test.go:20: String mismatch on test
FAIL
exit status 1
FAIL    _/home/jbowen   0.001s

$ go test -location 'Los Angeles'
--- FAIL: TestGreet (0.00s)
    hello_test.go:20: String mismatch on test
FAIL
exit status 1
FAIL    _/home/jbowen   0.001s

$ go test -location 'San Francisco'
PASS
ok      _/home/jbowen   0.001s

OK .. But Why?

Using a command line flag to determine whether a test will pass isn’t a great use case, but there are other use cases for flags that make this approach worth considering:

Selectively Running / Not Running Certain Tests

One use case I have used test flags for is to only run a test (or tests) if a boolean flag is set. This comes in handy when writing code to access a network based API. By defining a flag, offline, I can precede any tests requiring network access with a simple test:

package mypkg

import (
    "testing"
)

var offline = flag.Bool("offline", false, "only perform local tests")

func TestSomethingRemote(t *testing.T) {

    if *offline {
        t.Skip("skipping test in offline mode")
    }

    ...
}

With the above code, TestSomethingRemote() will only be run if *offline is false. By passing -offline to go test we can automatically skip this test … perfect for doing quick tests when you might not have online access, such as when flying.

Note that go test has a built-in form of this filter with the -short option. Tests functions can query testing.Short() to see if the option is set and decide whether to run or return via Skip().

Passing In Secrets / Configuration

Some tests (again often when working with remote resources) may require values that should not be shared or stored in code repositories (e.g. API keys, passwords, usernames). One possible approach to providing these to unit tests without saving them to the git repository would be to pass them in as flag arguments (or, if there are several, pass in the name of a config file to be parsed for the values).

Generating ‘Golden’ Results

Mitchell Hashimoto (founder of HashiCorp and creator of tools such as Vagrant), outlined a use case for using test flags to save ‘golden’ results when testing against a remote resource in his presentation, Advanced Testing With Go.

Golden results are a way of easily testing more complex output or state in some code by saving output from known good unit tests to a file and then using this output to compare against in future tests. This is especially useful when format or content might change in the future. Mitchell’s example shows how an -update flag can be used to trigger the go test program to generate new golden files rather than perform tests. In this way, the same code is used for generating and comparing the golden files.

Caveats

Piggybacking on the built-in behavior of go test does present a few issues that one needs to be aware of:

Overlapping Flag Names

The go test command has a number of flag options that it parses for its own use. If you define a flag using the same name as a built-in option, it will not be seen by the test binary itself as the go tooling will consume before executing the generated test binary.

For instance, if we were to define a boolean flag named short and try to access in our program it will not work because -short is flag for go test.

package main

import (
    "flag"
    "testing"
)

var short = flag.Bool("short", false, "short")

func TestShort(t *testing.T) {
    if !*short {
        t.Fail()
    }
}

If we call go test with the -short option, our test function still sees the short variable as false, because the test binary’s flag.Parse() never sees the option.

$ go test -short
--- FAIL: TestShort (0.00s)
FAIL

We can prove this is due to go test by compiling (-c) and saving the test binary as a file named “test” (-o test) and then calling it directly:

$ go test -c -o test
$ ./test
--- FAIL: TestShort (0.00s)
FAIL
$ ./test -short
PASS

One workaround for this issue is to use the -args option to go test to indicate all the remaining arguments are meant to be passed directly to the test binary:

$ go test -args -short
PASS

However, it is likely better to rename your arguments to not conflict with existing flags.

Positional Arguments

By default, tests from go test cannot parse positional arguments because, once again, go test intercepts them for its own use (as list of packages to test). As with the case above this can be remedied by using the -args option:

go test -args arg1 arg2

Wrapping Up

Hopefully this post has proven useful in understanding the hows and whys of using flags in Go tests. I plan to write more Go focused articles in the near future, so please stay tuned!