Go Modules have a v2+ Problem

Go has a problem. Go modules place a strange naming requirement on modules version 2 or greater. Module names on modules v2+ must end in the major version ala …/v2, and communication of this rule has been weak. It's non-obvious, and the community at large does not understand it.

I have seen many very large projects including Google owned projects get it wrong.

I brought the issue up at my local Go meetup, and no one had ever heard about the rule. They were very skeptical the rule existed at all.

A little history

For a long time Go did not include a built in method for versioning dependencies. It did include the sometimes controversial go get method of fetching packages. The go get tool until recently simply cloned the HEAD of a packages primary branch based on URL and placed it in your $GOPATH.

In what has been oft linked to Go's "extremist position" on backwards compatibility, the expectation was largely that if you published a library, you were expected to maintain compatibility when at all possible.

A plethora of tools popped up over the years to fill this void, gopkg.in, glide, and even the at one time "official experiment" dep.

dep was the heir apparent to dependency management with backing from Google.

Seemingly suddenly however Russ "rsc" Cox sprung vgo (aka Go Modules) onto the world as the official Go solution. There was seemingly no input from the community in its creation, to the chagrin of many.

Whereas dep was a largely standard dependency manager in the vein of npm and the like, Go modules is a decidedly opinionated "Go" solution. It's handled as part of the language as a build step. When you go build if you are missing the versioned dependency, the build tool will fetch it.

The dependencies requirements are listed in go.mod and the exact versions expected are in go.sum. Both files are in their own sort of domain specific language. The prior has a syntax reminiscent of Go itself. The latter is a space separated list of dependency versions largely not intended for human consumption.

The build step will add to these files as necessary, and running go mod tidy will clean them up.

The v2+ problem in detail

Go modules place a very strange requirement on package developers. When a module hits major version 2 or higher, the module name must end in the major version. The advantage to this is it creates a separate package. The result is that a project can now depend on multiple major versions of the same library.

There are two tactics to achieving /v2 modules:

The first method is to change the module name in your go.mod

An example may seen in the go.mod of github.com/google/go-github seems simple enough, but requires you to also change any cross references within your package. While not the end of the world, it is error prone and even Google missed some.

The biggest problem with this method is that it breaks subpackages in non-module-aware versions of Go. As time goes by this becomes less important, but in older versions of Go the package name had to exactly match it's location. https://github.com/pkg/foo could not be github.com/pkg/foo/v2 when there is no v2 directory. The Go Team acknowledging this trouble and patched rudimentary module support into point releases 1.9.7 and 1.10.3 of the Go language to help ease the transition.

The second method and the method the Go Team themselves promote is leaving your v1 project as is in the root of your repo, and actually creating a v2/ subfolder containing your v2 library.

This feels weird, but the advantage is it works cleanly with older versions of Go that do not understand modules. Off the top of my head I only knew of a single project that went this route, and they appear to have gone back on it.

For what it's worth, the official Go Blog tried to clear the situation up some, and posted a write-up about it.

blog.golang.org/v2-go-modules

Even as a seasoned developer, I found this write-up somewhat impenetrable. For how important and unusual of a requirement this is, the write-up is not accessible enough.

Many projects including Buildkite's terminal-to-html simply didn't know about the requirement until they forget to handle it, and end up publishing broken releases. A v2+ Go library with a go.mod file, but not respecting the module v2+ rules is a compile time error for dependants. This is worse for users than not being a Go module at all.

Some projects like GORM sidestep the issue entirely by tagging their 2.0 releases as far flung 1.x releases. That something of a solution, but smells terrible.

What's to be done?

First, I think the Go Team needs to do a better job of shining a light on this rule. They need to scream it from the rooftops. They need at the very least a section of the documentation that lays it out clearly and in layman's terms.

Beyond this, there's room for warnings from the Go tooling. Subpackages importing older versions of the root package should throw a warning. It's way too easy an issue to miss.

There's room for similar notes on godoc.org / pkg.go.dev, although the usefulness is questionable.

The best and least likely option would be for Go to drop the requirement and make it optional. I believe this is a workable solution. Packages who want to allow users to include multiple versions can follow the v2 guidelines, and others don't need to.

An option I would personally promote, but is likely never going to happen, is to require /v0 and /v1 as well, such that you'd hit the issue right away rather than years into a project. Teach the convention early.

Discussion at: news.ycombinator.com/item?id=24429045