me at Gullfoss

Henrique Vicente

Software Engineer

GitHub Linkedin Instagram Flickr rss feed henriquevicente@gmail.com
D0F4DFBD257C1575

The ecosystem of the Go programming language

Monday, 22 March 2021.

Go is one of the most prominent general-purpose programming languages nowadays. Google, Apple, Microsoft, Amazon, and Adobe, to name a few, have been using the language extensively. It’s the language of choice behind multiple cloud computing projects such as Kubernetes, and it’s steadily expanding towards numerous areas of software development. In this article, you’ll find resources to learn about Go and its ecosystem.

If you want to see how people are using Go, check out the Go Developer Survey 2020 Results.

Gopher

Read also: Back to basics: Writing an application using Go and PostgreSQL and My Go mistakes.

Fact sheet
Paradigm Multi-paradigm: mostly imperative, and concurrent.
Designed by Robert Griesemer, Rob Pike, and Ken Thompson. Russ Cox and Ian Lance Taylor quickly joined them.
Sponsored by Google
Dates Design began in late 2007; Publicly Announced November 10, 2009 by the Go Team.
Influenced by C, Pascal, Modula-2, Oberon-2, CSP, Occam, Newsqueak, Limbo, Alef, BCPL, Smalltalk, APL, etc.
Typing Inferred, static, strong, structural.
Known for being Modern, readable, concise, garbage-collected, fast to compile, statically linked.
Site golang.org
Release cycle Major release every 6 months
Essentials Effective Go, Code Review Comments, Specification, and FAQ

Hello World in Go

1
2
3
4
5
6
7
8
9

package main

import (
	"fmt"
)

func main() {
	fmt.Println("Hello, playground")
}

Run this code in The Go Playground

A stable platform

Go 1 has a compatibility promise at the source level with the aim of building a stable platform for the growth of program and projects made with the language:

It is intended that programs written to the Go 1 specification will continue to compile and run correctly, unchanged, over the lifetime of that specification. At some indefinite point, a Go 2 specification may arise, but until that time, Go programs that work today should continue to work even as future “point” releases of Go 1 arise (Go 1.1, Go 1.2, etc.).

Go 1 and the Future of Go Programs

The go1compat promise by the Go Team is worth highlighting in a world where most languages swiftly roll major versions with breaking changes and ever-changing APIs almost yearly. I highly suggest you read their compatibility document to understand the expectations you can have and how they might apply to topics such as bugs, spec errors, security issues, the use of unkeyed struct literals, and the unsafe package.

A simple language

Consulting the spec, you can find out that Go has only 25 reserved keywords.

1
2
3
4
5

break        default      func         interface    select
case         defer        go           map          struct
chan         else         goto         package      switch
const        fallthrough  if           range        type
continue     for          import       return       var

You’ll also discover it is a concise language with regular syntax. In fact, one with the goals the Go team had when designing it was to increase the productivity of software engineering at scale at Google.

Influence
Statement and expression syntax C
Declaration syntax Pascal
Packages Modula-2, Oberon-2
Concurrency CSP, Occam, Newsqueak, Limbo, Alef
The semicolon rule BCPL
Methods Smalltalk
<-, := Newsqueak
iota APL
Lessons good and bad C++, C#, Java, JavaScript, LISP, Python, Scala, etc.

Source: Hello, Gophers!

Learning

Effective Go, Code Review Comments, and Specification are must-read documents if you want to be serious about Go. I always recommend anyone using the language to follow the recommendations of Effective Go and Code Review Comments for the sake of consistency – even when not 100% sold to them.

In the end of 2019, the Go Team launched go.dev website, a hub for Go users providing centralized and curated resources from across the Go ecosystem. Go to learn.go.dev to see a list of Learning Resources with a range of tactics and focus subjects.

I started learning Go six years ago and mainly used the official documentation, A Tour of Go, and Go by Example to learn the language. No matter if you’re an experienced developer or a newcomer to programming, Go is a fun language to learn, and the community is friendly and respectful.

Quick start guides

Books

Articles, tutorials, talks, opinions, etc.

Training and workshops

Community

Popular communications channels about the language include:

Conferences & Meetups

There are many Go meetups and conferences nowadays. Here are some of the most popular conferences. Due to the current epidemic, many were moved online for the time being or have announced they’ll follow a dual online/presential approach.

Kudos to initiatives such as GoBridge, making it possible for many underrepresented developers to attend the conferences – as the cost of tickets and travel can get as much as a few thousand dollars!

Look up Awesome Go’s conferences and meetups list or the Gophers Slack to see if a local meetup group exists close to your location.

News

  • Golang Weekly, a weekly newsletter about the Go programming language.

Podcasts

YouTube and Twitch channels

Besides the aforementioned conference channels, plenty of Gophers have Go channels.

Text editors and IDEs

Thanks to the Language Server Protocol, a protocol originally designed for Microsoft Visual Studio Code, which allows programming language support to be implemented and distributed independently of any given editor or IDE, Go’s support on development environments is widespread.

Visual Studio Code

Visual Studio Code is an open-source source code editor by Microsoft with a minimal GUI that doesn’t get in your own. I use it daily to work with Go code, and in my opinion, it’s the best option for most people. The first time you open a Go file with it, it should recognize the file and suggest you install the official Go extension. Read more about Go in Visual Studio Code to learn about shortcuts, debug Go code with Delve, and more.

Go extension

Acme

Acme is a text editor and graphical shell created by Rob Pike for the Plan 9 operating system (more on that below). Famously used by C’s designer Dennis Ritchie, it’s Vim’s antithesis: it heavily relies on mouse chording.

A Tour of the Acme Editor – Russ Cox (details)

Vim

Vim is a text-based user interface editor. It requires time and effort to learn and memorize its many commands. I use it primarily for punctual changes on configuration files, and when accessing remote machines.

For using it with Go, you’ll want to install the vim-go plugin by Fatih Arslan.

vim-go plugin

GoLand

GoLand is an Integrated Development Environment (IDE) by JetBrains (the authors of IntelliJ). It’s a full-featured IDE, and the obvious choice if you must do some heavy refactoring. I use it occasionally – whenever I find myself doing a repetitive refactoring task.

GoLand

Coding style and code quality

Go has a very consistent style. Thanks to its simplicity, it is fairly easy to write tools to do static analysis of Go code, especially since the x/tools/go/analysis package appeared.

I recommend taking a light approach to enforcing code style. Thanks to go fmt, the fight between Tabs versus Spaces is over. You don’t have an option, and have to use TABs. And this is a good thing, even if you prefer spaces: you make a small concession of style for the great benefit of consistency. You’ll also notice most Go static analyzers also don’t let users configure their preferences, and in my opinion this is a good thing. It’s hard to make everyone happy, so why not try to achieve a consistent style instead? For me this is an easy trade-off.

 

Testing

Go testing library is pretty straight-forward. Go has two types of tests: *testing.T (regular tests) and *testing.B (benchmark tests).

For people used to other programming languages, the lack of assertion functions might seem a bit odd at first. Still, it’s pretty good as it makes it crystal clear what you’re testing, and your testing code looks more like production code.

  • Heavy assertion libraries slows down testing due to increased build time.
  • BDD testing libraries make it especially hard to debug code using common tooling.
1
2
3
4
5
6
7

func TestAbs(t *testing.T) {
	got := Abs(-1)
	if got != 1 {
		// You almost always want to use t.Error* functions instead of t.Fatal*.
		t.Errorf("Abs(-1) = %d; want 1", got)
	}
}

To run the tests, I use an alias I called gotest:

1

$ go test -race -coverprofile=coverage.out && go tool cover -html coverage.out -o coverage.html
  • -race flag enables the Data Race Detector
  • -coverprofile flag enables code coverage for my code

Most of the time, I’ve been running this all from within Visual Studio Code, in any case. The -run flag enables you to run functions matching a specific regex – really useful when you need to run a specific test or group of related tests. Use go test helpflag to learn more about the available flags.

1

$ go test -run=TestValidUser

httptest

The http package is one of the best packages in Go. You don’t need a web framework to write web services effectively. To write integration tests for your endpoints, the http/httptest package might be useful.

Fuzzy testing

Fuzz testing is a technique that involves providing invalid, unexpected, or random data as inputs to a computer program. There is a proposal to add fuzz test support to the standard library. Meanwhile, you can use gofuzz and go-fuzz.

Static analysis tools

Before pushing changes upstream, I like to run a script that executes some programs to check code quality. I also find it very productive to have a Continuous Integration system such as GitHub Actions set up to run most of them once code is submitted upstream.

go vet

Vet examines Go source code and reports suspicious constructs, such as Printf calls whose arguments do not align with the format string. Vet uses heuristics that do not guarantee all reports are genuine problems, but it can find errors not caught by the compilers.

Source: documentation

The go vet is part of the Go toolchain, and have a list of analyzers you can see with go tool vet help. By default, all analyzers are run.

1
2

$ # ./... to walk the working directory recursively
$ go vet ./...

As with typical Unix-like commands, no output or exit error = 0 means you’re good.

Staticcheck

Staticcheck is a state of the art linter for Go by Dominik Honnef. You don’t really need any other linter besides go vet and staticcheck.

1

$ staticcheck ./...

GitHub Security code scanning

Suppose you’re using GitHub to host an open source project’s repository or willing to pay extra for their Enterprise plan. In that case, GitHub Security has you covered with its CodeQL security scanner. Sonar is another option.

gosec

gosec is a security checker for Go. It scans your code and pinpoints unsafe security usage. By default, I find it quite noisy as it reports unhandled errors ignoring what I see as many false-positive cases, so I exclude this specific rule when I decide to use this tool on one of my projects.

1
2

$ # Ignoring gosec unhandled errors warning due to many false-positives.
$ gosec -quiet -exclude G104 ./...

unparam

unparam helps you find unused function parameters and results in your code. I must warn that I use this tool manually, not in a script or CI system, as I find it distracting there, especially at the beginning of a project.

mispell

mispell is a great tool for checking common English mistakes on your source code. I also often use Grammarly to review my writing on documentation.

1

$ misspell cmd/**/*.{go,sh} internal/**/*.{go} README.md

I also use other static analyzer tools from time-to-time, manually, and that I’d advise against adding to a Continuous Integration pipeline.

gocyclo

gocyclo calculates cyclomatic complexity of functions in your Go source code.

I’ve used this in the past to find complicated code and to decide where to start refactoring code. I also prefer to use this manually, but I see some value in adding it as a step of a CI pipeline – as long as you err by being generous and configure the tool to scream only in case of undeniable high values.

1

$ gocyclo -over 15 -ignore "generated.go|pb.go|_test|vendor/" .

Standard library

Go’s standard library is outstanding both in terms of covering a wide range of needs and in terms of quality. You’ll find that with Go, you can rely on the standard library for many things where you’d typically require adding external dependencies to your project.

I always suggest to people starting with Go to take some time to read both the docs, and the source code for net/http, as it is the best example of a well-organized and successful package I can think of.

x packages

Packages inside the /x/ have looser compatibility requirements than the rest of the standard library. A package that shows to be helpful throughout the ecosystem might be promoted from /x/ to the status of a regular package of the standard library, like context.

Code in sub-repositories of the main go tree, such as golang.org/x/net, may be developed under looser compatibility requirements. However, the sub-repositories will be tagged as appropriate to identify versions that are compatible with the Go 1 point releases.

Go 1 and the Future of Go Programs: Sub-repositories:

Third-party packages

You can go really far with the standard library, and it’s not uncommon to write production-quality programs relying solely on it. Before you jump and start importing a plethora of external packages on your code, do yourself a favor and remember the Go proverb:

A little copying is better than a little dependency.

It’s important to be aware of the cost of importing dependencies to avoid pitfalls like the leftpad fiasco, security vulnerabilities, quality problems, or worse.

I’ve also noticed that in Go I need fewer dependencies, and my dependencies themselves have fewer dependencies. Go doesn’t have a culture of exporting as-much-logic-as-possible to external dependencies. Code duplication is more acceptable in the Go community than elsewhere. This can be frustrating. Sometimes, you just want a good library that performs some type of sanitation or parsing. Many times, you’ll need to write that functionality yourself, or copy/paste it from a StackOverflow answer. Generally I think this is a positive. Fewer dependencies means fewer things that break while you let a project sit idle for a couple months.

The Value in Go’s Simplicity

You can search for Go packages in pkg.go.dev, and discover commonly used ones in the go.dev pages. If you want an extensive and comprehensive list of Go packages with a short one-line description, Awesome Go is what you want!

Did you know Go programs can only build so fast because it doesn’t allow circular imports? This way, the linker can be simpler, and we mitigate one of the common problems of dependency hell.

cobra

cobra is a framework for Modern CLI Apps by Steve Francia.

pgx (PostgreSQL driver)

pgx is the best PostgreSQL driver and toolkit for Go. While normally you want to use database/sql interfaces for connecting to a database, you probably want to use pgx without it due to a number of advantages thanks to PostgreSQL’s binary protocol.

gRPC-Go

gRPC-Go is the Go implementation of the high performance, open source universal RPC framework gRPC, which has a strict specification compared to the more popular HTTP APIs with JSON.

paseto (alternative to JWT)

paseto is a Platform-Agnostic Security Tokens implementation of PASETO tokens. This standard is everything you love about JOSE (JWT, JWE, JWS) without any of the many design deficits that plague the JOSE standards.

GoReleaser

GoReleaser is a powerful packaging tool. With GoReleaser, you can:

  • Cross-compile your Go project
  • Release to GitHub, GitLab, and Gitea
  • Create Docker images and manifests
  • Create Linux packages and Homebrew taps
  • … and much more!

equinox.io

equinox.io is a tool and service that you can use to package and distribute your Go programs (library). It has a slightly more limited packaging scope than GoReleaser, but provides some killer features such as different distribution channels (i.e., you can create channels stable, unstable, etc.) and code signing.

To learn more about distributing Go binaries for multiple platforms safely, read my post Counter-Strike code leaked: should you worry?

goexpect

goexpect is an implementation of Expect in Go (I created pseudoterm, something quite similar).

subcommands

subcommands is a Go package that implements a simple way for a single command to have many subcommands. I created clino about a year ago, unaware of this.

avo

avo makes high-performance Go assembly easier to write, review and maintain.

httpretty

httpretty is a package I created to help me debug HTTP requests on CLI applications.

Lip Gloss

lipgloss is Charm’s package for writing Text-based user interfaces (TUIs) (just like ncurses, but better).

Bubble Tea

Bubble Tea is a framework to build terminal apps.

Bubbles

Bubbles is a package containing TUI components for Bubble Tea.

SQL Databases

Hardware integration

The Hybrid Group has some quite exciting projects. If you’re interested in interfacing with hardware, running Go in embedded computers, or computer vision, you want to check what they’re up to!

Gobot GoCV TinyGo

Small is Going Big: Go on Microcontrollers by Ron Evans at GopherCon 2019.

  • Gobot is a framework for robotics, physical computing, and the Internet of Things (IoT) with support for dozens of different hardware/software platforms.
  • GoCV is a package for computer vision using OpenCV 4 and beyond.
  • TinyGo is a Go compiler for small places. Microcontrollers, WebAssembly, and command-line tools. Based on LLVM.

Go modules and vendoring

A module is a collection of packages that are released, versioned, and distributed together. Modules may be downloaded directly from version control repositories or from module proxy servers.

Go Modules Reference

Go modules appeared in 2019 after the language and its ecosystem were mature and the community already experimented with a diverse number of approaches to solve the problem of managing dependencies.

See also:

Auditing

Once you’ve go mod set up, you might want to audit your dependencies. There are two ways:

  • You could set up your own Go modules proxy and whitelist what dependencies to allow.
  • You can vendor your dependencies.

I don’t have any suggestions regarding auditing with a go proxy, but I audit dependencies when vendoring (or vendorizing) code I control – more on that in a minute! Auditing is an important step to verify the quality of your dependencies and to check if no malicious code was injected somewhere.

Keeping the number of dependencies low also means you’ve less to audit!

Vendoring

Vendoring (or vendorizing) is copying dependencies to your own project, versioning them along with your codebase. Not everyone is comfortable with the idea of copying external code to their projects, but I consider this the safest approach to guarantee long-term access to your dependencies. You avoid the risk of losing access to your dependencies for a variety of reasons, including the evil DMCA takedown notices.

  • Go dependencies are mostly small textual .go files, so they don’t take up much space.
  • It mitigates the risk of losing access to the exact version of the dependency you use in case a dependency is taken down.
  • Faster binary search for bad commits when using git bisect to debug code.

Go modules proxy

Alternatively, suppose you want control over what dependencies your team uses. In that case, you might consider setting up your own Go proxy. This might work well for enterprises with a strict requirement for tight control. Maybe solutions like JFrog Artifactory might help.

Tools for software development

godoc

godoc is a program that extracts and generates documentation for Go programs. You can use it to browse the documentation for your code and the standard library off-line easily, using a web browser.

It is also what originally powered godoc.org, before it was sunset by pkg.go.dev, which is powered by the more complex pkgsite.

1
2
3

$ go install golang.org/x/tools/cmd/godoc@latest
$ godoc
$ open http://localhost:6060

Use godoc flags -play to enable playground and -index to enable the search index.

swag

swag is a tool to automatically generate RESTful API documentation with Swagger 2.0 (it still doesn’t support the newest OpenAPI Specification, though).

Goda

Goda is a Go dependency analysis toolkit. It contains tools to figure out what your program is using. For example, you can print out dependency trees or draw a dependency graph, and see the impact of cutting a package from your program.

depaware

depaware is a Go dependencies analysis tool by Tailscale.

gops

gops is a Google tool to list and diagnose Go processes currently running on your system.

goversion

goversion scans a directory tree and, for every executable it finds, prints the Go version used to build that executable. It also has a flag to show module info.

sqlc

sqlc generates fully-type safe idiomatic Go code for use with database/sql from SQL. Take a look at their Playground to see some examples.

hey

hey is a tiny program by Jaana Dogan (@rakyll) that sends some load to a web application (similar to Apache’s ab).

vegeta

vegeta is another HTTP load testing tool.

Go Guru

guru is a tool for answering questions such as the ones below about your source code. Read the documentation for more information.

  • Where is this identifier declared?
  • Where are all the references to this declaration?
  • What are the fields and methods of this this expression?
  • What is the API of this package?
  • Which concrete types implement this interface?
  • What are the possible callees of this dynamic call?
  • What are the possible callers of this function?
  • Where might a value sent on this channel be received?

Pythia

Pythia is a browser-based user interface for guru.

Pythia screenshot

Wails

Wails is a framework for building applications using Go and Web Technologies. I haven’t had the opportunity to try it out yet, but I’m excited about the possibilities!

mkcert

mkcert is a simple tool for making locally-trusted development certificates.

Productivity tools

present

present is a tool that you can use to create articles or slides. It’s useful for writing simple and direct presentations when you don’t care about setting a specific visual style, as it uses a simple black & white theme that you cannot configure.

1

$ go install golang.org/x/tools/cmd/present@latest

Hugo

Hugo is the world’s fastest framework for building websites by Bjørn Erik Pedersen and Steve Francia, and it’s able to compile whole websites in a fraction of a second! It’s a natural choice if you want to write your own website or the documentation for a project – using Go templates!

I’ve been using it for almost two years for generating this website. You can create your own theme or use one of the many available online. I decided to create my own, so I used a minimal one as the base in the very beginning to understand how Hugo works.

Hugo

Hugo in Action is an upcoming book about Hugo (Summer 2021). You can learn more about it on the publisher’s page.

Caddy

Caddy is an extensible HTTP server platform written in Go.

I’ve been using it for local software development instead of nginx to serve static files and to expose multiple HTTP services on the same address using a reverse proxy.

gophernotes

gophernotes is a Go kernel for Jupyter notebooks and nteract.

The Jupyter Notebook is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text. Uses include: data cleaning and transformation, numerical simulation, statistical modeling, data visualization, machine learning, and much more.

xbar

xbar is a macOS program written in Go that lets you put the output of any script/program in your macOS menu bar. It’s like iStat Menus, but for your menus! It’s an open-source project written primarily in Go by Mat Ryer!

ghz

ghz is a gRPC benchmarking and load testing tool.

Evans

Evans is gRPC client with REPL and CLI modes.

Reporting bugs and proposing feature requests

  • The project uses publicly accessible GitHub issues to track and discuss those.
  • Before you submit a proposal, search to see if what you’ve in mind was already discussed in the past. Use your findings wisely.
  • If you want to report a security bug, please follow the Go Security Policy.

Code review process

Contributing back

  • You probably want to start by drafting a proposal.
  • You should read the Contribution Guide.
  • Sign and submit a simple and straight-forward Contributor License Agreement (CLA).
  • Please make sure you check at least a few proposals and reasons why they were either approved or rejected before submitting your own proposal.
  • In my blog post signal.NotifyContext: handling cancelation with Unix signals using context, I share a bit of my experience adding a feature to the signal package.

Gopher

The Go gopher

The gopher is an iconic mascot of the language. It was created by Renée French, who also created the cute Glenda, the Plan 9 Bunny. You can create your own Gopher at Gopherize.me. But what does Plan 9 has to do with Go? Well, a lot!

Plan 9 influence

Plan 9 is a distributed operating system created at Bell Labs by members of the same group that originally developed Unix and the C programming language. Part of the Plan 9 team included Rob Pike, Ken Thompson, and Russ Cox – an avid contributor to the Plan 9 from User Space fork – and they brought many ideas and concepts of the operating system to Go.

Glenda, the Plan 9 Bunny

Plan 9 is an interesting topic on its own, but my experience with it is fairly limited. Besides the aforementioned Acme, two interesting things about Plan 9:

  • Files are key objects in Plan 9 with their 9P protocol (even more than in UNIX), and even a mouse is a file.
  • UTF-8, a variable-width character encoding that is de facto the core of Unicode, was also created by Ken Thompson and Rob Pike, and the original implementation was for Plan 9. Not surprisingly, Go has great UTF-8 support!
1
2
3
4
5
6
7
8

// Quadratic takes a general quadratic equation of the form
// ax² + bx + c = 0.
func Quadratic(a, b, c float64) (x1, x2, Δ float64) {
	Δ = math.Pow(b, 2) - (4 * a * c)
	x1 = (-b + math.Sqrt(Δ)) / (2 * a)
	x2 = (-b - math.Sqrt(Δ)) / (2 * a)
	return x1, x2, Δ // Just don't go this wild!!!
}

Thanks! If you liked what I wrote, maybe you want to check out my other posts, such as my experience creating a homelab to play with different operating systems or learn about my Go mistakes.

If you click and buy any of these from Amazon after visiting the links above, I might get a commission from their Affiliate program.