The ecosystem of the Go programming language
Posted in Monday, 22 March 2021.
Go is one of the most prominent general-purpose programming languages nowadays. Google, Apple, Microsoft, Amazon, and Adobe, to name a few, have been using the language extensively. It’s the language of choice behind multiple cloud computing projects such as Kubernetes, and it’s steadily expanding towards numerous areas of software development. In this article, you’ll find resources to learn about Go and its ecosystem.
If you want to see how people are using Go, check out the Go Developer Survey 2020 Results.
|Paradigm||Multi-paradigm: mostly imperative, and concurrent.|
|Designed by||Robert Griesemer, Rob Pike, and Ken Thompson. Russ Cox and Ian Lance Taylor quickly joined them.|
|Dates||Design began in late 2007; Publicly Announced November 10, 2009 by the Go Team.|
|Influenced by||C, Pascal, Modula-2, Oberon-2, CSP, Occam, Newsqueak, Limbo, Alef, BCPL, Smalltalk, APL, etc.|
|Typing||Inferred, static, strong, structural.|
|Known for being||Modern, readable, concise, garbage-collected, fast to compile, statically linked.|
|Release cycle||Major release every 6 months|
|Essentials||Effective Go, Code Review Comments, Specification, and FAQ|
Hello World in Go
I bought an Intel NUC 10th generation and installed the VMware ESXi hypervisor, FreeBSD, FreeNAS, Clear Linux, and more to serve as my homeserver, run containers, backup data, etc. :)— Henrique Vicente (@henriquev) April 20, 2020
Sharing my experience here...https://t.co/gscoxPDuTa
A stable platform
Go 1 has a compatibility promise at the source level with the aim of building a stable platform for the growth of program and projects made with the language:
It is intended that programs written to the Go 1 specification will continue to compile and run correctly, unchanged, over the lifetime of that specification. At some indefinite point, a Go 2 specification may arise, but until that time, Go programs that work today should continue to work even as future “point” releases of Go 1 arise (Go 1.1, Go 1.2, etc.).
The go1compat promise by the Go Team is worth highlighting in a world where most languages swiftly roll major versions with breaking changes and ever-changing APIs almost yearly. I highly suggest you read their compatibility document to understand the expectations you can have and how they might apply to topics such as bugs, spec errors, security issues, the use of unkeyed struct literals, and the unsafe package.
A simple language
You’ll also discover it is a concise language with regular syntax. In fact, one with the goals the Go team had when designing it was to increase the productivity of software engineering at scale at Google.
|Statement and expression syntax||C|
|Concurrency||CSP, Occam, Newsqueak, Limbo, Alef|
|The semicolon rule||BCPL|
Source: Hello, Gophers!
Effective Go, Code Review Comments, and Specification are must-read documents if you want to be serious about Go. I always recommend anyone using the language to follow the recommendations of Effective Go and Code Review Comments for the sake of consistency – even when not 100% sold to them.
In the end of 2019, the Go Team launched go.dev website, a hub for Go users providing centralized and curated resources from across the Go ecosystem. Go to learn.go.dev to see a list of Learning Resources with a range of tactics and focus subjects.
I started learning Go six years ago and mainly used the official documentation, A Tour of Go, and Go by Example to learn the language. No matter if you’re an experienced developer or a newcomer to programming, Go is a fun language to learn, and the community is friendly and respectful.
Quick start guides
- The Go Programming Language (website) was written by Alan A. A. Donovan and Brian W. Kernighan (co-author of The C Programming Language with Dennis Ritchie and The Unix Programming Environment with Rob Pike)
- For a concise and comprehensive guide, try Go in Action (online version and on Amazon) by Bill Kennedy and Brian Ketelsen (known for co-organizing GopherCon)
- Learning Go: An Idiomatic Approach to Real-World Go Programming by Jon Bodner (published by O’Reilly)
- Practical Go Lessons by Maximilien Andile
Articles, tutorials, talks, opinions, etc.
- Go wiki
- Go talks
- The Go Memory Model
- Data Race Detector
- research!rsc by Russ Cox
- Articles and talks by Peter Bourgon
- Practical Go and High Performance Go by Dave Cheney
- Writing Accessible Go 2018 GopherCon talk (slides) by Julia Ferraioli
- Rust vs. Go: Why They’re Better Together
- GopherCon 2018: Bryan C. Mills - Rethinking Classical Concurrency Patterns (slides and transcript)
- TutorialEdge Go Development and course
- Darker Corners of Go by Rytis Biel
Training and workshops
- Many conferences, such as GopherCon offers hands-on workshops before or after the main event.
- Ultimate Go is a workshop by Bill Kennedy’s Ardan Labs consulting company and praised by many!
Popular communications channels about the language include:
- Go Forum
- Gophers Slack
- golang-nuts general discussion list
- #golang hashtag @ Twitter
- Reddit’s Go community
- #go-nuts IRC channel @ Libera Chat) (you’ll need an IRC client or webchat like IRCCloud)
Conferences & Meetups
There are many Go meetups and conferences nowadays. Here are some of the most popular conferences. Due to the current epidemic, many were moved online for the time being or have announced they’ll follow a dual online/presential approach.
- GopherCon (videos)
- GopherCon EU (videos)
- GopherCon UK (videos)
- GopherCon Brasil (videos)
- dotGo (videos)
Kudos to initiatives such as GoBridge, making it possible for many underrepresented developers to attend the conferences – as the cost of tickets and travel can get as much as a few thousand dollars!
- Golang Weekly, a weekly newsletter about the Go programming language.
YouTube and Twitch channels
Besides the aforementioned conference channels, plenty of Gophers have Go channels.
- justforfunc: Programming in Go by Francesc Campoy
- Filippo Valsorda’s Twitch
- Ardan Labs channel
- Rob Evans a.k.a. @deadprogram Twitch
Text editors and IDEs
Thanks to the Language Server Protocol, a protocol originally designed for Microsoft Visual Studio Code, which allows programming language support to be implemented and distributed independently of any given editor or IDE, Go’s support on development environments is widespread.
Visual Studio Code
Visual Studio Code is an open-source source code editor by Microsoft with a minimal GUI that doesn’t get in your own. I use it daily to work with Go code, and in my opinion, it’s the best option for most people. The first time you open a Go file with it, it should recognize the file and suggest you install the official Go extension. Read more about Go in Visual Studio Code to learn about shortcuts, debug Go code with Delve, and more.
Acme is a text editor and graphical shell created by Rob Pike for the Plan 9 operating system (more on that below). Famously used by C’s designer Dennis Ritchie, it’s Vim’s antithesis: it heavily relies on mouse chording.
Vim is a text-based user interface editor. It requires time and effort to learn and memorize its many commands. I use it primarily for punctual changes on configuration files, and when accessing remote machines.
GoLand is an Integrated Development Environment (IDE) by JetBrains (the authors of IntelliJ). It’s a full-featured IDE, and the obvious choice if you must do some heavy refactoring. I use it occasionally – whenever I find myself doing a repetitive refactoring task.
Coding style and code quality
Go has a very consistent style. Thanks to its simplicity, it is fairly easy to write tools to do static analysis of Go code, especially since the x/tools/go/analysis package appeared.
I recommend taking a light approach to enforcing code style.
go fmt, the fight between Tabs versus Spaces is over.
You don’t have an option, and have to use TABs.
And this is a good thing, even if you prefer spaces: you make a small concession of style for the great benefit of consistency.
You’ll also notice most Go static analyzers also don’t let users configure their preferences, and in my opinion this is a good thing.
It’s hard to make everyone happy, so why not try to achieve a consistent style instead?
For me this is an easy trade-off.
Go testing library is pretty straight-forward.
Go has two types of tests:
*testing.T (regular tests) and
*testing.B (benchmark tests).
For people used to other programming languages, the lack of assertion functions might seem a bit odd at first. Still, it’s pretty good as it makes it crystal clear what you’re testing, and your testing code looks more like production code.
- Heavy assertion libraries slows down testing due to increased build time.
- BDD testing libraries make it especially hard to debug code using common tooling.
To run the tests, I use an alias I called
-raceflag enables the Data Race Detector
-coverprofileflag enables code coverage for my code
Most of the time, I’ve been running this all from within Visual Studio Code, in any case.
-run flag enables you to run functions matching a specific regex – really useful when you need to run a specific test or group of related tests.
go test helpflag to learn more about the available flags.
Visual Studio Code shortcuts I use with Go:— Henrique Vicente (@henriquev) March 12, 2021
F13 Show Editor Context Menu
F16 Test Function At Cursor
F17 Debug Test At Cursor
F18 Generate Unit Tests for Function
F19 Toggle Test Coverage
I wanted bright Magic KB shortcut key caps.https://t.co/TnGNoWcmWthttps://t.co/KjQ0ApbEdT
The http package is one of the best packages in Go. You don’t need a web framework to write web services effectively. To write integration tests for your endpoints, the http/httptest package might be useful.
Fuzz testing is a technique that involves providing invalid, unexpected, or random data as inputs to a computer program. There is a proposal to add fuzz test support to the standard library. Meanwhile, you can use gofuzz and go-fuzz.
Static analysis tools
Before pushing changes upstream, I like to run a script that executes some programs to check code quality. I also find it very productive to have a Continuous Integration system such as GitHub Actions set up to run most of them once code is submitted upstream.
Vet examines Go source code and reports suspicious constructs, such as Printf calls whose arguments do not align with the format string. Vet uses heuristics that do not guarantee all reports are genuine problems, but it can find errors not caught by the compilers.
go vet is part of the Go toolchain, and have a list of analyzers you can see with
go tool vet help.
By default, all analyzers are run.
As with typical Unix-like commands, no output or exit error = 0 means you’re good.
GitHub Security code scanning
Suppose you’re using GitHub to host an open source project’s repository or willing to pay extra for their Enterprise plan. In that case, GitHub Security has you covered with its CodeQL security scanner. Sonar is another option.
gosec is a security checker for Go. It scans your code and pinpoints unsafe security usage. By default, I find it quite noisy as it reports unhandled errors ignoring what I see as many false-positive cases, so I exclude this specific rule when I decide to use this tool on one of my projects.
unparam helps you find unused function parameters and results in your code. I must warn that I use this tool manually, not in a script or CI system, as I find it distracting there, especially at the beginning of a project.
I also use other static analyzer tools from time-to-time, manually, and that I’d advise against adding to a Continuous Integration pipeline.
I’ve used this in the past to find complicated code and to decide where to start refactoring code. I also prefer to use this manually, but I see some value in adding it as a step of a CI pipeline – as long as you err by being generous and configure the tool to scream only in case of undeniable high values.
Go’s standard library is outstanding both in terms of covering a wide range of needs and in terms of quality. You’ll find that with Go, you can rely on the standard library for many things where you’d typically require adding external dependencies to your project.
I always suggest to people starting with Go to take some time to read both the docs, and the source code for net/http, as it is the best example of a well-organized and successful package I can think of.
Packages inside the /x/ have looser compatibility requirements than the rest of the standard library. A package that shows to be helpful throughout the ecosystem might be promoted from /x/ to the status of a regular package of the standard library, like context.
Code in sub-repositories of the main go tree, such as golang.org/x/net, may be developed under looser compatibility requirements. However, the sub-repositories will be tagged as appropriate to identify versions that are compatible with the Go 1 point releases.
You can go really far with the standard library, and it’s not uncommon to write production-quality programs relying solely on it. Before you jump and start importing a plethora of external packages on your code, do yourself a favor and remember the Go proverb:
I’ve also noticed that in Go I need fewer dependencies, and my dependencies themselves have fewer dependencies. Go doesn’t have a culture of exporting as-much-logic-as-possible to external dependencies. Code duplication is more acceptable in the Go community than elsewhere. This can be frustrating. Sometimes, you just want a good library that performs some type of sanitation or parsing. Many times, you’ll need to write that functionality yourself, or copy/paste it from a StackOverflow answer. Generally I think this is a positive. Fewer dependencies means fewer things that break while you let a project sit idle for a couple months.
You can search for Go packages in pkg.go.dev, and discover commonly used ones in the go.dev pages. If you want an extensive and comprehensive list of Go packages with a short one-line description, Awesome Go is what you want!
Did you know Go programs can only build so fast because it doesn’t allow circular imports? This way, the linker can be simpler, and we mitigate one of the common problems of dependency hell.
pgx (PostgreSQL driver)
pgx is the best PostgreSQL driver and toolkit for Go. While normally you want to use database/sql interfaces for connecting to a database, you probably want to use pgx without it due to a number of advantages thanks to PostgreSQL’s binary protocol.
paseto (alternative to JWT)
paseto is a Platform-Agnostic Security Tokens implementation of PASETO tokens. This standard is everything you love about JOSE (JWT, JWE, JWS) without any of the many design deficits that plague the JOSE standards.
GoReleaser is a powerful packaging tool. With GoReleaser, you can:
- Cross-compile your Go project
- Release to GitHub, GitLab, and Gitea
- Create Docker images and manifests
- Create Linux packages and Homebrew taps
- … and much more!
equinox.io is a tool and service that you can use to package and distribute your Go programs (library). It has a slightly more limited packaging scope than GoReleaser, but provides some killer features such as different distribution channels (i.e., you can create channels stable, unstable, etc.) and code signing.
To learn more about distributing Go binaries for multiple platforms safely, read my post Counter-Strike code leaked: should you worry?
avo makes high-performance Go assembly easier to write, review and maintain.
httpretty is a package I created to help me debug HTTP requests on CLI applications.
- Writing SQL yourself is always the best option to write sane and fast queries. Use sqlc if you want to reduce the amount of code you need to write yourself, instead of using an ORM and making your application slower and your program more error-prone and harder to debug.
The Hybrid Group has some quite exciting projects. If you’re interested in interfacing with hardware, running Go in embedded computers, or computer vision, you want to check what they’re up to!
- Gobot is a framework for robotics, physical computing, and the Internet of Things (IoT) with support for dozens of different hardware/software platforms.
- GoCV is a package for computer vision using OpenCV 4 and beyond.
- TinyGo is a Go compiler for small places. Microcontrollers, WebAssembly, and command-line tools. Based on LLVM.
Go modules and vendoring
A module is a collection of packages that are released, versioned, and distributed together. Modules may be downloaded directly from version control repositories or from module proxy servers.
Go modules appeared in 2019 after the language and its ecosystem were mature and the community already experimented with a diverse number of approaches to solve the problem of managing dependencies.
Once you’ve go mod set up, you might want to audit your dependencies. There are two ways:
- You could set up your own Go modules proxy and whitelist what dependencies to allow.
- You can vendor your dependencies.
I don’t have any suggestions regarding auditing with a go proxy, but I audit dependencies when vendoring (or vendorizing) code I control – more on that in a minute! Auditing is an important step to verify the quality of your dependencies and to check if no malicious code was injected somewhere.
Keeping the number of dependencies low also means you’ve less to audit!
Vendoring (or vendorizing) is copying dependencies to your own project, versioning them along with your codebase. Not everyone is comfortable with the idea of copying external code to their projects, but I consider this the safest approach to guarantee long-term access to your dependencies. You avoid the risk of losing access to your dependencies for a variety of reasons, including the evil DMCA takedown notices.
- Go dependencies are mostly small textual .go files, so they don’t take up much space.
- It mitigates the risk of losing access to the exact version of the dependency you use in case a dependency is taken down.
- Faster binary search for bad commits when using
git bisectto debug code.
Go modules proxy
Alternatively, suppose you want control over what dependencies your team uses. In that case, you might consider setting up your own Go proxy. This might work well for enterprises with a strict requirement for tight control. Maybe solutions like JFrog Artifactory might help.
Tools for software development
godoc is a program that extracts and generates documentation for Go programs. You can use it to browse the documentation for your code and the standard library off-line easily, using a web browser.
-play to enable playground and
-index to enable the search index.
Goda is a Go dependency analysis toolkit. It contains tools to figure out what your program is using. For example, you can print out dependency trees or draw a dependency graph, and see the impact of cutting a package from your program.
gops is a Google tool to list and diagnose Go processes currently running on your system.
goversion scans a directory tree and, for every executable it finds, prints the Go version used to build that executable. It also has a flag to show module info.
vegeta is another HTTP load testing tool.
- Where is this identifier declared?
- Where are all the references to this declaration?
- What are the fields and methods of this this expression?
- What is the API of this package?
- Which concrete types implement this interface?
- What are the possible callees of this dynamic call?
- What are the possible callers of this function?
- Where might a value sent on this channel be received?
Wails is a framework for building applications using Go and Web Technologies. I haven’t had the opportunity to try it out yet, but I’m excited about the possibilities!
mkcert is a simple tool for making locally-trusted development certificates.
present is a tool that you can use to create articles or slides. It’s useful for writing simple and direct presentations when you don’t care about setting a specific visual style, as it uses a simple black & white theme that you cannot configure.
Hugo is the world’s fastest framework for building websites by Bjørn Erik Pedersen and Steve Francia, and it’s able to compile whole websites in a fraction of a second! It’s a natural choice if you want to write your own website or the documentation for a project – using Go templates!
I’ve been using it for almost two years for generating this website. You can create your own theme or use one of the many available online. I decided to create my own, so I used a minimal one as the base in the very beginning to understand how Hugo works.
Caddy is an extensible HTTP server platform written in Go.
I’ve been using it for local software development instead of nginx to serve static files and to expose multiple HTTP services on the same address using a reverse proxy.
The Jupyter Notebook is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text. Uses include: data cleaning and transformation, numerical simulation, statistical modeling, data visualization, machine learning, and much more.
xbar is a macOS program written in Go that lets you put the output of any script/program in your macOS menu bar. It’s like iStat Menus, but for your menus! It’s an open-source project written primarily in Go by Mat Ryer!
Reporting bugs and proposing feature requests
- The project uses publicly accessible GitHub issues to track and discuss those.
- Before you submit a proposal, search to see if what you’ve in mind was already discussed in the past. Use your findings wisely.
- If you want to report a security bug, please follow the Go Security Policy.
Code review process
- See go-review. The Go project and several other Google products like Chromium (see chromium-review) use the Gerrit Code Review to manage collaboration: it’s like GitHub’s Pull-Request, but on steroids!
- You probably want to start by drafting a proposal.
- You should read the Contribution Guide.
- Sign and submit a simple and straight-forward Contributor License Agreement (CLA).
- Please make sure you check at least a few proposals and reasons why they were either approved or rejected before submitting your own proposal.
- In my blog post signal.NotifyContext: handling cancelation with Unix signals using context, I share a bit of my experience adding a feature to the signal package.
The gopher is an iconic mascot of the language. It was created by Renée French, who also created the cute Glenda, the Plan 9 Bunny. You can create your own Gopher at Gopherize.me. But what does Plan 9 has to do with Go? Well, a lot!
Plan 9 influence
Plan 9 is a distributed operating system created at Bell Labs by members of the same group that originally developed Unix and the C programming language. Part of the Plan 9 team included Rob Pike, Ken Thompson, and Russ Cox – an avid contributor to the Plan 9 from User Space fork – and they brought many ideas and concepts of the operating system to Go.
Plan 9 is an interesting topic on its own, but my experience with it is fairly limited. Besides the aforementioned Acme, two interesting things about Plan 9:
- Files are key objects in Plan 9 with their 9P protocol (even more than in UNIX), and even a mouse is a file.
- UTF-8, a variable-width character encoding that is de facto the core of Unicode, was also created by Ken Thompson and Rob Pike, and the original implementation was for Plan 9. Not surprisingly, Go has great UTF-8 support!
If you click and buy any of these from Amazon after visiting the links above, I might get a commission from their Affiliate program.Tweet