Go Language Tools & Make

Go Language Tools & Make

STATUS: July 2019 - Draft / Writing Now based on work with Commento implementation

STATUS: October 2022 - Revisiting as part of Commento Go 1.18 Update preparation

Go (or GOLAN or Go Language) is programming language, a set of packages and tools. Both the language and the tools have some characteristics that are signficantly different from our old curly friends { C / C++ / Java }.

This this provides a list of some of the things that new Go'ers need to be aware of as it will help you to get your head around Go's idioms and features and be productive quickly.

The Language

Object OrientiationGo does not support classic Object Oriented class implementation inheritence (is-a).
It does however support a number of Object Oriented features including:
  • Types (rather then Classes),
  • Interfaces and Method Sets,
  • Functions (which can support Recieving instance as part of definition to provide Method / Message invocation style and
  • Composition by Containment which with Anonymous variables provides automatic delegation.
  • Hiding and Encapsulation via its lower/Upper case Private / Public visibility semantics
The impact if this is that Go does not generate automatic "self", "this", "super" handles for access to instance attributes/methods and traverse up the class.
Buts its delegation mechanism allows creation of polymorphic behavior.
Its features are sufficient to allow Go idiomatic OO code to be written
Scope and VisibilityThe rules of visibility have changed over different language release (see secton below on Visibility - Repositories, Modules, Packages)
Return TypesGo functions can return multiple values and types
Complex NumbersGo has native support for Complex numbers (square root of -1)

Visibility and Importing- Repositories, Modules, Package, Files, Build Trees etc

Let start this section with a quote - "‘When I use a word,’ Humpty Dumpty said in rather a scornful tone, ‘it means just what I choose it to mean — neither more nor less.’" from Alice in Wonderland.

The reason is that in its evolution Go has changed it concepts around code packaging and how visibility and inclusion (import) are managed and like Humpty Dumpty is sometimes "to clever by half" so understanding its behaviour can be a point of frustraton.

The way Go behaves and the concepts behind this vary considerablely across:

  • Go 1.11 / 1.12 - Introduced "modules" which provided a new dependency management systems that made version explicit. This meant that when creating a module you explicitly provide version information (Go uses a "semantic version" convention based on: Major, Minor and Patch release - i.e. v1.1.1) and when you import a module you explicity provide details of which version is acceptable (ie must be greater v1.2.1). To control if historical or new module behaviour as used the "GO111MODULE" environment variable was introduced (more on this later)
  • Go 1.13 - Changed the the default behavior and management of "modules" based on enviromment variables: "GOPATH" and "GO111MODULE".

So Go's concepts and behaviour can be looked at pre-module and post-module.

NOTE: I am assuing that Go version is > 1.11 and so using the GO111MODULE environment variable, if you are using verson lower than this then GO111MODULE will have no impact and you will always get the "Pre-Module" behaviour.


Pre-module Go worked best if you structure your file systems source tree using Go convention:

project/  <=== workspace - gopath env should reference this bin goprog pkg <="==" go install puts library files here linux_amd64 first level is os_arch, so dependent on your org.url next reflects: from the src tree [owner ] reflecting: lib.a. reflecting lib package [.git] <<="==" optional: repository location #1 name space (import path) identifier (like "github.com") [.git]. #2 url above owner project respository prog for main goprog.go veryusefule.go use: import "lib veryuseful" helper.go. helper"< code>

The key Go structuring and visibility concepts where:

  • Workspace - the OS file tree where Go source code and generated programs are. The source tree root should be defined via the GOPATH environment variable
  • package - the collection of Go sources files within a single directory
  • repository - the version control directory/s that could be part of content of source tree. Note that the .git respository placement is independent of Go and Go toes not interact with this.
  • import path - this is the qualificaton path that it used when importing the package. This is not explicity defined "pre-module", but automtically determined based on where the package its in the directory structure relative to the "src/" directory that the package sits in.

To build your go project you should first go into the library tree and do:

  • Pre-module library build / install - assuming Go 1.11 or greater: "GOPATH=<WORKSPACE> GO111MODULE=no go [build | install]"

Based on above tree example the library will have import path: "org.url[/owner]/lib", so ensure you have correct import path. Now build/install the local go program from "prog" directory (as per workshop above): "GOPATH=<WORKSPACE> GO111MODULE=no go [build | install]"

The buid /install process will place code in different locations relative to GOPATH based on package type:

  • "main" package - will be put into <GOPATH>/bin
  • other (library) packages - will be port into: <GOPATH>/pkg/<OS-ARCH>/<URL>[/<OWNER>]/<LIB>.a

You can see the symbols within your library ".a" (object archives> using "go tool nm <LIB>.a"

To get import your library packages into another Go program you use the "import" directive.

So with the structure above to import my "lib" I wold use: "import "org.url[\owner]\lib"

To invoke a function within this package I would have to qualify it with the package name to call one of the public visible function: " lib.PrintIt(s)". Note that the qualification only has to use the final package name and the function is upper case. Uppercase names are used by code to control visibility outside of the package.

So in go all source files within the same package have full visibility to all attributes / functions / classes within the same package (source tree directory).

Functions within source files that are in a different package only have visibilty to the "public" (upper case) declarations of another package.


So now post-module... where on top of above "pre-module" concepts Go introduces the "module".

  • modules - which is a collection of related Go packages that are released together. Unlike the "packages" above which are delimited by the common source directory they reside, the modules is declared by the "go.mod" file within the package source directly and includes all the sub-directory below this unless this contains further "go.mod" file.

While the pre-module package import path was generated based on the position of the package directory relative to "src/" directory, for modues this is defined excplicity through the "go.mod" file within the package directory.

In additional to decoupling the naming from the position relative to "src/" introduction of modules also introduces explicit version management, which is also controlled via "go.mod" file.

So looking at source tree above, Post-Module this would have:

project/  <=== workspace - gopath env should reference this bin goprog pkg <="==" go install puts library files here linux_amd64 first level is os_arch, dependent on your org.url next reflects: from the src tree [owner ] reflecting: lib.a. reflecting lib package mod downloaded versioned copies of imported [.git] <<="==" optional: repository location #1 name space (import path) identifier (like "github.com") [.git]. #2 url above owner project respository prog for main goprog.go [go.mod] to control version modules include module veryusefule.go use: import "lib veryuseful" helper.go. helper" go.mod mandatory definition [vendor depenencies ...< code>

As per pre-module the behaviour can be controlled by the GO111MODULE environment variable but as of Go 1.15 the default behaviour is as per GO111MODULE=yes, so if you have pre-module packages without "go.mod" definition file, then this should be added or the control flag changed to maintain pre-module behavior.

The "vendor" directory contained cached code based on go.mod and this cached coded and version will take priority of the same package that might have been installed higher up in the hiearchy. The vendor directory cache was avialable "pre-module" as well, but was not controlled by the go.mod file (rather by invocation flags).

Visibillty of functions , attributes and classse is controlled by using upper case and with modules you can now have a hiearchy of packages all as part of the same module.

The Tools

CompilingGo does have a compiler, but generally everyting is done with the "go" command tool. This does much more that just compile the code ..
It does builds, install and in a simple golang only project makes "make" an unnecessary accessory
  • build - does a compile check, but throws away result
  • install - either builds a binary or library
LibrariesGo has libraries, known is packages and modules but these are not like C/C++, where you have seperate header (.h) and generated object file (.o), which get collected into libaryes archive (.a)
Object FilesGo does not provide visible object files. Rather if you do an "go install" it will generate the binary executable or a go library (.a library) containing all functions/classes within your go package / module in single invocation
Environment VariablesGo uses a set of environment variable to control its behavior and where it places things. This is differnt to C/C++ where control is via -flags
Cache and Clearing it..The go tool is doing a lot work in the background. The problems is that things can get mixed up, with libraries not resolving.
A give away is when it it trying to find packages that are in your local code base, but is treating them like external packages, with the following error: blah blah blah
If you have done a clean, removed your ~/go directory and vendor generated diretory then you might need to reset the cache:
go clean -cache -modcache -i -r

Go & Make

If Go automatically builds the binary then what is the role of Make ?

While Go automatically already does many things that would otherwise be managed by Make (or Ant in Java context), it only deal with go code and libraries. So if you have code that also has seperate sql, JavaScript, CSS etc then you will likely still need to use Make to pull all these other parts together as part of your overall build process.

Targets and Build Process

Lets cover basic program & library build process. Firstly some basic Make build target conventions. Generally standard Make build conventions define the following targets:

  • build - builds the set of target: program, library in local build tree but do not install it for use
  • clean - clean out all the generated files
  • install - build and install the projects programs, libraries and documentation onto hosting machine for use
  • uninstall - remove the build program, libraries and documentation from the machine (ie reverse install)
  • dist - build distribution package for project which is targetted for deployment on another machine

Typically a Makefile will define the target programs and libraries and the source files that are used to build this.  The dependency rules then define how to compile, build archives and link these together to create the executable. With make this is ussually done a file at a time (based on the rule) by make based on dependency rule.

This is different to Go where you run "go build" in a "main" package directory and it will automatically compile all the .go files in that directory (so I do not ask go to build a specific .go program, but will throw away the result.. which is why you use "go install" which does the build but keeps the result based the output directive (-o flag).

For a library package or module, you can go into the directory and it will automatically download all the dependencies (based on go.mod) and compile all the .go files.

So here are things that typical "make" and Go do differently.

  • go build - throws away results, while "make build" keep result and aims to build the target
  • go install - had behaviour more like "make build", but it acts on entire directories while make dependency rules typically results in actions on individual files
  • go clean - has simillar behavior to a "make clean" target
  • make install - does not have go equivalent, this would be something that you could still rely on make to do...

So to align Go and make working cooperatively together you need to have dependency rules in you Makefile tha take into account the Go behaviour.

For my "Commento" build Makefile I have top level Makefile that just contains the sub-directories as make targets.

For each of the Go sub-directory targets the build target has no dependencies, the result is that the build will always fire the "Go build / install / clean" command, which will operate on all the files within the package directory. So unlike a typical C/C++ Makefile, where make needs to be appear of the files in the directory, for the Go make, this is managed by Go and make just needs to know the packages/directories that is shoud enter and run the next Makefile for.

References & Links: