gcp-hashi-cluster: a Consul/Nomad cluster for Google Cloud

gcp-hashi-cluster is a pre-configured cluster for running Hashicorp’s Consul, Nomad and Vault on Google Cloud Platform.

It is designed to get you up and running quickly with minimal configuration and sensible defaults. No prior knowledge of Consul, Nomad or GCP is assumed. This guide will have you running containerized services, accessible from HTTPS endpoints, in under an hour.

The cluster looks something like this:

_images/hashi-cluster2.png

Typically there are 3 Consul/Nomad servers and at least 1 Vault server. Your services run on the hashi-clients and communicate over a Consul Connect service mesh. The GCP load-balancer receives incoming HTTPs requests and Traefik routes them to services using a Consul sidecar proxy.

Limitations and opinionated configurations

  • The cluster is configured to run in a single region, though VMs are spread across multiple zones within this region for high availability.

  • All VMs run Ubuntu 20.04 (18.04 is also supported). To use another OS you will need to modify the packer scripts. The the local build scripts (mainly bash and Terraform) have only been tested on a Ubuntu development machine.

  • It is assumed you have a DNS domain name for this cluster. One domain name per cluster is assumed, though you may add sub-domains. Traefik is used as the cluster’s reverse proxy, fronted by a Google load-balancer that manages the SSL certificates.

  • This guide uses a Shared VPC setup so we can run separate “service projects” with shared networking configuration at the organization level. This is good practice generally but it means that a regular Gmail account won’t do, you’ll need a Cloud Identity or GSuite user account so that a GCP Organization can be created.

  • The GCP IAM user/resource permissions configurations used here are intended to get you started quickly as a lone developer. You may want more fine-grained IAM restrictions if you’re using this within a large development team. The same is true for the Consul, Nomad and Vault ACL configurations.

Remaining work

  • Metrics and logging isn’t fully integrated with Google Cloud. It’s working with some services but not yet with others.

  • There is an issue with Nomad being unable to authenticate with the GCP Container Registry, where you typically store private Docker images. Here is a workaround.

  • Although the build scripts will get you up and running, some scripts for day-to-day operations are missing. For example to add a new node to an already-running cluster, there is no script for automating this.

  • The instances run in a single region but the load-balancer and public IP address are configured as global resources, this isn’t ideal, it means external HTTP clients’ may experience higher response latency. Changing this shouldn’t be too difficult, see this guide.

  • Forseti will be added in future. This allows you to visualize and monitor your GCP IAM roles, permissions and service accounts.

Caution

This project is a work in progress, it hasn’t been battle-tested in production or audited by security professionals.

License

The project is licensed under the 3-clause BSD license.

1. Install requirements →