Skip to content

Google Cloud GKE Module#

This Terraform module creates and manages Google Kubernetes Engine (GKE) clusters with comprehensive configuration options for production-ready Kubernetes deployments. It supports both standard and autopilot GKE clusters, and supports binary authorization configuration.

Overview#

The module provisions Google Kubernetes Engine clusters with enterprise-grade features including:

  • Standard and Autopilot GKE Clusters: Support for both production-ready standard Kubernetes clusters and fully managed autopilot clusters.
  • Multi-Node Pool Support: Flexible node pool configurations for standard clusters.
  • Binary Authorization: Container image security policies to ensure only trusted images are deployed.
  • Private Clusters: Secure network isolation for your clusters.
  • Auto-scaling: Horizontal and vertical pod autoscaling.
  • Backup & Recovery: GKE backup configurations for disaster recovery.
  • Monitoring & Logging: Comprehensive observability with Google Cloud's operations suite.

Features#

  • ✅ Standard GKE Clusters: Production-ready Kubernetes clusters.
  • ✅ Autopilot GKE Clusters: Fully managed GKE clusters with a hands-off operational model.
  • ✅ Multi-Node Pools: Multiple node pools with different configurations for standard clusters.
  • ✅ Binary Authorization: Container image security and attestation.
  • ✅ Private Clusters: Secure private networking.
  • ✅ Cluster Autoscaling: Automatic node provisioning and scaling.
  • ✅ Backup for GKE: Automated backup and recovery.
  • ✅ Workload Identity: Secure service-to-service authentication.
  • ✅ Add-ons Support: Configurable GKE add-ons (Istio, Config Connector, etc.).
  • ✅ Release Channels: Automated cluster upgrades.
  • ✅ Monitoring & Logging: Cloud Monitoring and Logging integration.
  • ✅ Network Policies: Pod-level network security.
  • ✅ Maintenance Windows: Controlled cluster maintenance.

Usage#

Autopilot GKE Cluster#

To create an Autopilot cluster, set the autopilot variable to true. In this mode, node pools are managed by Google, so the gke_nodepools variable is not used.

module "gke_cluster" {
  source = "./modules/terraform-google-gke"

  project_id = "my-project-id"
  name       = "my-autopilot-cluster"
  location   = "me-west1"
  autopilot  = true

  vpc_config = {
    network    = "projects/my-project-id/global/networks/shared-vpc-network"
    subnetwork = "projects/my-project-id/regions/me-west1/subnetworks/subnet-shared-prod-me-west1"
  }

  access_config = {}

  release_channel = "REGULAR"

  enable_addons = {
    horizontal_pod_autoscaling = true
    http_load_balancing        = true
    network_policy             = true
  }

  enable_features = {
    workload_identity = true
    gateway_api       = true
  }

  logging_config = {
    enable_system_logs             = true
  }

  monitoring_config = {
    enable_system_metrics     = true
    enable_managed_prometheus = true #  Managed Service for Prometheus cannot be disabled in Autopilot clusters
  }

  maintenance_config = {
    daily_window_start_time = "03:00"
  }

  deletion_protection = false

  binary_authorization = {
    enabled = false
  }
}

Standard GKE Cluster with Node Pools#

For a standard cluster, set autopilot to false and define your node pools in the gke_nodepools map.

module "gke_cluster_standard" {
  source = "./modules/terraform-google-gke"

  project_id = "my-project-id"
  name       = "my-standard-cluster"
  location   = "us-central1"
  autopilot  = false

  vpc_config = {
    network    = "projects/my-project-id/global/networks/my-vpc"
    subnetwork = "projects/my-project-id/regions/us-central1/subnetworks/my-subnet"
  }

  default_nodepool = {
    remove_pool = true
  }

  gke_nodepools = {
    "default-pool" = {
      cluster_name = "my-standard-cluster"
      location     = "us-central1"
      project_id   = "my-project-id"
      node_count = {
        initial = 1
      }
      node_config = {
        machine_type = "e2-medium"
      }
    }
  }
}

Service Account#

The service account used by the GKE nodes requires the following IAM roles on the project:

  • roles/monitoring.viewer
  • roles/monitoring.metricWriter
  • roles/logging.logWriter
  • roles/stackdriver.resourceMetadata.writer
  • roles/container.defaultNodeServiceAccount

APIs#

The following APIs must be enabled on the project:

  • container.googleapis.com
  • monitoring.googleapis.com
  • logging.googleapis.com

License#

This module is covered by a commercial license. Please see LICENSE for details.

Support#

This module is developed and maintained by MyOps Limited. For any support inquiries, please contact us at info@myops.co.il.

Requirements#

Name Version
terraform ~> 1
google >= 6.28.0, < 7.0.0
google-beta >= 6.28.0, < 7.0.0

Providers#

Name Version
google >= 6.28.0, < 7.0.0

Modules#

Name Source Version
nodepool github.com/GoogleCloudPlatform/cloud-foundation-fabric//modules/gke-nodepool v42.1.0
terraform-google-gke-autopilot github.com/GoogleCloudPlatform/cloud-foundation-fabric//modules/gke-cluster-autopilot v42.1.0
terraform-google-gke-standard github.com/GoogleCloudPlatform/cloud-foundation-fabric//modules/gke-cluster-standard v42.1.0

Resources#

Name Type
google_binary_authorization_policy.deployment_policy resource

Inputs#

Name Description Type Default Required
access_config Control plane endpoint and nodes access configurations. 
object({
    dns_access = optional(bool, true)
    ip_access = optional(object({
      authorized_ranges               = optional(map(string), {})
      gcp_public_cidrs_access_enabled = optional(bool)
      disable_public_endpoint         = optional(bool, true)
      private_endpoint_config = optional(object({
        endpoint_subnetwork = optional(string)
        global_access       = optional(bool, true)
      }), {})
    }), {})
    private_nodes = optional(bool, true)
  })
 {} no
autopilot Enable autopilot mode. bool false no
backup_configs Configuration for Backup for GKE. 
object({
    enable_backup_agent = optional(bool, false)
    backup_plans = optional(map(object({
      region                            = string
      applications                      = optional(map(list(string)))
      encryption_key                    = optional(string)
      include_secrets                   = optional(bool, true)
      include_volume_data               = optional(bool, true)
      labels                            = optional(map(string))
      namespaces                        = optional(list(string))
      schedule                          = optional(string)
      retention_policy_days             = optional(number)
      retention_policy_lock             = optional(bool, false)
      retention_policy_delete_lock_days = optional(number)
    })), {})
  })
 {} no
binary_authorization Configurations for binary authorizations 
object({
    enabled = optional(bool, false)

    admission_whitelist_patterns = optional(map(object({
      name_pattern = string
    })), {})

    default_evaluation_mode       = optional(string, "ALWAYS_DENY")
    default_enforcement_mode      = optional(string, "ENFORCED_BLOCK_AND_AUDIT_LOG")
    global_policy_evaluation_mode = optional(string, "ENABLE")
    evaluation_mode               = optional(string, "REQUIRE_ATTESTATION")
    enforcement_mode              = optional(string, "ENFORCED_BLOCK_AND_AUDIT_LOG")
    attestors                     = optional(list(string))
  })
 {} no
cluster_autoscaling Enable and configure limits for Node Auto-Provisioning with Cluster Autoscaler. 
object({
    enabled             = optional(bool, true)
    autoscaling_profile = optional(string, "BALANCED")
    auto_provisioning_defaults = optional(object({
      boot_disk_kms_key = optional(string)
      disk_size         = optional(number)
      disk_type         = optional(string, "pd-standard")
      image_type        = optional(string)
      oauth_scopes      = optional(list(string))
      service_account   = optional(string)
      management = optional(object({
        auto_repair  = optional(bool, true)
        auto_upgrade = optional(bool, true)
      }))
      shielded_instance_config = optional(object({
        integrity_monitoring = optional(bool, true)
        secure_boot          = optional(bool, false)
      }))
      upgrade_settings = optional(object({
        blue_green = optional(object({
          node_pool_soak_duration = optional(string)
          standard_rollout_policy = optional(object({
            batch_percentage    = optional(number)
            batch_node_count    = optional(number)
            batch_soak_duration = optional(string)
          }))
        }))
        surge = optional(object({
          max         = optional(number)
          unavailable = optional(number)
        }))
      }))
      # add validation rule to ensure only one is present if upgrade settings is defined
    }))
    cpu_limits = optional(object({
      min = optional(number, 0)
      max = number
    }))
    mem_limits = optional(object({
      min = optional(number, 0)
      max = number
    }))
    accelerator_resources = optional(list(object({
      resource_type = string
      min           = optional(number, 0)
      max           = number
    })))
  })
 null no
default_nodepool Enable default nodepool. 
object({
    remove_pool        = optional(bool, true)
    initial_node_count = optional(number, 1)
  })
 {} no
deletion_protection Whether or not to allow Terraform to destroy the cluster. Unless this field is set to false in Terraform state, a terraform destroy or terraform apply that would delete the cluster will fail. bool true no
description Cluster description. string null no
enable_addons Addons enabled in the cluster (true means enabled). 
object({
    cloudrun                       = optional(bool, false)
    config_connector               = optional(bool, false)
    dns_cache                      = optional(bool, false)
    gce_persistent_disk_csi_driver = optional(bool, false)
    gcp_filestore_csi_driver       = optional(bool, false)
    gcs_fuse_csi_driver            = optional(bool, false)
    horizontal_pod_autoscaling     = optional(bool, false)
    http_load_balancing            = optional(bool, false)
    istio = optional(object({
      enable_tls = bool
    }))
    kalm           = optional(bool, false)
    network_policy = optional(bool, false)
    stateful_ha    = optional(bool, false)
  })
 
{
  "horizontal_pod_autoscaling": true,
  "http_load_balancing": true
}
 no
enable_features Enable cluster-level features. Certain features allow configuration. 
object({
    beta_apis                         = optional(list(string))
    binary_authorization              = optional(bool, false)
    cilium_clusterwide_network_policy = optional(bool, false)
    cost_management                   = optional(bool, false)
    dns = optional(object({
      provider = optional(string)
      scope    = optional(string)
      domain   = optional(string)
    }))
    database_encryption = optional(object({
      state    = string
      key_name = string
    }))
    dataplane_v2         = optional(bool, false)
    fqdn_network_policy  = optional(bool, false)
    gateway_api          = optional(bool, false)
    groups_for_rbac      = optional(string)
    image_streaming      = optional(bool, false)
    intranode_visibility = optional(bool, false)
    l4_ilb_subsetting    = optional(bool, false)
    mesh_certificates    = optional(bool)
    pod_security_policy  = optional(bool, false)
    resource_usage_export = optional(object({
      dataset                              = string
      enable_network_egress_metering       = optional(bool)
      enable_resource_consumption_metering = optional(bool)
    }))
    service_external_ips = optional(bool, true)
    shielded_nodes       = optional(bool, false)
    tpu                  = optional(bool, false)
    upgrade_notifications = optional(object({
      topic_id = optional(string)
    }))
    vertical_pod_autoscaling = optional(bool, false)
    workload_identity        = optional(bool, true)
  })
 
{
  "workload_identity": true
}
 no
gke_nodepools ## https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/blob/master/modules/gke-nodepool/variables.tf 
map(object({
    cluster_id        = optional(string, null)
    cluster_name      = string
    gke_version       = optional(string, null)
    k8s_labels        = optional(map(string), {})
    labels            = optional(map(string), {})
    location          = string
    max_pods_per_node = optional(number, null)
    name              = optional(string, null)
    network_config = optional(object({
      enable_private_nodes = optional(bool)
      pod_range = optional(object({
        cidr   = optional(string)
        create = optional(bool, false)
        name   = optional(string)
      }), {})
      additional_node_network_configs = optional(list(object({
        network    = string
        subnetwork = string
      })), [])
      additional_pod_network_configs = optional(list(object({
        subnetwork          = string
        secondary_pod_range = string
        max_pods_per_node   = string
      })), [])
      total_egress_bandwidth_tier        = optional(string)
      pod_cidr_overprovisioning_disabled = optional(bool, false)
    }), null)
    node_config = optional(object({
      boot_disk_kms_key   = optional(string)
      disk_size_gb        = optional(number)
      disk_type           = optional(string)
      ephemeral_ssd_count = optional(number)
      gcfs                = optional(bool, false)
      guest_accelerator = optional(object({
        count = number
        type  = string
        gpu_driver = optional(object({
          version                    = string
          partition_size             = optional(string)
          max_shared_clients_per_gpu = optional(number)
        }))
      }))
      local_nvme_ssd_count = optional(number)
      gvnic                = optional(bool, false)
      image_type           = optional(string)
      kubelet_config = optional(object({
        cpu_manager_policy                     = string
        cpu_cfs_quota                          = optional(bool)
        cpu_cfs_quota_period                   = optional(string)
        insecure_kubelet_readonly_port_enabled = optional(string)
        pod_pids_limit                         = optional(number)
        container_log_max_size                 = optional(string)
        container_log_max_files                = optional(number)
        image_gc_low_threshold_percent         = optional(number)
        image_gc_high_threshold_percent        = optional(number)
        image_minimum_gc_age                   = optional(string)
        image_maximum_gc_age                   = optional(string)
        allowed_unsafe_sysctls                 = optional(list(string), [])
      }))
      linux_node_config = optional(object({
        sysctls     = optional(map(string))
        cgroup_mode = optional(string)
      }))
      local_ssd_count       = optional(number)
      machine_type          = optional(string)
      metadata              = optional(map(string))
      min_cpu_platform      = optional(string)
      preemptible           = optional(bool)
      sandbox_config_gvisor = optional(bool)
      shielded_instance_config = optional(object({
        enable_integrity_monitoring = optional(bool)
        enable_secure_boot          = optional(bool)
      }))
      spot                          = optional(bool)
      workload_metadata_config_mode = optional(string)
      }), {
      disk_type = "pd-balanced"
    })
    node_count = optional(object({
      current = optional(number)
      initial = number
      }), {
      initial = 1
    })
    node_locations = optional(list(string), null)
    nodepool_config = optional(object({
      autoscaling = optional(object({
        location_policy = optional(string)
        max_node_count  = optional(number)
        min_node_count  = optional(number)
        use_total_nodes = optional(bool, false)
      }))
      management = optional(object({
        auto_repair  = optional(bool)
        auto_upgrade = optional(bool)
      }))
      placement_policy = optional(object({
        type         = string
        policy_name  = optional(string)
        tpu_topology = optional(string)
      }))
      queued_provisioning = optional(bool, false)
      upgrade_settings = optional(object({
        max_surge       = number
        max_unavailable = number
        strategy        = optional(string)
        blue_green_settings = optional(object({
          node_pool_soak_duration = optional(string)
          standard_rollout_policy = optional(object({
            batch_percentage    = optional(number)
            batch_node_count    = optional(number)
            batch_soak_duration = optional(string)
          }))
        }))
      }))
    }), null)
    project_id = string
    reservation_affinity = optional(object({
      consume_reservation_type = string
      key                      = optional(string)
      values                   = optional(list(string))
    }), null)
    service_account = optional(object({
      create       = optional(bool, false)
      email        = optional(string)
      oauth_scopes = optional(list(string))
      display_name = optional(string)
    }), {})
    sole_tenant_nodegroup = optional(string, null)
    tags                  = optional(list(string), null)
    taints = optional(map(object({
      value  = string
      effect = string
    })), {})
  }))
 {} no
issue_client_certificate Enable issuing client certificate. bool false no
labels Cluster resource labels. map(string) {} no
location Cluster zone or region. string n/a yes
logging_config Logging configuration. 
object({
    enable_system_logs             = optional(bool, true)
    enable_workloads_logs          = optional(bool, true)
    enable_api_server_logs         = optional(bool, false)
    enable_scheduler_logs          = optional(bool, false)
    enable_controller_manager_logs = optional(bool, false)
  })
 {} no
maintenance_config Maintenance window configuration. 
object({
    daily_window_start_time = optional(string)
    recurring_window = optional(object({
      start_time = string
      end_time   = string
      recurrence = string
    }))
    maintenance_exclusions = optional(list(object({
      name       = string
      start_time = string
      end_time   = string
      scope      = optional(string)
    })))
  })
 
{
  "daily_window_start_time": "03:00",
  "maintenance_exclusion": [],
  "recurring_window": null
}
 no
max_pods_per_node Maximum number of pods per node in this cluster. number 110 no
min_master_version Minimum version of the master, defaults to the version of the most recent official release. string null no
monitoring_config Monitoring configuration. Google Cloud Managed Service for Prometheus is enabled by default. 
object({
    enable_system_metrics = optional(bool, true)
    # Control plane metrics
    enable_api_server_metrics         = optional(bool, false)
    enable_controller_manager_metrics = optional(bool, false)
    enable_scheduler_metrics          = optional(bool, false)
    # Kube state metrics
    enable_daemonset_metrics   = optional(bool, false)
    enable_deployment_metrics  = optional(bool, false)
    enable_hpa_metrics         = optional(bool, false)
    enable_pod_metrics         = optional(bool, false)
    enable_statefulset_metrics = optional(bool, false)
    enable_storage_metrics     = optional(bool, false)
    # Google Cloud Managed Service for Prometheus
    enable_managed_prometheus = optional(bool, true)
    advanced_datapath_observability = optional(object({
      enable_metrics = bool
      enable_relay   = optional(bool)
    }))
  })
 {} no
name Cluster name. string n/a yes
node_config Node-level configuration. 
object({
    boot_disk_kms_key = optional(string)
    k8s_labels        = optional(map(string))
    labels            = optional(map(string))
    service_account   = optional(string)
    tags              = optional(list(string))
  })
 {} no
node_locations Zones in which the cluster's nodes are located. list(string) [] no
project_id Cluster project id. string n/a yes
release_channel Release channel for GKE upgrades. Clusters created in the Autopilot mode must use a release channel. Choose between "RAPID", "REGULAR", and "STABLE". string "REGULAR" no
vpc_config VPC-level configuration. 
object({
    disable_default_snat = optional(bool)
    network              = string
    subnetwork           = string
    secondary_range_blocks = optional(object({
      pods     = string
      services = string
    }))
    secondary_range_names = optional(object({
      pods     = optional(string)
      services = optional(string)
    }))
    additional_ranges = optional(list(string))
    stack_type        = optional(string)
  })
 n/a yes

Outputs#

Name Description
ca_certificate Public certificate of the cluster (base64-encoded).
cluster Cluster resource.
endpoint Cluster endpoint.
id Fully qualified cluster id.
location Cluster location.
master_version Master version.
name Cluster name.
nodepool_ids Fully qualified nodepool id.
nodepool_names Nodepool name.
nodepool_service_account_email Service account email.
notifications GKE PubSub notifications topic.
self_link Cluster self link.
workload_identity_pool Workload identity pool.