Skip to content

Google Cloud Storage Module#

This module enhances the standard Google Cloud Storage module with additional security and management features, making it easier to create and manage GCS buckets with best practices in mind.

This module extends the upstream Google Cloud Storage module with several key enhancements:

  • Integrated KMS Key Management: Automatically creates and manages KMS keys for bucket encryption, including proper IAM bindings for the Google-managed service account.
  • Simplified Encryption Setup: Provides a streamlined interface for configuring bucket encryption with customer-managed keys (CMKs).
  • Consistent Security Posture: Enforces secure defaults and best practices for bucket configuration.
  • Reduced Boilerplate: Handles common patterns like key rotation and IAM bindings automatically.

Compatibility#

This module is compatible with Terraform version ~> 1.0 and has been tested with the following provider versions:

  • Google Provider: >= 5.43.0, < 7
  • Random Provider: >= 2.1

Usage#

The module creates one or more GCS buckets with optional KMS encryption, IAM bindings, and advanced configuration options. It handles the complexity of setting up proper encryption key management and service account permissions automatically.

Basic Example#

module "gcs_buckets" {
  source = "github.com/myops-co/gcp-terraform-modules//modules/terraform-google-cloud-storage?ref=vX.Y.Z"

  names      = ["my-bucket-001", "my-bucket-002"]
  project_id = "my-project-id"
  location   = "europe-west1"
}

Advanced Example with Encryption and Lifecycle#

module "gcs_buckets" {
  source = "github.com/myops-co/gcp-terraform-modules//modules/terraform-google-cloud-storage?ref=vX.Y.Z"

  names      = ["my-bucket-001", "my-bucket-002"]
  project_id = "my-project-id"
  location   = "europe-west1"

  encryption_keys = {
    "my-bucket-key-01" = {
      key_name        = "my-kms-key-01"
      rotation_period = "7776000s"
    },
    "my-bucket-key-02" = {
      key_name        = "my-kms-key-02"
      rotation_period = "7776000s"
    }
  }

  website = {
    main_page_suffix = "index.html"
    not_found_page   = "404.html"
  }

  cors = [{
    origin          = ["http://example.com"]
    method          = ["GET", "HEAD", "PUT", "POST", "DELETE"]
    response_header = ["*"]
    max_age_seconds = 3600
  }]

  lifecycle_rules = [{
    action = {
      type = "Delete"
    }
    condition = {
      age            = 365
      with_state     = "ANY"
      matches_prefix = "logs/"
    }
  }]

  custom_placement_config = {
    data_locations = ["EU-WEST1", "EU-WEST2"]
  }

  autoclass = {
    "my-bucket-001" = true,
    "my-bucket-002" = true
  }

  retention_policy = {
    "my-bucket-001" = {
      retention_period = 30
    },
    "my-bucket-002" = {
      retention_period = 30
    }
  }

  admins = ["user:admin@example.com"]
  bucket_viewers = {
    "my-bucket-001" = "user:viewer@example.com"
  }
  set_admin_roles  = true
  set_viewer_roles = true
}

Software#

This module requires the following software dependencies:

Service Account#

The service account used to run this module must have the following IAM roles:

  • roles/storage.admin - To create and manage Cloud Storage buckets
  • roles/cloudkms.admin - To create and manage KMS keys for bucket encryption
  • roles/cloudkms.cryptoKeyEncrypterDecrypter - To use KMS keys for encryption/decryption
  • roles/iam.serviceAccountAdmin - To bind service accounts to KMS keys

APIs#

The following GCP APIs must be enabled in the target project:

  • Cloud Storage JSON API (storage-api.googleapis.com)
  • Cloud Key Management Service (KMS) API (cloudkms.googleapis.com)
  • Cloud Resource Manager API (cloudresourcemanager.googleapis.com)

License#

This module is covered by a commercial license. Please see LICENSE for details.

Support#

This module is developed and maintained by MyOps Limited. For any support inquiries, please contact us at info@myops.co.il.

Requirements#

Name Version
terraform ~> 1
google >= 5.43.0, < 7
random >= 2.1

Providers#

Name Version
google >= 5.43.0, < 7

Modules#

Name Source Version
storage terraform-google-modules/cloud-storage/google ~> 11.0.0

Resources#

Name Type
google_kms_crypto_key.this resource
google_kms_crypto_key_iam_binding.storage_sa_kms_access resource
google_kms_key_ring.this resource
google_project.project data source

Inputs#

Name Description Type Default Required
admins IAM-style members who will be granted roles/storage.objectAdmin on all buckets. list(string) [] no
autoclass Optional map of lowercase unprefixed bucket name => boolean, defaults to false. map(bool) {} no
bucket_admins Map of lowercase unprefixed name => comma-delimited IAM-style per-bucket admins. map(string) {} no
bucket_creators Map of lowercase unprefixed name => comma-delimited IAM-style per-bucket creators. map(string) {} no
bucket_hmac_key_admins Map of lowercase unprefixed name => comma-delimited IAM-style per-bucket HMAC Key admins. map(string) {} no
bucket_lifecycle_rules Additional lifecycle_rules for specific buckets. Map of lowercase unprefixed name => list of lifecycle rules to configure. 
map(set(object({
 # Object with keys:
 # - type - The type of the action of this Lifecycle Rule. Supported values: Delete and SetStorageClass.
 # - storage_class - (Required if action type is SetStorageClass) The target Storage Class of objects affected by this Lifecycle Rule.
 action = map(string)

 # Object with keys:
 # - age - (Optional) Minimum age of an object in days to satisfy this condition.
 # - created_before - (Optional) Creation date of an object in RFC 3339 (e.g. 2017-06-13) to satisfy this condition.
 # - with_state - (Optional) Match to live and/or archived objects. Supported values include: "LIVE", "ARCHIVED", "ANY".
 # - matches_storage_class - (Optional) Comma delimited string for storage class of objects to satisfy this condition. Supported values include: MULTI_REGIONAL, REGIONAL, NEARLINE, COLDLINE, STANDARD, DURABLE_REDUCED_AVAILABILITY.
 # - num_newer_versions - (Optional) Relevant only for versioned objects. The number of newer versions of an object to satisfy this condition.
 # - custom_time_before - (Optional) A date in the RFC 3339 format YYYY-MM-DD. This condition is satisfied when the customTime metadata for the object is set to an earlier date than the date used in this lifecycle condition.
 # - days_since_custom_time - (Optional) The number of days from the Custom-Time metadata attribute after which this condition becomes true.
 # - days_since_noncurrent_time - (Optional) Relevant only for versioned objects. Number of days elapsed since the noncurrent timestamp of an object.
 # - noncurrent_time_before - (Optional) Relevant only for versioned objects. The date in RFC 3339 (e.g. 2017-06-13) when the object became nonconcurrent.
 condition = map(string)
 })))
 {} no
bucket_policy_only Disable ad-hoc ACLs on specified buckets. Defaults to true. Map of lowercase unprefixed name => boolean map(bool) {} no
bucket_storage_admins Map of lowercase unprefixed name => comma-delimited IAM-style per-bucket storage admins. map(string) {} no
bucket_viewers Map of lowercase unprefixed name => comma-delimited IAM-style per-bucket viewers. map(string) {} no
cors Set of maps of mixed type attributes for CORS values. See appropriate attribute types here: https://www.terraform.io/docs/providers/google/r/storage_bucket.html#cors set(any) [] no
creators IAM-style members who will be granted roles/storage.objectCreators on all buckets. list(string) [] no
custom_placement_config Map of lowercase unprefixed name => custom placement config object. Format is the same as described in provider documentation https://www.terraform.io/docs/providers/google/r/storage_bucket#custom_placement_config any {} no
default_event_based_hold Enable event based hold to new objects added to specific bucket. Defaults to false. Map of lowercase unprefixed name => boolean map(bool) {} no
encryption_keys Optional map of lowercase unprefixed name => object containing key_name and rotation_period. The rotation_period defaults to 90 days (7776000s) if not specified. 
map(object({
 key_name = string
 rotation_period = optional(string, "7776000s")
 }))
 {} no
folders Map of lowercase unprefixed name => list of top level folder objects. map(list(string)) {} no
force_destroy Optional map of lowercase unprefixed name => boolean, defaults to false. map(bool) {} no
hmac_key_admins IAM-style members who will be granted roles/storage.hmacKeyAdmin on all buckets. list(string) [] no
hmac_service_accounts List of HMAC service accounts to grant access to GCS. map(string) {} no
labels Labels to be attached to the buckets map(string) {} no
lifecycle_rules List of lifecycle rules to configure. Format is the same as described in provider documentation https://www.terraform.io/docs/providers/google/r/storage_bucket.html#lifecycle_rule except condition.matches_storage_class should be a comma delimited string. 
set(object({
 # Object with keys:
 # - type - The type of the action of this Lifecycle Rule. Supported values: Delete and SetStorageClass.
 # - storage_class - (Required if action type is SetStorageClass) The target Storage Class of objects affected by this Lifecycle Rule.
 action = map(string)

 # Object with keys:
 # - age - (Optional) Minimum age of an object in days to satisfy this condition.
 # - created_before - (Optional) Creation date of an object in RFC 3339 (e.g. 2017-06-13) to satisfy this condition.
 # - with_state - (Optional) Match to live and/or archived objects. Supported values include: "LIVE", "ARCHIVED", "ANY".
 # - matches_storage_class - (Optional) Comma delimited string for storage class of objects to satisfy this condition. Supported values include: MULTI_REGIONAL, REGIONAL, NEARLINE, COLDLINE, STANDARD, DURABLE_REDUCED_AVAILABILITY.
 # - matches_prefix - (Optional) One or more matching name prefixes to satisfy this condition.
 # - matches_suffix - (Optional) One or more matching name suffixes to satisfy this condition.
 # - num_newer_versions - (Optional) Relevant only for versioned objects. The number of newer versions of an object to satisfy this condition.
 # - custom_time_before - (Optional) A date in the RFC 3339 format YYYY-MM-DD. This condition is satisfied when the customTime metadata for the object is set to an earlier date than the date used in this lifecycle condition.
 # - days_since_custom_time - (Optional) The number of days from the Custom-Time metadata attribute after which this condition becomes true.
 # - days_since_noncurrent_time - (Optional) Relevant only for versioned objects. Number of days elapsed since the noncurrent timestamp of an object.
 # - noncurrent_time_before - (Optional) Relevant only for versioned objects. The date in RFC 3339 (e.g. 2017-06-13) when the object became nonconcurrent.
 condition = map(string)
 }))
 [] no
location Bucket location. string "EU" no
logging Map of lowercase unprefixed name => bucket logging config object. Format is the same as described in provider documentation https://www.terraform.io/docs/providers/google/r/storage_bucket.html#logging any {} no
names Bucket name suffixes. list(string) n/a yes
prefix Prefix used to generate the bucket name. string "" no
project_id Bucket project id. string n/a yes
public_access_prevention Prevents public access to a bucket. Acceptable values are inherited or enforced. If inherited, the bucket uses public access prevention, only if the bucket is subject to the public access prevention organization policy constraint. string "inherited" no
randomize_suffix Adds an identical, but randomized 4-character suffix to all bucket names bool false no
retention_policy Map of retention policy values. Format is the same as described in provider documentation https://www.terraform.io/docs/providers/google/r/storage_bucket#retention_policy any {} no
set_admin_roles Grant roles/storage.objectAdmin role to admins and bucket_admins. bool false no
set_creator_roles Grant roles/storage.objectCreator role to creators and bucket_creators. bool false no
set_hmac_access Set S3 compatible access to GCS. bool false no
set_hmac_key_admin_roles Grant roles/storage.hmacKeyAdmin role to hmac_key_admins and bucket_hmac_key_admins. bool false no
set_storage_admin_roles Grant roles/storage.admin role to storage_admins and bucket_storage_admins. bool false no
set_viewer_roles Grant roles/storage.objectViewer role to viewers and bucket_viewers. bool false no
soft_delete_policy Soft delete policies to apply. Map of lowercase unprefixed name => soft delete policy. Format is the same as described in provider documentation https://www.terraform.io/docs/providers/google/r/storage_bucket.html#nested_soft_delete_policy map(any) {} no
storage_admins IAM-style members who will be granted roles/storage.admin on all buckets. list(string) [] no
storage_class Bucket storage class. string "STANDARD" no
versioning Optional map of lowercase unprefixed name => boolean, defaults to false. map(bool) {} no
viewers IAM-style members who will be granted roles/storage.objectViewer on all buckets. list(string) [] no
website Map of website values. Supported attributes: main_page_suffix, not_found_page map(any) {} no

Outputs#

Name Description
bucket Bucket resource (for single use).
buckets Bucket resources as list.
buckets_map Bucket resources by name.
hmac_keys List of HMAC keys.
name Bucket name (for single use).
names Bucket names.
names_list List of bucket names.
url Bucket URL (for single use).
urls Bucket URLs.
urls_list List of bucket URLs.