Getting Started with Terraform and VMware vCloud Director

If you have heard the announcements earlier this year and at VMworld 2019, you will know that VMware is embracing Terraform for VMware vCloud Director.

This means that your Automation for VMware vCloud Director just got a whole heap simpler saving you time and money.

What I’d like to demonstrate in this blog post is how I got started with Terraform, explaining what you’ll need, some of the code I use and give you a demonstration on how you can automate the creation of a new:

• vCloud Organization
• VDC
• External Network
• Organization vApp

What You Will Need for Terraform

The Terraform installation is super simple. In fact, it is just a single file. For Linux based system it is purely ‘terraform’ and for Windows based systems it is terraform.exe

You can download the latest version of Terraform from the Hashicorp website using this direct link: https://www.terraform.io/downloads.html

I have used both CentOS and Windows. Firstly starting with Centos and then migrating over to Windows so that I can easily run Powershell and Terraform together.

For my Centos system, I simply used the below command to download the Terraform executable into /opt/terraform. After downloading simply unzip it by typing ‘unzip terraform_0.12.16_linux_amd64.zip’ :

wget https://releases.hashicorp.com/terraform/0.12.16/terraform_0.12.16_linux_amd64.zip

For my Windows based system, I simply downloaded the Terraform Windows x64 file into a folder I created on D:\Scripts called Terraform. I then added this folder into my PATHS variable so that I could run terraform.exe from anywhere. This is done in System Properties – Advanced – Environment Variables

For this lab I’m running the following versions of software:

• vCloud Director 9.7
• vSphere 6.5U3
• Terraform 0.12.16
• Terraform provider.vcd 2.5.0
• Visual Studio Code 1.40.1

Terraform vCloud Provisioning Example

We’ll now move into the exciting part of this post, which is examining the code used to provision a basic vCloud Organization, VDC, External Network and vApp. All code is up on my GitHub page which is located at: https://github.com/sysadmintutorials/terraform-vcloud-create-new-org-external-network

If you have been following my Ansible posts, you will see that I now use Microsoft Visual Studio Code to build and work with my automation scripts. It makes it way more simple, then working with different programs and multiple windows. It’s a free download, check it out.

There are 3 main files that we’ll be working with:

• variables.tf (defines the variable name as a variable with a short description)
• terraform.tfvars (contains the actual variable values)
• main.tf (contains providers and resources)

vCloud Terraform variables.tf

First off we’ll take a look at our variables.tf file. In this file we can define a few settings for the variable. For example, variable “vcd_user” is saying that anytime terraform see’s var.vcd_user (which we’ll see later on in the main.tf file), refer back to this variables.tf file, find the variable and check the following settings type, default, description. In this demo we simply give a short description so we know what the variable is used for.

variable "vcd_user" {
    description = "vCloud user"
}
variable "vcd_pass" {
    description = "vCloud pass"
}

vCloud Terraform terraform.tfvars

Next, we’ll take a look at terraform.tfvars file. This is where we give value to the variables. For example org_name = “Terraform1”, means that anytime var.org_name is referenced in the main.tf file it will = Terraform1. Most variables below are self-explanatory but if you have any questions please leave a comment.

# vCloud Director Connection Variables
vcd_user = "terraform"
vcd_pass = "Password123"
vcd_url = "https://vcloud8director1.vmlab.local/api"
vcd_max_retry_timeout = "60"
vcd_allow_unverified_ssl = "true"
#vCloud Director External Network
extnet_name = "Terraform1-Lan"
extnet_description = "Terraform1 LAN - External VLAN"
extnet_gw = "192.168.10.1"
extnet_mask = "255.255.255.0"
extnet_dns1 = "8.8.8.8"
extnet_dns2 = "8.8.4.4"
extnet_suffix = "terraform1cust.local"
extnet_ip_pool_start = "192.168.10.16"
extnet_ip_pool_end = "192.168.10.31"
extnet_vcenter = "vcloud8vcenter" # vCenter Instance Name as it appears in vCloud Director
# vCloud Director Organization Variables
org_name = "Terraform1"
org_full_name = "My Terraform Organization"
org_description = "Terraform1 Create Org"
# vCloud Director Organization VDC Variables
vdc_alloc_model = "AllocationVApp" # Pay-As-You-Go
vdc_net_pool = "VMLAB pVDC A-VXLAN-NP"
vdc_pvdc_name = "VMLAB pVDC A"
vdc_name = "Terraform1-VDC-A"
vdc_description = "Terraform1 VDC Description"
vdc_storage_name = "Gold Storage Policy"
vdc_storage_limit = "102400"

vCloud Terraform main.tf

We now move onto the main.tf file. This is where we code how we want our end environment to look like.

We do this by using a provider and multiple resources. The provider we are using in this demonstration is “vcd” (vCloud Director). The resources are then responsible for different parts of vCloud Director. For example “vcd_org” is responsible for creating, modifying or deleting an Organization.

Each resource contains multiple argument references. For example, the resource “vcd_org” will have an argument reference called name, where we define the name of the Organization. I have set most of the argument references to variables. Why did I do this? Because if I want to duplicate this terraform folder and use it to set up another vCloud Organization, I only need to change the values in the terraform.tfvars file. I don’t need to scroll through the main.tf file and make changes to each line, especially if the same value is used multiple times, such as Organization Name.

The first section specifies the provider to use, in this case, ‘vcd’. We then specify our connection settings to vCloud Director

# Connect VMware vCloud Director Provider
provider "vcd" {
  user                 = var.vcd_user
  password             = var.vcd_pass
  org                  = "System"
  url                  = var.vcd_url
  max_retry_timeout    = var.vcd_max_retry_timeout
  allow_unverified_ssl = var.vcd_allow_unverified_ssl
}

The second section creates an external network within vCloud Director. This is a vlan backed vSphere portgroup attached in the back-end. Later on, we will add this external network to our Organization VDC.

# Create new External Network
resource "vcd_external_network" "extnet" {
  name        = var.extnet_name
  description = var.extnet_description
    vsphere_network {
    name    = var.extnet_name
    type    = "DV_PORTGROUP"
    vcenter = var.extnet_vcenter
  }
  ip_scope {
    gateway    = var.extnet_gw
    netmask    = var.extnet_mask
    dns1       = var.extnet_dns1
    dns2       = var.extnet_dns2
    dns_suffix = var.extnet_suffix
    static_ip_pool {
      start_address = var.extnet_ip_pool_start
      end_address   = var.extnet_ip_pool_end
    }
  }
}

This next section creates a new vCloud Organisation by specifying the name, full name, and description. You will notice there is a ‘depends_on’ setting. This means that this resource depends on the resource specified before executing. In this instance, before Terraform creates a new Organization, it must have completed the creation of the external network. This is extremely important as it prevents Terraform trying to execute a resource randomly.

# Create new vCloud Org
resource "vcd_org" "org-name" {
  name                = var.org_name
  full_name           = var.org_full_name
  description         = var.org_description
  is_enabled          = "true"
  delete_recursive    = "true"
  delete_force        = "true"
  can_publish_catalogs = "false"
  depends_on = [vcd_external_network.extnet]
}

I found a setting that is missing for the resource vcd_org, and that is policy leases. There isn’t any option to change the leases with Terraform at the moment. Therefore it sets the maximum runtime lease to 7 days and maximum storage lease to 30 days. I have an issue opened on the Terraform GitHub page which you can track with this link: https://github.com/terraform-providers/terraform-provider-vcd/issues/385

Update March 2020: Terraform vcd_provider 2.7+ now includes the ability to set the vApp and vApp templates leases.

To do this you must add the vapp_lease and vapp_template_lease as seen in the code below:

# Create new vCloud Org
resource "vcd_org" "org-name" {
  name                = var.org_name
  full_name           = var.org_full_name
  description         = var.org_description
  is_enabled          = "true"
  delete_recursive    = "true"
  delete_force        = "true"
  can_publish_catalogs = "false"
  depends_on = [vcd_external_network.extnet]
  vapp_lease {
   maximum_runtime_lease_in_sec = "0"
   power_off_on_runtime_lease_expiration = true
   maximum_storage_lease_in_sec = "0"
   delete_on_storage_lease_expiration = false
  }
 vapp_template_lease {
  maximum_storage_lease_in_sec = "0"
  delete_on_storage_lease_expiration = false
 }
}

We now create our Organization VDC. When I write the automation code, I think about the steps involved to create a VDC manually and then convert that to code.

To create a VDC we need to specify

• Name
• Description
• Organization
• Allocation Model
• Network Pool
• Provider VDC
• Compute Capacity (i.e. CPU speed and garantee’s)
• Storage Profiles
• Network Quota (how many networks the VDC can create)
• Thin Provisioning (are the VM’s thin provisioned)
• Fast Provisioning (use a parent VM as the base, track changes in a new file for this VM)

Once we have this information, we then write the code for it. You will notice the ‘depends_on’ setting once again. In order to create an Organization VDC, you must have an Organization created, therefore in this resource, we depend on the creation of the Organization before trying to execute. The code for this section looks like this:

# Create new VDC
resource "vcd_org_vdc" "vdc-name" {
  name        = var.vdc_name
  description = var.vdc_description
  org         = var.org_name
  allocation_model = var.vdc_alloc_model
  network_pool_name = var.vdc_net_pool
  provider_vdc_name = var.vdc_pvdc_name
  compute_capacity {
    cpu {
      allocated = 0
    }
    memory {
      allocated = 0
    }
  }
  storage_profile {
    name = var.vdc_storage_name
    limit = var.vdc_storage_limit
    default = true
  }
  cpu_guaranteed = 0
  memory_guaranteed = 0
  cpu_speed = 2000
  network_quota = 10
  enabled = true
  enable_thin_provisioning = true
  enable_fast_provisioning = false
  delete_force = true
  delete_recursive = true 
  depends_on = [vcd_org.org-name]
}

We’re almost done. In the second last section, I need to add the external network we created earlier to our Organization VDC. The code ‘depends_on’ the creation of an Organization VDC first.

# Org External Network
 resource "vcd_network_direct" "netdirect" {
   org = var.org_name
   vdc = var.vdc_name
   name = "Terraform1-Lan"
   external_network = "Terraform1-Lan"
   depends_on = [vcd_org_vdc.vdc-name]
 }

Lastly, we have the creation of a Server vApp

# Org vApp - Servers
 resource "vcd_vapp" "vapp" {
   name = "Servers"
   org = var.org_name
   vdc = var.vdc_name
   depends_on = [vcd_network_direct.netdirect]
 }

Running Terraform Plan

This is where all our hard work in writing this code pays off. It’s time to execute our plan.

Within the same folder where we have our main.tf, terraform.tfvars and variables.tf files, type in ‘terraform init’, this will download the vcd provider. I then type in ‘terraform plan’. This command will go through your files and check the settings against your vCloud Director environment. It will then highlight with a green + (for additions) or red – (for removal) what settings will be changed, without actually making any changes.

At the end of the plan, it will give a summary of how many additions, changes or deletions will be made

Once you have reviewed all the changes being made, it’s time to run ‘terraform apply’. When you run this command, it will give a summary of all the changes being made.

Below is a list of changes being made. Type ‘yes’ to apply.

You will then get a summary of the resources being provisioned and how many additions, changes, and deletions occurred.

Let’s say you want to make a change to the plan. For example, if we change the variable for the vApp name from Servers to Web_Servers, we can simply run ‘terraform apply’, it will recognize there is a change and it will rename the vApp for you.

If you want to go ahead and delete the whole environment, let’s say this is a dev environment that was needed for a short period of time, then you can simply type in ‘terraform destroy’ and the whole environment will be removed.

Terraform VMware vCloud Director Video Demonstration