In this session, we will be learning about the Terraform state
What is Terraform State?
Terraform logs information about the resources it has created in a state file. This enables Terraform to know which resources are under its control and when to update and destroy them. The terraform state file, by default, is named terraform.tfstate and is held in the same directory where Terraform is run. It is created after running terraform apply.
The actual content of this file is a JSON-formatted mapping of the resources defined in the configuration and those that exist in your infrastructure. When Terraform is run, it can then use this mapping to compare infrastructure to the code and make any adjustments as necessary.
Storing State Files
State files, by default, are stored in the local directory where Terraform is run. If you are using Terraform to test or for a personal project, this is fine (as long as your state file is secure and backed up!). However, when working on Terraform projects in a team, this becomes a problem because multiple people will need to access the state file.
Also, when using automation and CI/CD pipelines to run Terraform, the state file needs to be accessible, and permission must be given to the service principal running the pipeline to access the storage account container that holds the state file. This makes shared storage the perfect candidate to hold the state file. Permissions can be granted as needed. Azure Storage accounts or Amazon S3 buckets are an ideal choice. You can also use a tool such as Spacelift to manage your state for you.
You should store your state files remotely, not on your local machine! The location of the remote state file can then be referenced using a backendblock in the terraform block (which is usually in the main.tf file).
TASK 1: Storing state file in s3 Backend
Create a terraform.tf
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 4.0"
}
}
}
Create a provider.tf
provider "aws" {
region = "var.aws_region"
}
Create a varaible.tf, which contains all the varibles
variable "ami" {
default = "ami-007855ac798b5175e"
}
variable "instance_type" {
default = "t2.micro"
}
variable "instance_name" {
default = "batch3-demo-instance"
}
variable "bucket_name" {
default = "batch3-demo-bucket"
}
variable "state_bucket_name" {
default = "batch3-demo-state-bucket"
}
variable "state_table_name" {
default = "batch3-demo-state-table"
}
variable "aws_region" {
default = "us-east-1"
}
Create EC2.tf
resource "aws_instance" "my_instance" {
ami = var.ami
instance_type = var.instance_type
tags = {
Name = var.instance_name
}
}
Create output.tf
output "aws_ec2_instance_ip" {
value = aws_instance.my_instance.public_ip
}
Create s3.tf to create an s3 bucket
resource "aws_s3_bucket" "my_state_bucket" {
bucket = var.state_bucket_name
tags = {
Name = var.state_bucket_name
}
}
Create dynamo.tf to create a dynamodb
resource "aws_dynamodb_table" "my_state_table" {
name = var.state_table_name
billing_mode = "PAY_PER_REQUEST"
hash_key = "LockID"
attribute {
name = "LockID"
type = "S"
}
tags = {
Name = var.state_table_name
}
}
Run terraform Apply
Make sure you provide S3 and dynamodb access to the IAM user
Run the terraform commands again
We have created an EC2 instance, an s3 bucket to store the state file and dynamodb
Adding backend script
Now in terraform.tf add backend script with s3 bucket name and state file name in Key
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 4.0"
}
}
}
terraform {
backend "s3" {
bucket = "batch3-demo-state-bucket"
key = "terraform.tfstate"
region = "us-east-1"
dynamodb_table = "batch3-demo-state-table"
}
}
Now apply to Terraform init, plan and apply
Now if you go back and check the bucket, it will have a state file stored in it. This is how we store the state files in s3 Backend.
Modules project
In this project, we will be using the modules concept to create different environments like DEV, PROD and STG and add resources into it instantly
Create a provider.tf
provider "aws" {
region = "us-east-1"
}
Create a terraform.tf
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 4.0"
}
}
}
Create main.tf, where we will be using modules concept
# dev
module "dev-app" {
source = "./my_app_infra_module"
my_env = "dev"
instance_type = "t2.micro"
ami = "ami-007855ac798b5175e"
}
#prd
module "prd-app" {
source = "./my_app_infra_module"
my_env = "prd"
instance_type = "t2.medium"
ami = "ami-007855ac798b5175e"
}
#stg
module "stg-app" {
source = "./my_app_infra_module"
my_env = "stg"
instance_type = "t2.small"
ami = "ami-007855ac798b5175e"
}
Also, create a backend.tf to create an s3 bucket and dynamodb to store the state file at the time of running Backend s3.
resource "aws_s3_bucket" "my_state_bucket" {
bucket = var.state_bucket_name
tags = {
Name = var.state_bucket_name
}
}
resource "aws_dynamodb_table" "my_state_table" {
name = var.state_table_name
billing_mode = "PAY_PER_REQUEST"
hash_key = "LockID"
attribute {
name = "LockID"
type = "S"
}
tags = {
Name = var.state_table_name
}
}
variable "state_bucket_name" {
default = "batch3-demo-state-bucket"
}
variable "state_table_name" {
default = "batch3-demo-state-table"
}
variable "aws_region" {
default = "us-east-1"
}
Now create a folder, where we will be placing all our infra files
Create a variable.tf
We will be creating all the variables like my-env, instance_type, ami as string type and my_env will be the environment that we declare in the module
variable "my_env" {
description = "The environment for the app"
type = string
}
variable "instance_type" {
description = "value of the instance type"
type = string
}
variable "ami" {
description = "value of the ami"
type = string
}
Create myserver.tf
resource "aws_instance" "my_app_server" {
count = 2
ami = var.ami
instance_type = var.instance_type
tags = {
Name = "${var.my_env}-batch3-app-server"
}
}
Create a mybucket.tf
resource "aws_instance" "my_app_server" {
count = 2
ami = var.ami
instance_type = var.instance_type
tags = {
Name = "${var.my_env}-batch3-app-server"
}
}
create mytable.tf
resource "aws_dynamodb_table" "my_app_table" {
name = "${var.my_env}-batch3-app-table"
billing_mode = "PAY_PER_REQUEST"
hash_key = "userID"
attribute {
name = "userID"
type = "S"
}
tags = {
Name = "${var.my_env}-batch3-app-table"
}
}
Apply terraform commands
We have created 6 instances, 4 buckets in which one bucket will be used to store the state file when we run the backend in S3 and 4 Dynamodb tables out of which 1db is for the state file.
Backend S3
Now we will run the Backend for S3 in main.tf by adding the below code
Apply the terraform commands again
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 4.0"
}
}
}
terraform {
backend "s3" {
bucket = "batch3-demo-state-bucket"
key = "terraform.tfstate"
region = "us-east-1"
dynamodb_table = "batch3-demo-state-table"
}
}
Verify if the state file is stored in the desired bucket or not
