Automate AWS Resource Reporting with Bash

"The difference between a DevOps engineer and someone who just runs commands is one word: automation."

Why This Article Exists

Every team running workloads on AWS faces the same quiet problem:

"What resources are we actually running right now — and is anything provisioned that should not be?"

It sounds like a five-minute task. In reality, most teams handle it in one of three inefficient ways:

Logging into the AWS Console manually and clicking through dashboards
Running ad-hoc CLI commands reactively, only when something breaks
Paying for third-party visibility tools before they even understand their own baseline

None of these scale. None of them are auditable. And none of them run without a human involved.

The production-correct approach is a scheduled, automated script that collects resource data daily, formats it clearly, saves it with a timestamp, and runs completely unattended. That is what this article builds — step by step, from a blank file to a live deployment on AWS EC2.

This is not a tutorial about commands. It is about engineering a real solution end to end.

What You Will Learn

Topic	What It Covers
Production Bash scripting	`set -euo pipefail`, functions, logging, error handling
AWS CLI with `--query`	Extracting structured data from AWS APIs programmatically
Output redirection	`>` vs `>>`, `2>&1`, `tee` — and why each matters
EC2 deployment	SCP upload, SSH access, `chmod`, live testing
Cron job scheduling	Syntax, timezone handling, log capture
IAM Role authentication	Why stored access keys are dangerous and how to avoid them
Report lifecycle management	Auto-cleanup of old files, disk hygiene

Architecture — Design Before You Code

Understanding the full system flow before writing a single line is what separates a script from a solution.

Three clean layers. Your local machine is only involved at setup time. After that, the EC2 instance runs everything independently. This is the correct mental model for production automation — it should not depend on your laptop being open.

Prerequisites

Set these up before starting. Do not skip the IAM Role step — it is the difference between a learning script and a production-safe one.

Requirement	How to Verify
AWS Account with EC2 running (Ubuntu 22.04)	AWS Console → EC2 → Instances
SSH key pair `.pem` file	Should have been downloaded when launching EC2
SSH access confirmed	`ssh -i key.pem ubuntu@YOUR_EC2_IP` connects successfully
AWS CLI installed on EC2	`aws --version` (install: `sudo apt install awscli -y`)
IAM Role attached to EC2	EC2 → Actions → Security → Modify IAM Role

Setting up the IAM Role (recommended over aws configure):

AWS Console → IAM → Roles → Create Role
Trusted entity: AWS Service → EC2
Attach policy: ReadOnlyAccess (minimum needed for this script)
Name the role: ec2-resource-reporter
EC2 Console → select your instance → Actions → Security → Modify IAM Role
Attach ec2-resource-reporter

Once attached, your EC2 instance authenticates automatically. No keys stored. No aws configure needed.

Why this matters: Access keys stored on an EC2 instance are a known attack surface. If an attacker gains access to the instance, they inherit those credentials. An IAM Role removes this risk entirely. This is an AWS security best practice, not optional advice.

Bash Foundations — What This Script Uses

If you have a development background, Bash will feel familiar in structure but strict in syntax. Here are the core patterns used in the script — explained clearly so you can read and modify the code confidently.

Variables

REPORT_DIR="/home/ubuntu/aws-reports"     # static string
DATE=$(date +"%Y-%m-%d")                  # dynamic — runs command, captures output
REGION=$(aws configure get region 2>/dev/null || echo "us-east-1")

$( ) is command substitution — it runs the command inside and replaces itself with the output. DATE is not a hardcoded string; it is computed fresh every time the script runs.

The || on the REGION line is a fallback operator — if aws configure get region fails or returns nothing, use "us-east-1" instead. Defensive scripting from the start.

File Redirection

echo "Header line" > "$REPORT_FILE"    # Creates file fresh (overwrites if exists)
echo "Next line"  >> "$REPORT_FILE"    # Appends — does NOT overwrite

The first > is intentional — it starts a clean report file every day. Every subsequent write uses >> to append to the same file. This is a precise design choice, not a random mix.

Error Stream Capture

aws ec2 describe-instances --output table >> "$REPORT_FILE" 2>&1

2>&1 redirects stderr (file descriptor 2) into wherever stdout (file descriptor 1) is going — in this case, the report file. Without this, if an AWS API call fails, the error message disappears and you get an empty section with no explanation. With it, the error is captured inline and visible when you read the report.

Logging with `tee`

log() {
    echo "[\((date +"%Y-%m-%d %H:%M:%S")]  \)1" | tee -a "$LOG_FILE"
}

tee -a does two things simultaneously: appends to the log file AND prints to the terminal. When the script runs manually (via SSH), you see output in real time. When it runs via cron (no terminal), it writes silently to the file. Both cases produce a timestamped audit trail.

`set -euo pipefail` — The Safety Net

set -euo pipefail

Flag	Behaviour Without It	Behaviour With It
`-e`	Script continues after a failing command	Script exits immediately on failure
`-u`	Undefined variables expand to empty string silently	Script exits with an error
`-o pipefail`	Pipe failure is hidden if final command succeeds	Pipe failure is caught and script exits

Most tutorials only mention set -e. The full combination is what production scripts use. Add it as the second line of every Bash script you write.

The Full Production Script

#!/bin/bash

# ================================================================
# aws_resource_report.sh
#
# Purpose  : Collect and report AWS resource usage daily
# Schedule : Cron — every day at 20:00
# Auth     : IAM Role attached to EC2 (preferred) or aws configure
# Output   : /home/ubuntu/aws-reports/aws_report_YYYY-MM-DD.txt
# Tested on: Ubuntu 22.04 / AWS CLI v2
# ================================================================

set -euo pipefail

# ── Configuration ────────────────────────────────────────────────
REPORT_DIR="/home/ubuntu/aws-reports"
LOG_FILE="$REPORT_DIR/script.log"
DATE=$(date +"%Y-%m-%d")
TIME=$(date +"%H:%M:%S")
REPORT_FILE="\(REPORT_DIR/aws_report_\)DATE.txt"
REGION=$(aws configure get region 2>/dev/null || echo "us-east-1")

# Create report directory if it does not exist
mkdir -p "$REPORT_DIR"

# ── Logging Helper ───────────────────────────────────────────────
# tee -a writes to log file AND prints to terminal simultaneously
log() {
    echo "[\((date +"%Y-%m-%d %H:%M:%S")]  \)1" | tee -a "$LOG_FILE"
}

# ── Section Header Helper ────────────────────────────────────────
# $1 = section title passed as argument
print_section() {
    {
        echo ""
        echo "============================================"
        echo "  $1"
        echo "============================================"
    } >> "$REPORT_FILE"
}

# ── Report Header ────────────────────────────────────────────────
log "Starting AWS resource report..."

# > creates fresh file (overwrites previous run if same day)
{
    echo "============================================"
    echo "   AWS DAILY RESOURCE USAGE REPORT"
    echo "   Date   : $DATE"
    echo "   Time   : $TIME"
    echo "   Region : $REGION"
    echo "============================================"
} > "$REPORT_FILE"

# ── Section 1: EC2 Instances ─────────────────────────────────────
log "Collecting EC2 data..."
print_section "EC2 INSTANCES"

aws ec2 describe-instances \
    --region "$REGION" \
    --query 'Reservations[*].Instances[*].[
        InstanceId,
        InstanceType,
        State.Name,
        PublicIpAddress,
        Tags[?Key==`Name`].Value|[0]
    ]' \
    --output table >> "$REPORT_FILE" 2>&1

# Count instances separately (avoids polluting table output)
TOTAL_EC2=$(aws ec2 describe-instances \
    --region "$REGION" \
    --query 'Reservations[*].Instances[*].InstanceId' \
    --output text 2>/dev/null | wc -w)
echo "Total EC2 Instances: \(TOTAL_EC2" >> "\)REPORT_FILE"

# ── Section 2: S3 Buckets ────────────────────────────────────────
log "Collecting S3 data..."
print_section "S3 BUCKETS"

aws s3 ls >> "$REPORT_FILE" 2>&1
TOTAL_S3=$(aws s3 ls 2>/dev/null | wc -l)
echo "Total S3 Buckets: \(TOTAL_S3" >> "\)REPORT_FILE"

# ── Section 3: IAM Users ─────────────────────────────────────────
log "Collecting IAM data..."
print_section "IAM USERS"

aws iam list-users \
    --query 'Users[*].[UserName, CreateDate, PasswordLastUsed]' \
    --output table >> "$REPORT_FILE" 2>&1

TOTAL_IAM=$(aws iam list-users \
    --query 'Users[*].UserName' \
    --output text 2>/dev/null | wc -w)
echo "Total IAM Users: \(TOTAL_IAM" >> "\)REPORT_FILE"

# ── Section 4: RDS Instances ─────────────────────────────────────
log "Collecting RDS data..."
print_section "RDS INSTANCES"

aws rds describe-db-instances \
    --region "$REGION" \
    --query 'DBInstances[*].[
        DBInstanceIdentifier,
        Engine,
        EngineVersion,
        DBInstanceStatus,
        DBInstanceClass
    ]' \
    --output table >> "$REPORT_FILE" 2>&1

TOTAL_RDS=$(aws rds describe-db-instances \
    --region "$REGION" \
    --query 'DBInstances[*].DBInstanceIdentifier' \
    --output text 2>/dev/null | wc -w)
echo "Total RDS Instances: \(TOTAL_RDS" >> "\)REPORT_FILE"

# ── Section 5: Elastic IPs ───────────────────────────────────────
log "Collecting Elastic IP data..."
print_section "ELASTIC IPs"

aws ec2 describe-addresses \
    --region "$REGION" \
    --query 'Addresses[*].[PublicIp, InstanceId, AllocationId, AssociationId]' \
    --output table >> "$REPORT_FILE" 2>&1

# ── Section 6: Security Groups ───────────────────────────────────
log "Collecting Security Group data..."
print_section "SECURITY GROUPS"

aws ec2 describe-security-groups \
    --region "$REGION" \
    --query 'SecurityGroups[*].[GroupName, GroupId, VpcId, Description]' \
    --output table >> "$REPORT_FILE" 2>&1

# ── Summary ──────────────────────────────────────────────────────
print_section "RESOURCE SUMMARY"

{
    echo "Resource               | Count"
    echo "----------------------------"
    echo "EC2 Instances          | $TOTAL_EC2"
    echo "S3 Buckets             | $TOTAL_S3"
    echo "IAM Users              | $TOTAL_IAM"
    echo "RDS Instances          | $TOTAL_RDS"
    echo "----------------------------"
    echo "Report generated at    : $TIME"
    echo "Report saved to        : $REPORT_FILE"
    echo "============================================"
} >> "$REPORT_FILE"

# ── Cleanup: Delete Reports Older Than 30 Days ───────────────────
find "$REPORT_DIR" -name "aws_report_*.txt" -mtime +30 -delete
log "Cleanup complete. Reports older than 30 days removed."

# ── Print to Terminal and Finish ─────────────────────────────────
cat "$REPORT_FILE"
log "Report generation complete. File: $REPORT_FILE"

Deploying to EC2 — Step by Step

Step 1 — Save the Script Locally

Save it as aws_resource_report.sh on your machine.

Step 2 — Upload to EC2 via SCP

scp -i /path/to/key.pem aws_resource_report.sh ubuntu@YOUR_EC2_IP:/home/ubuntu/

Command breakdown:

Part	Meaning
`scp`	Secure Copy — transfers files over SSH
`-i /path/to/key.pem`	Identity file (your private key)
`aws_resource_report.sh`	Source file on your local machine
`ubuntu@YOUR_EC2_IP`	Destination user and host
`:/home/ubuntu/`	Destination path on EC2

Verify upload was successful:

# SSH in and confirm the file is there
ssh -i /path/to/key.pem ubuntu@YOUR_EC2_IP
ls -lh /home/ubuntu/aws_resource_report.sh

Step 3 — Give Execute Permission

chmod +x /home/ubuntu/aws_resource_report.sh

Uploaded files have no execute permission by default. chmod +x grants it.

Verify:

ls -l /home/ubuntu/aws_resource_report.sh
# Should show: -rwxr-xr-x
# The 'x' bits confirm it is executable

Step 4 — Test Run Manually

Always test before scheduling. Never set up a cron job for something you have not verified manually.

bash /home/ubuntu/aws_resource_report.sh

Check the exit code:

echo "Exit status: $?"
# 0 = success
# Non-zero = something failed

Confirm the report was saved:

ls -lh /home/ubuntu/aws-reports/
cat /home/ubuntu/aws-reports/aws_report_$(date +%Y-%m-%d).txt

Check the log:

cat /home/ubuntu/aws-reports/script.log

Step 5 — Schedule with Cron

crontab -e

Add this line at the bottom:

0 20 * * * /bin/bash /home/ubuntu/aws_resource_report.sh >> /home/ubuntu/aws-reports/cron.log 2>&1

Cron syntax explained:

  ┌────── minute     (0)   = at minute :00
  │  ┌─── hour       (20)  = 8 PM (20:00)
  │  │  ┌── day       (*)  = every day
  │  │  │  ┌─ month   (*)  = every month
  │  │  │  │  ┌ weekday (*) = every day of the week
  │  │  │  │  │
  0  20  *  *  *    /bin/bash /home/ubuntu/aws_resource_report.sh

The >> cron.log 2>&1 at the end captures both normal output and errors from the cron execution itself — separate from the script's own log file.

Verify cron is registered:

crontab -l
# Should display your scheduled line

Check server timezone (important for time accuracy):

timedatectl
# Look for "Time zone" — make sure it matches your intended 8 PM

Step 6 — Confirm Cron Ran (Next Day Verification)

After 8 PM passes:

# Check cron log
cat /home/ubuntu/aws-reports/cron.log

# Check script log
cat /home/ubuntu/aws-reports/script.log

# List all reports generated
ls -lh /home/ubuntu/aws-reports/aws_report_*.txt

What the Output Looks Like

============================================
   AWS DAILY RESOURCE USAGE REPORT
   Date   : 2026-05-03
   Time   : 20:00:04
   Region : us-east-1
============================================

============================================
  EC2 INSTANCES
============================================
+------------------+----------+---------+---------------+----------+
| InstanceId       | Type     | State   | Public IP     | Name     |
+------------------+----------+---------+---------------+----------+
| i-0abc123def456  | t2.micro | running | 54.210.195.72 | web-server|
+------------------+----------+---------+---------------+----------+
Total EC2 Instances: 1

============================================
  S3 BUCKETS
============================================
2026-01-10 10:32:00 my-app-bucket
2026-03-22 09:14:11 my-logs-bucket
Total S3 Buckets: 2

============================================
  IAM USERS
============================================
+------------------+----------------------+----------------------+
| UserName         | CreateDate           | PasswordLastUsed     |
+------------------+----------------------+----------------------+
| admin-user       | 2025-11-01T08:00:00Z | 2026-05-02T14:22:00Z |
| deploy-bot       | 2026-01-15T10:00:00Z | None                 |
+------------------+----------------------+----------------------+
Total IAM Users: 2

============================================
  RESOURCE SUMMARY
============================================
Resource               | Count
----------------------------
EC2 Instances          | 1
S3 Buckets             | 2
IAM Users              | 2
RDS Instances          | 0
----------------------------
Report generated at    : 20:00:04
Report saved to        : /home/ubuntu/aws-reports/aws_report_2026-05-03.txt
============================================

Production Best Practices — Explained

`set -euo pipefail` — The Full Safety Combination

Most scripts use only set -e. Here is why all three flags matter together:

# Without -u:
BUCKET_NAME=""
aws s3 ls $BUCKET_NAME   # expands to: aws s3 ls  (lists ALL buckets — not intended)

# With -u:
# Script exits with: "unbound variable" error — catches the bug early

# Without -o pipefail:
aws ec2 describe-instances | grep "running" | wc -l
# If aws command fails, grep still runs on empty input
# wc -l returns 0, script continues, report shows wrong data

# With -o pipefail:
# Pipe failure is caught. Script exits. You know something broke.

IAM Role — Production-Correct Authentication

Method	Risk Level	Appropriate For
`aws configure` with Access Keys stored on EC2	⚠️ High	Local development only
IAM Role attached to EC2 instance	✅ Safe	All production workloads

An IAM Role grants permissions directly to the EC2 instance through the AWS metadata service (169.254.169.254). No keys are stored anywhere on disk. No keys can be stolen. The AWS CLI automatically detects and uses the role.

Auto-Cleanup

find "$REPORT_DIR" -name "aws_report_*.txt" -mtime +30 -delete

Part	Meaning
`find "$REPORT_DIR"`	Search in the report directory
`-name "aws_report_*.txt"`	Match only report files (not logs)
`-mtime +30`	Files older than 30 days
`-delete`	Delete them

Without this, 365 daily reports accumulate per year. EC2 root volumes are typically 8–20 GB. This is how disk fills up silently and breaks things at 3 AM.

Logging with Timestamps

log() {
    echo "[\((date +"%Y-%m-%d %H:%M:%S")]  \)1" | tee -a "$LOG_FILE"
}

Every log call produces a timestamped line:

[2026-05-03 20:00:01]  Starting AWS resource report...
[2026-05-03 20:00:02]  Collecting EC2 data...
[2026-05-03 20:00:03]  Collecting S3 data...
[2026-05-03 20:00:04]  Report generation complete.

This is the first thing you check when a cron job produces unexpected output. Without it, debugging an automated script is guesswork.

Troubleshooting Reference

Symptom	Root Cause	Resolution
`Permission denied` when running script	No execute bit set	`chmod +x script.sh`
`aws: command not found`	AWS CLI not installed on EC2	`sudo apt install awscli -y`
`Unable to locate credentials`	No IAM Role and no `aws configure`	Attach IAM Role to EC2 or run `aws configure`
Empty sections in report	AWS API call failing silently	Temporarily remove `2>/dev/null` and re-run
Cron runs but report is wrong time	Server timezone mismatch	Check with `timedatectl`, adjust cron hour
`unbound variable` error	`set -u` catching undefined variable	Check all variables are set before use
Old reports not being deleted	`find` path or pattern mismatch	Run `find` command manually to debug
`cron.log` shows no output	Cron not using full binary path	Use absolute path: `/bin/bash /full/path/script.sh`

Summary

## AWS Daily Resource Reporter

A production-grade Bash script deployed on AWS EC2 that automatically
generates structured daily reports of AWS resource usage.

### What It Covers
- EC2 instances (ID, type, state, IP, name)
- S3 buckets (name, creation date)
- IAM users (username, creation date, last login)
- RDS databases (identifier, engine, version, status, class)
- Elastic IPs (allocation, association)
- Security Groups (name, ID, VPC, description)

### How It Works
Runs daily at 8 PM via Linux cron. Saves formatted output to
/home/ubuntu/aws-reports/ with date-stamped filenames.
Auto-deletes reports older than 30 days.

### Security
Uses IAM Role attached to EC2 — no stored access keys.

### Stack
Bash · AWS CLI v2 · Ubuntu 22.04 · EC2 · IAM · Cron

Final Thought

The value of a script like this is not the output it produces today. It is the habit it represents.

Automation only earns its keep when it runs without human involvement — correctly, quietly, and reliably. That means understanding why each line exists, handling errors deliberately, following security practices from the start, and building systems that clean up after themselves.

A Bash script that does all of that is not a small project. It is a foundation.

Found this useful? Drop a comment or share it with someone building on AWS. Questions or corrections — the comments are open.

Tags: #DevOps #AWS #BashScripting #Linux #Automation #EC2 #CloudEngineering #IAM #ShellScript #ProductionReady

Automate AWS Resource Reporting with a Production-Grade Bash Script — End to End

Why This Article Exists

What You Will Learn

Architecture — Design Before You Code

Prerequisites

Bash Foundations — What This Script Uses

Variables

File Redirection

Error Stream Capture

Logging with `tee`

`set -euo pipefail` — The Safety Net

The Full Production Script

Deploying to EC2 — Step by Step

Step 1 — Save the Script Locally

Step 2 — Upload to EC2 via SCP

Step 3 — Give Execute Permission

Step 4 — Test Run Manually

Step 5 — Schedule with Cron

Step 6 — Confirm Cron Ran (Next Day Verification)

What the Output Looks Like

Production Best Practices — Explained

`set -euo pipefail` — The Full Safety Combination

IAM Role — Production-Correct Authentication

Auto-Cleanup

Logging with Timestamps

Troubleshooting Reference

Summary

Final Thought

Comments

Devops End to End Projects

Command Palette

Why This Article Exists

What You Will Learn

Architecture — Design Before You Code

Prerequisites

Bash Foundations — What This Script Uses

Variables

File Redirection

Error Stream Capture

Logging with tee

set -euo pipefail — The Safety Net

The Full Production Script

Deploying to EC2 — Step by Step

Step 1 — Save the Script Locally

Step 2 — Upload to EC2 via SCP

Step 3 — Give Execute Permission

Step 4 — Test Run Manually

Step 5 — Schedule with Cron

Step 6 — Confirm Cron Ran (Next Day Verification)

What the Output Looks Like

Production Best Practices — Explained

set -euo pipefail — The Full Safety Combination

IAM Role — Production-Correct Authentication

Auto-Cleanup

Logging with Timestamps

Troubleshooting Reference

Summary

Final Thought

Comments

Devops End to End Projects

Logging with `tee`

`set -euo pipefail` — The Safety Net

`set -euo pipefail` — The Full Safety Combination