Infra Audit5 min readUpdated November 4, 2026

A Backup Is Not a Backup Until It Restores

We found our own backup cron running daily while its output directories were empty. The schedule looked perfect. The directories told the truth.

Verify backups by listing output files and restoring the newest one to a scratch location. A cron line that looks correct proves nothing; the restore is the only test that counts.

Backup jobs are set up once and trusted forever. The cron line is correct, the script exists, the schedule fires. Inside the script, a path changed or a credential expired, and the job started producing nothing. Nothing alerts, because the job did not fail: it succeeded at producing empty output.

See How We Work

In This Guide

The failure mode nobody watches

Backup jobs are set up once and trusted forever. The cron line is correct, the script exists, the schedule fires. Inside the script, a path changed or a credential expired, and the job started producing nothing. Nothing alerts, because the job did not fail: it succeeded at producing empty output.
Backup jobs are set up once and trusted forever. The cron line is correct, the script exists, the schedule fires. Inside the script, a path changed or a credential expired, and the job started producing nothing. Nothing alerts, because the job did not fail: it succeeded at producing empty output. The audit's rule: never read the schedule, always read the destination.

The restore test, step by step

Pick the newest backup file. Copy it to a scratch location, never production. Restore it: for a database, into a throwaway database; for files, into a temp directory. Count rows or files and compare against expectations. Record the result with a date.
Pick the newest backup file. Copy it to a scratch location, never production. Restore it: for a database, into a throwaway database; for files, into a temp directory. Count rows or files and compare against expectations. Record the result with a date. A restore test that is not written down did not happen, because next month nobody remembers whether it passed.

If it stays untested

The backup becomes a belief, not a capability. The day the disk dies or the ransomware lands, the team discovers the truth at the worst possible moment. The exposure window is the entire period since the last successful restore test, and for most teams that period is unbounded.
The backup becomes a belief, not a capability. The day the disk dies or the ransomware lands, the team discovers the truth at the worst possible moment. The exposure window is the entire period since the last successful restore test, and for most teams that period is unbounded.

Get the skill

The checks in this guide come from the roqy audit skill, open source at https://github.com/roncoo-2/roqy-audit. It runs read-only against your own account, produces findings with verdicts (confirmed, needs_validation, rejected), and was validated on a live canary AWS account where 9 planted misconfigurations were detected 9 out of 9 times.
The checks in this guide come from the roqy audit skill, open source at https://github.com/roncoo-2/roqy-audit. It runs read-only against your own account, produces findings with verdicts (confirmed, needs_validation, rejected), and was validated on a live canary AWS account where 9 planted misconfigurations were detected 9 out of 9 times. The full canary report is in the repository under docs/runs. Copy the skill into any coding agent, point it at your stack, and get the same report for your account.

4 Tips to Get Started

1

Check the backup destination by content: file count, newest file date, size trend.

2

Restore to a scratch location, never over production. Count rows or files after restore.

3

A config step inside the backup script can silently fail while the cron reports success.

4

Run the restore test monthly. Backup rot is silent until the outage.

Ready to Stop Worrying About Infra?

Knowledge is the first step. Get a free 1-hour diagnostic and see where your infrastructure leaks time, money, and focus.

Related Guides

Is My S3 Bucket Public? How to Check in One Command

One command tells you if your S3 bucket is public, and one curl proves it. The policy check shows intent, the anonymous request shows reality.

The Most Common AWS Misconfiguration (It Is One Line of Config)

Public buckets and Principal * policies cause most disclosed AWS incidents. Each is one line of config with account-wide blast radius. Here is the pattern.

Terraform State File Leak: Every Secret in Plaintext

terraform.tfstate stores database passwords, IAM keys and TLS private keys in plaintext. sensitive=true hides nothing. Here is how leaks happen and the check.

Docker Socket Mounted in a Container: Root on the Host, No Exploit Needed

A container with /var/run/docker.sock mounted controls the host daemon and can mount your entire filesystem. No kernel exploit required. Here is the mechanism and the check.

R

Ron Cohen — DevOps engineer & founder, roqy.io

Content reviewed by the roqy.io team. Our guides are based on publicly available data.

Disclaimer: This content is for informational and educational purposes only. It does not constitute professional advice or a guarantee of specific results. Infrastructure needs vary by team, stack, and stage. Consult a qualified engineer for advice tailored to your situation.

Frequently Asked Questions

How do I verify my backups actually work?

Two checks. First, list the backup destination: file count, newest file timestamp, size trend. An empty directory behind a healthy-looking cron is a common P0. Second, restore the newest backup to a scratch location and count what comes back. If the restore fails or the counts are wrong, the backup does not exist in any meaningful sense.

Why did our backup cron run but write nothing?

A step inside the script failed silently: a config change, a moved path, a permission. The cron exits and the schedule looks perfect. This is why the audit verifies backups by listing output files, never by reading the schedule. On our own stack, the backup directories were empty for weeks while the cron reported success daily.

How often should a restore test run?

Monthly for critical systems, and after every backup configuration change. The test is cheap: restore to scratch, count, delete. The cost of skipping it is discovering the answer during the outage, when the backup is the only thing that matters.

Get the DevOps Ecosystem Behind You

One accountable engineer. Full platform behind him. Month-to-month, no lock-in.

See How It Works