Glue Cost Insights
A Glue crawler scheduled hourly runs about 720 times a month whether the data it crawls changed or not — and every run bills DPU time. CloudPouch Glue Cost Insights compare each crawler’s schedule against what it actually found, so you can see which schedules are paying for discovery that never discovers anything.
What CloudPouch checks
Section titled “What CloudPouch checks”- Crawler execution frequency against results: last catalog change date, last run, and runs this month, side by side.
- Direct and indirect schedules — including crawlers triggered through Glue workflows, which are easy to miss when auditing schedules by hand.
- Glue job configuration improvement opportunities.
- Failed Glue jobs and their cost impact, where supported.
Why this costs money
Section titled “Why this costs money”AWS Glue bills crawlers and jobs per DPU-hour, metered per second with a per-run minimum. Crawler cost is therefore schedule × runtime: a crawler set to hourly “to be safe” during development keeps that schedule in production forever, and each run costs the same whether it finds a new partition or nothing at all. Failed jobs are the other leak — a job that runs for 40 minutes and then fails still bills 40 minutes of DPU time, and jobs on retry loops pay that repeatedly. Rates are on the AWS Glue pricing page.
The catalog-change comparison is the useful signal here. A crawler with 300 runs this month and a last catalog change from four months ago is measurably wasting its schedule; switching it to daily, or to event-driven triggering on S3 changes, keeps discovery working at a fraction of the runs.
Reading failed-job costs
Section titled “Reading failed-job costs”Treat failed-job cost figures as investigation context, not accounting. Verify current AWS Glue pricing coverage for your account and region before using failed-job estimates in financial commitments — the point of the number is to tell you which failing job to fix first.
When to use it
Section titled “When to use it”Crawler schedules set once and never revisited are the classic trigger. This insight also pays off when Glue is a growing line in the bill, or when ETL failures are frequent enough that their cost is worth quantifying.
Required AWS permissions
Section titled “Required AWS permissions”CloudPouch needs read-only access to AWS Glue crawler, job, and workflow metadata, CloudWatch metrics where used, and cost data. See AWS permissions.