There's a new term circulating among IT directors at small businesses: Token Shock. It refers to the bill that arrives after deploying an AI feature, running a few automated workflows, or integrating a third-party AI tool into your stack, and discovering that consumption-based pricing doesn't behave like anything you've budgeted for before.
According to Techaisle's 2026 SMB research, budget constraints and cost predictability returned to the top IT challenge for SMBs this year. The anxiety has shifted. It's not just about controlling general cloud spend anymore. It's about the unpredictability of AI consumption-based pricing layered on top of existing cloud costs that were already hard to forecast.
Flexera's 2025 State of the Cloud Report found organisations waste an average of 27% of their cloud budget on idle resources, unused licences, and overlapping subscriptions. Fix that, and you've created room for the AI workloads that actually drive value. Don't fix it, and you'll be over budget before you've deployed anything meaningful.
This isn't a listicle of generic tips. It's a framework for managing cloud costs in an environment where traditional forecasting models are breaking down.
Why Traditional Cloud Forecasting Is Failing
For most of the past decade, cloud budgeting was straightforward enough. You counted your instances, estimated your storage growth, factored in some buffer, and submitted a number. Actual spend usually tracked reasonably close to forecast.
AI workloads don't behave that way. CloudZero's analysis of enterprise cloud spending shows that AI and ML workloads now account for 22% of cloud costs on average, and they're harder to forecast than any other workload category. They scale non-linearly with usage. A feature that costs $200/month in testing can cost $8,000/month in production if adoption exceeds expectations. A vector database that seemed cheap at 10,000 queries per day becomes expensive at 2 million.
Sixty-eight per cent of SMBs anticipate increased cloud spending in 2026, according to IDC, with a growing shift toward multi-cloud strategies. More cloud providers means more billing systems, more pricing models, and more opportunities for costs to accumulate unnoticed.
The 27% You're Already Wasting
Before worrying about AI cost management, address the baseline waste. Flexera's figure of 27% wasted cloud budget is consistent across organisation sizes and cloud providers. The composition is usually:
Idle compute resources. Virtual machines that are running but not doing useful work. Development and test environments left on at weekends. Autoscaling groups that scale up but never back down.
Orphaned storage. Snapshots taken for projects that finished. Data volumes detached from instances that were terminated. Log files accumulating in storage buckets with no lifecycle policy.
Unused licences. SaaS products billed per seat where half the seats belong to people who left. Database licences for environments that no longer exist. Support tiers on services you rarely need to contact.
Over-provisioned databases. Database instances sized for peak load that occurs twice a year. Read replicas deployed for redundancy that was never actually needed.
The audit process is simple, if time-consuming. Go through your cloud billing dashboard line by line. For each resource, ask: is this running? Is anything using it? If the answer to either is no, terminate it.
Right-Sizing: The Uncomfortable Conversation
Right-sizing means matching your cloud resources to your actual workload requirements. It's uncomfortable because it requires admitting that resources were over-provisioned in the first place, usually by whoever set up the environment.
AWS, Azure, and Google Cloud all provide right-sizing recommendations in their cost management tools. These recommendations are based on actual utilisation data from your environment. They're free. Most businesses ignore them.
A practical rule of thumb: if a compute resource is running below 40% CPU utilisation on average over 30 days, it's a candidate for downsizing. If it's below 20%, it's almost certainly over-provisioned.
The fear that holds people back is the downside risk. What if we downsize and it can't handle a load spike? The answer is autoscaling. Configure your infrastructure to scale automatically when utilisation exceeds a threshold, rather than statically provisioning for the worst case.
Reserved vs On-Demand: A Simple Decision Framework
On-demand pricing is the most expensive way to run a cloud workload you know you'll need long-term. It's the right choice for unpredictable workloads, temporary projects, and environments you're still evaluating. It's the wrong choice for anything that's been running continuously for six months.
Reserved instances and savings plans offer discounts of 30 to 60% in exchange for a 1 or 3-year commitment. The maths are straightforward: if you know a workload will run for at least 12 months, the breakeven point on a 1-year reserved instance is typically around 5 to 6 months.
For AI workloads specifically, the calculus is harder because usage patterns are less predictable. Start with on-demand, monitor for 90 days, and commit once you have a stable baseline. Don't lock in reserved capacity for workloads that are still experimental.
FinOps for SMBs: A Discipline, Not a Team
FinOps (Financial Operations for cloud) is often presented as something that requires a dedicated team and specialised tooling. At enterprise scale, it does. For an SMB, FinOps is a monthly meeting and a dashboard.
Here's what a functional FinOps practice looks like for a 50-person business:
Monthly cloud cost review. One hour per month. Someone who owns cloud costs sits down with the billing dashboard, reviews the previous month's spend against budget, identifies any anomalies, and checks the right-sizing recommendations. That's it.
Tagging. Every cloud resource should be tagged with at minimum: the project or application it belongs to, the environment (production, development, test), and the team or person responsible. Without tags, your billing data is a single number with no context. With tags, you can attribute costs, identify the expensive projects, and have informed conversations about whether they're worth it.
Budget alerts. Set alerts at 80% and 100% of your expected monthly spend. This gives you advance warning before you exceed budget rather than a surprise at month end.
Anomaly detection. Amnic's FinOps data shows 48% of FinOps teams adopted AI-driven anomaly detection tools in 2025. For SMBs, the built-in anomaly detection in AWS Cost Explorer and Azure Cost Management is sufficient. Enable it. It will alert you when spending deviates significantly from historical patterns, which is usually the first sign that something has been misconfigured or left running accidentally.
Organisations that invest in long-term cloud optimisation programmes achieve 20 to 40% overall savings, according to Safebox Tech analysis. You don't need a dedicated team to capture most of that. You need a consistent monthly practice.
Managing AI Workload Costs Specifically
If you're deploying AI features, whether through API calls to OpenAI or Anthropic, running inference on your own cloud infrastructure, or using AI-enabled SaaS tools with consumption-based pricing, the standard cloud cost management approach needs adjustment.
A few practices that help:
Model caching. For LLM API calls, caching responses to common queries can dramatically reduce token consumption. If your application calls an LLM with the same context repeatedly, prompt caching (available in most major LLM APIs) can cut costs by 50 to 90% for repeated queries.
Batch processing over real-time where acceptable. Real-time inference is more expensive than batch processing for workloads where immediate responses aren't required. Document processing, data enrichment, and report generation are often fine to run as scheduled batch jobs.
Model tiering. Using a large, expensive model for every query is like using a sledgehammer for every nail. For simple classification tasks, routing to smaller, cheaper models reduces costs significantly without affecting quality for those specific use cases.
Hard spending limits. Set spending limits on AI API accounts. This won't save you from a genuinely over-budget workload, but it will protect you from runaway costs caused by bugs, infinite loops, or misconfigurations.
The Monthly Review Cadence
For SMBs without a dedicated FinOps hire, here's a practical monthly review agenda:
- Review last month's total cloud spend against budget. Identify any categories that came in significantly over or under.
- Check right-sizing recommendations in your cloud provider's dashboard. Action any recommendations where the potential saving exceeds $50/month.
- Review any new AI/ML spend. Understand what drove it and whether it tracked to your forecast.
- Check for orphaned resources: unattached storage volumes, unused reserved IPs, stopped instances that have been stopped for more than 30 days.
- Review active SaaS licences. Check utilisation data if available. Flag any seats belonging to departed employees.
This takes 60 to 90 minutes and, for most businesses, surfaces $500 to $2,000 in monthly waste in the first few sessions.
One Number to Watch
Beyond your total cloud bill, track cloud cost as a percentage of revenue. Organisations spend an average of 10% of revenues on cloud services, according to CloudZero. For most SMBs, anything above 12 to 15% warrants a detailed review. Anything below 5% might indicate under-investment in infrastructure that will create problems as you scale.
The ratio matters more than the absolute number because it adjusts automatically as your business grows. A rising ratio in a flat-revenue period is a warning sign. A rising ratio alongside strong revenue growth is usually fine.
