Planning, Tracking, and Controlling Costs
The place to find account-level financial information is the Billing and Cost Management Dashboard, which is accessed via the My Billing Dashboard link in the account drop-down menu at the top of the console. That’s where you can view past bills, manage credits, and enter your tax registration information. The page also includes a quick visual overview of the current month’s cost activities. Right now, however, you’re more interested in how the Billing and Cost Management
Dashboard also links you to tools you’ll use to monitor and control your AWS spending. The precise function of these tools—and even the names AWS gives them—will probably change over time. But their overall goal is clear: to provide deep insight into what your running resources are costing you and how you can closely track them.
AWS Budgets
The Preferences link takes you to a simple page where you choose the way you’d like to consume billing information. You can tell AWS to send ongoing billing reports to an S3 bucket you specify. You can also have AWS email you billing alerts created through either Amazon CloudWatch Alarms (as you saw in Chapter 7, “CloudTrail, CloudWatch, and AWS Config”) or the newer AWS Budgets. Budgets are meant to track your ongoing resource-related usage and costs and alert
you if projected usage levels fall outside of a predefined threshold. You could, for instance, have an email sent alerting you when the total quarterly costs of transferring S3-based data between regions threatens to rise above a certain dollar amount. Or you could create a budget to tell you when the total volume of outbound data transferred to the Internet from On Demand EC2 instances running in a specified AWS region exceeds, say, 100GB.
You can create and apply cost allocation tags to your resources and use them in your budgets as filters. This will let you limit your alerts to only those resources belonging to a particular class. You could use this kind of filtering to track only your staging or development environments, but not production.
You can activate and manage tags from the Cost Allocation Tags page, which is accessed by clicking the Cost Allocation Tags link in Billing. User-defined tags can be created and managed using the Tag Editor accessed through the Resource Groups drop-down menu in the AWS Console. You use that page to find active resources such as EC2 instances and S3 buckets and then create and/or edit tags for those resources. Note the following:
■ Tags can take up to 24 hours before they appear in the Billing and Cost Management Dashboard.
■ Tags can’t be applied to resources that were launched before the tags were created.
You’re allowed two free budgets per account. Subsequent tags will cost around $0.02 each per month.
You configure a budget by defining whether you want to track resource cost, usage, or—to be sure you’re getting optimal value from your EC2 reserve instances—reserve instance utilization or reserve instance coverage. You can narrow the scope of a budget by filtering it using parameters such as services, regions, tags, and usage type. Notification triggers are defined when thresholds are met (for instance, “Forecasted usage is greater than 80 percent of the budgeted amount”) and are sent to Simple Notification Service (SNS) or email addresses.
Monitoring Tools
You’ve seen how to set limits to control your AWS spending using budget alerts. But the busier and more complicated your resource stacks get, the more you’ll need to dive deeply into spending patterns so you can understand and, if necessary, adjust developing trends. Cost Explorer and Reports can help with that.
Cost Explorer
Clicking Cost Explorer from the Billing and Cost Management Dashboard, and then on Launch Cost Explorer, takes you to a detailed visualization of your historical AWS usage and costs. The default view covers the last six months and, by clicking the Explore costs link, can be filtered by a prominent Usage Type Group metric (like EC2: Running Hours). The more you have running, you more you’ll want to drill down into the weeds of the data. You can adjust the view, filtering by things such as grouping or service, or download the data as a CSV file. Clicking New Report lets you open a report template, including Reserve Instance (RI) Utilization or RI Coverage—relevant, obviously, only if you’ve purchased reserve instances.
AWS Cost and Usage Reports
At first glance, the content delivered by Cost and Usage Reports seems to overlap a bit with Cost Explorer. After all, they both provide data covering your billing costs, rates, and product and pricing attributes. However, the unique value proposition of Reports is hinted to in the Select Content page of the Report creation process. There, you can choose to enable support for either Redshift or QuickSight. Redshift, as you know, is a serious tool for handling serious amounts of data. And Amazon QuickSight is a pay-per-session tool for squeezing business intelligence out of data stores. So, AWS Reports are meant for accounts that are so busy you struggle to keep track of many moving parts. Reports are optimized for this kind of analytics through their normalized data model that organizes discrete cost components in individual columns. Reports are configured to be regularly written to an S3 bucket in your account for which you’ve previously added appropriate permissions. Sample bucket policies are available from the Create Report dialog. The Report configuration process replaces a few deprecated (and soon to be removed) “other” reports.
AWS Organizations
You can consolidate the management of multiple AWS accounts using AWS Organizations. Why bother? Because it’s not uncommon for larger organizations to have so many projects and resources running on AWS that they need some degree of resource isolation to keep a handle on them. It can also be hugely convenient to maintain some level of bird’s-eye-view supervision over the whole thing—and to channel billing through a single payer. AWS Organizations lets you create new accounts and invite existing accounts into an organization. Once your accounts are connected, the organization administration can create global access policies much the same way as you might for individual users within a single account. Account security for an organization is even more critical than for a single account since a breach could potentially impact all the resources in all member accounts. You should therefore be especially careful to enforce all the normal security best practices, including multifactor authorization (MFA), strong passwords, and secured root accounts. AWS Organizations has replaced the old and deprecated Consolidated Billing tool.
AWS Trusted Advisor
You can get a quick view of how closely your account configurations are currently following AWS best practices from the Trusted Advisor Dashboard (accessed from the main AWS service menu). The compliance and health data is divided into five categories.
■ Cost Optimization checks for underutilized or running but idle resources that might be costing you money.
■ Performance checks for configuration mismatches—such as excessive use of EBS magnetic volumes—that might unnecessarily impact performance.
■ Security checks for potentially vulnerable configurations, such as wide open S3 bucket permissions or security groups.
■ Fault Tolerance checks for appropriate replication and redundancy configurations to ensure that a single service failure would be unlikely to cause you a catastrophic loss.
■ Service Limits checks for your usage of AWS services to make sure you’re not approaching default limits.
The results of each check are color coded to tell you whether you’re compliant and are
accompanied by a helpful description of the issue. Sometimes, for an S3 bucket-hosted static website that needs open permissions, for instance, you’ll want to ignore the warning. But it’s nevertheless important to regularly check in with Trusted Advisor to see whether it’s caught anything new. The first thing you’ll notice when you visit Trusted Advisor is that most of the metrics are disabled unless you upgrade your support—to either the Business or Enterprise plan.
Also read this topic: Introduction to Cloud Computing and AWS -1
Online Calculator Tools
One powerful way to model the cost of multiple resource stacks is by gathering comprehensive pricing information and using it to calculate the price tag for each element of a stack. However, when you consider just how many permutations and combinations of services and price levels can coexist within a single stack—and how often those prices change—then accurate cost modeling can feel impossibly complicated. Fortunately, AWS provides two regularly updated calculators that do all the number crunching for you: the Simple Monthly Calculator and the AWS TCO Calculator.
Simple Monthly Calculator
The Simple Monthly Calculator (https://calculator.s3.amazonaws.com/index.html) lets you select service resources at a fine detail (three EC2 instances built on a Linux m4.large instance type with a 500 GB Provisioned IOPS SSD EBS volume set for 1,000 IOPS in the US East N. Virginia region, for example). You can also enter the portion of a month you expect your instance to actually be running. You should be aware that costs can differ between AWS regions, so make sure you price resources within each of the regions that would, practically, work for your application. The same applies to different EBS and RDS storage types: explore pricing for SSD, Provisioned IOPS, and other storage types. When you’re done entering the stack resources from every relevant service your project
will use, the estimated monthly bill will appear in the tab at the top of the page. Clicking that tab will provide a fully itemized breakdown of the monthly cost of your stack, which can be downloaded in CSV format. Saving your estimate will generate a URL that can be shared with colleagues or clients.
AWS Total Cost of Ownership Calculator
The “other” AWS calculator (https://aws.amazon.com/tco-calculator/) estimates thetotal cost of ownership (TCO) to help you make an apples-to-apples comparison of what a complex workload would cost on-premises as opposed to what it would cost on the AWS cloud. The interface asks you to define the infrastructure you’d need to run your application locally. You’ll enter the number and size of the servers, whether they’re physical or virtual, which virtualization hypervisor you’ll be using (if applicable), and the total storage capacity you’ll need. When you run the calculation you’ll be shown a highly detailed itemized cost sheet calculated over 3 years. The output will include estimates of on-premises networking and hardware acquisition costs along with AWS support costs, and a detailed explanation of the calculator’s methodology and assumptions.
Cost-Optimizing Compute
Perhaps more than anything else, the virtualization revolution makes greatly increased server density possible. That is, you can run more compute workloads sharing the physical assets of a single server than ever before. And of course, getting “more production out of a server” is just another way of saying “getting more value for your investment.” Since cloud computing platforms like AWS live and breathe virtualization, you might assume that following AWS best practices can help you get the most out of your assets. And you’d be correct. Here’s where I’ll spend some time talking about squeezing every drop of digital goodness
out of your AWS infrastructure. You’ll explore some basic server density tricks such as selecting the EC2 instance type that’s the best match for your workload, and, although I introduced them in Chapter 2, “Amazon Elastic Compute Cloud and Amazon Elastic Block Store,” I’ll dive a bit deeper into effectively using reserved and spot instances.
Maximizing Server Density
You have a keen interest in getting the most out of a server instance—whether it’s for an application running on EC2 or a database with AWS Relational Database Service (RDS). Matching your workload to the right instance type can make a big difference not only in performance but also in overall cost.
The M5 instance type family, for instance, is optimized to take advantage of the higher core counts found on Intel Xeon Scalable processors. You can typically pack multiple resource-hungry workloads onto a single, high-end M5 instance (like the m5.24xlarge type), potentially requiring fewer instances overall. Similarly, the high-performing and parallel processing NVIDIA K80 GPU cores, enhanced networking, and default EBS optimization of a single P2 instance can often replace multiple instances of other families for high-demand analytics and machine learning workloads. The features unique to each EC2 instance type are detailed and compared at https://aws.amazon.com/ec2/instance-types/. AWS Lambda also provides a kind of server density: it may not be your server resources
you’re conserving by running Lambda functions, but you’re certainly getting a lot more value from your compute dollar by following the “serverless” model.
The flexibility potential of virtualization is nowhere more obvious than with container technologies like Docker. Whether you’re running Docker clusters on Amazon’s Elastic Container Service (ECS—see Chapter 2), running the Amazon Elastic Container Service for Kubernetes (Amazon EKS), or manually provisioning your own EC2 instances as Docker hosts, you’ll be able to tightly pack virtual applications or microservices into every corne of your host hardware.
EC2 Reserved Instances
You learned about purchasing reserved instances (RIs) for 24/7 compute needs lasting at least 12 months in Chapter 2. But spending a few extra minutes on some of the finer details will help you make smarter and more effective choices. The basics are straightforward: you can have AWS set aside lower-cost, long-term compute capacity for you by purchasing a reserved instance. (You’re not actually “purchasing” something called a reserved instance; instead, you’re just buying the right to use a regular EC2 instance.) You can also purchase an instance through the Amazon EC2 Reserved Instance Marketplace, where you effectively take over the time remaining on a reserved instance owned by other AWS users who have changed their usage plans. Just as you would when launching a normal EC2 instance, you select an RI that matches the configuration details you’re after. Therefore, you’ll search by tenancy (default or dedicated),
instance type, and platform (choosing from OSs including Linux, SUSE, RHEL, Microsoft Windows Server, and Microsoft SQL Server). However, exactly how much money you’ll save on your instance and how much flexibility it’ll give you depends on decisions you make up front. You can, for example, pay a bit more for a Convertible RI that will allow you to exchange your instance later as long as the new instance has equal or great value than the original. Convertible RIs provide
a discount of up to 54 percent less than on-demand. A Standard RI can save you up to 75 percent off on-demand, while, obviously, locking you in for the duration of the term to the instance profile you initially selected. Optionally, you can also specify a single availability zone or, for a cost premium,
maintain the flexibility to run your RI in any AZ within your selected region. You can also schedule an RI to launch within a specified recurring time window. You
purchase on a schedule from the Scheduled Instances link in the EC2 Dashboard (rather than the regular Reserved Instances link).
Finally, you can pay for your RI using any one of three payment options: All Upfront (the lowest price), Partial Upfront, and No Upfront (billed hourly).
EC2 Spot Instances
In Chapter 2, you also learned about EC2 spot instances—short-term “rentals” of EC2
instances that offer very low prices in exchange for volatile lifecycles and unpredictable shutdowns. As you just did with RIs, let’s look at the details. I’ll begin with a few important definitions.
■ Spot Price—The current going rate for spot instances of a given set of launch specifications (type, region, and profile values). Spot prices can rise or fall without warning, sometimes forcing a running instance to shut down.
■ Spot Instance Interruption—You can choose one of three ways for EC2 to react when your maximum price is exceeded by the spot price: Terminate (permanently delete all associated resources and volumes), Stop (possible only when you’re using an EBS-backed AMI), and Hibernate.
■ Spot Instance Pool—All the unused EC2 instances matching a particular set of launch specifications. Spot fleets are drawn from matching spot instance pools.
■ Spot Fleet—A group of spot instances (sometimes including on-demand instances) that is launched to meet a spot fleet request. A spot fleet can be made up of instances using multiple launch specifications drawn from multiple spot instance pools.
■ Request Type—When you configure your request from the Request Spot Instances page, you’re offered three request types: Request (a one-time instance request), Request And Maintain (to maintain target capacity using a fleet), and Reserve For Duration (request an uninterrupted instance for between one and six hours).
You can configure your request for spot instances from the EC2 Dashboard. You use the Request Spot Instances page to set details such as your total target capacity (the maximum instances or vCPUs you want running), the AMI and instance type you want launched, whether you want to incorporate load balancing, and any user data you want to pass to your instances at launch time. You can define your workload by using a private AMI you built for the task or by passing user data (the way you did for yourself in Exercise 10.1 from Chapter 10, “The Performance Efficiency Pillar”). The Spot Advisor, which is available from the Spot Instances Dashboard, can helpfully recommend sample configuration profiles for a few common tasks. Once you accept a recommendation, you can either customize it or send it right to the fleet configuration stage with a single click.
Elastic Block Store Lifecycle Manager
The EBS volumes attached to your EC2 instances are well protected by replication, but, like all technology, there’s no guarantee that nothing bad will ever happen. The best practice has always been to back up your volumes by making snapshots, but adding a new snapshot every few hours can, over time, add up to some intimidating storage costs. The obvious solution is to delete older snapshots, leaving only fresher copies online. Since, however, you’re not likely to get around to regularly doing that on your own, you’re better off automating. The EBS Lifecyle Manager (accessed through the EC2 Dashboard) lets you create a rotation policy where you set the frequency at which new snapshots will be created and the maximum number of snapshots to retain.
People also ask this Questions
- What is a defense in depth security strategy how is it implemented?
- What is AWS Solution Architect?
- What is the role of AWS Solution Architect?
- Is AWS Solution Architect easy?
- What is AWS associate solutions architect?
- Is AWS Solutions Architect Associate exam hard?
Infocerts, 5B 306 Riverside Greens, Panvel, Raigad 410206 Maharashtra, India
Contact us – https://www.infocerts.com
