Amazon CloudWatch Integration

The Amazon CloudWatch integration allows you to be notified when critical metrics in your infrastructure breach a threshold or stop receiving data, as well as when these metrics recover.

This data is emitted from AWS via CloudWatch Alarms, which notify an Amazon SNS topic configured to send data to Ready Five.

Create SNS topic

First, we configure the connection between SNS and Ready Five. In the AWS Console, open the Amazon SNS console and create a new topic. If you have an existing topic you'd prefer to use, you can skip this step.

Choose a generic name, typically something describing the contents that go through the topic, not where the data comes from or where it's going (you probably don't want to include Ready Five in the name, as topics can have a nearly unlimited number of subscribers). In this case, it would make sense for it to have CloudWatch as part of its name to document the expectation that only CloudWatch events should be published to the topic.

Create the Ready Five integration

In your web browser, navigate to the "Integrations" tab in the team that should own this integration and click the "Add Integration" button.

For the integration type, click the "Add" button in the Amazon CloudWatch box. Although the source of this integration is Amazon SNS, the Amazon SNS integration is different, as it is not prepared to handle the shape of the event sent by CloudWatch.

Give the integration a name (or keep the default) and an optional description and click "Add".

The integration is now created, and you now have a URL on this screen that you'll need in a minute. Keep this tab open and open another.

Add the Ready Five integration as an SNS subscription

Copy this URL and head back to the AWS SNS Console. From your SNS topic screen, click the "Create subscription" button.

The Topic ARN should be pre-filled with your topic. If it isn't for some reason, search for it and select it. Choose "HTTPS" as the Protocol. Paste the URL provided on the Ready Five integration screen and click "Create subscription".

Confirm the SNS subscription

Amazon SNS subscriptions must be confirmed before messages will be published to the subscriber. Upon creating the subscription, a confirmation message is sent to the URL, which creates an incident in Ready Five.

Head over to the Ready Five incidents screen. You'll find a newly created incident in the Open / All tab if you're an account owner or administrator or in the Open / My Teams tab if you're a team administrator.

Open this incident and scroll through its details until you find the "SubscribeURL" attribute.

Click the URL to confirm this subscription. The confirmation screen displays some lovely XML. Though this may seem like an error at first glance, it's expected.

Take another look at the subscription in the AWS console and it will show as confirmed.

This confirmation means that any messages sent to this topic will be forwarded to the Ready Five integration specified in the subscription.

You can resolve the incident created by the subscription confirmation message.

Add a CloudWatch Alarm

Now that the SNS topic, Ready Five integration, and SNS subscription are properly configured, we can configure events to send to this topic.

Head over to the AWS CloudWatch console and open the Alarms section. You can attach alarms to any metric in your AWS account. Here we'll demonsrate creating an alarm by hand and configuring it to notify the SNS topic. This works well for configuring a handful of static alarms. If your infrastructure is dynamic though, you may want to codify alarm creation (with CloudFormation, Terraform, etc.).

Click "Create alarm" to begin crafting our new alarm.

Click "Select metric" to choose a metric.

You can choose any metric in your account. For this example we're going to choose Lambda => Across All Functions => Duration. Then click "Select metric".

In this case, we want to know as quickly as possible if our average Lambda invocation duration exceeds 150ms. Set the Statistic to "Average" and the Period to "1 minute".

Specify 150 as the threshold value and we want to "Treat missing data as good (not breaching threshold)". The "Missing data as bad" setting is useful for heartbeat / dead-man's switch cases where you want to consider the alarm breached if a metric is not published in specified time period.

Then click "Next.

Now we need to configure what to do when the alarm breaches (ALARM state), as well as what to do when it resolves (OK state). First we'll say that when the alarm is triggered, we'll send a notification to our topic. Then click "Add notification", and target the same SNS topic when the alarm is in the OK state.

Defining the OK action is optional. If you want Ready Five to automatically resolve the incident when the alarm resolves, you'll want to specify this OK action. There are many alarms that inherently require human intervention where a threshold breach requires a human to acknowledge and resolve the incident manually.

Click "Next".

Now we need give the alarm a name and a description. Since a lot of alarms use auto-generated names (CloudFormation), we've designed this integration to prefer the description field if it's provided when creating incident titles from an alarm. If there's no description, we'll use the name. Use detailed descriptions to help you out - remember these alerts might wake you up at night so make them helpful now!

Last step is creating the alarm. Within about 2 minutes, it will be active and will publish messages to SNS and notify your Ready Five account whenever the alarm breaches the defined threshold.

You can send any number of CloudWatch alarms at the same SNS topic (and the same Ready Five integration). The incidents created or resolved are aware of which alarm is breaching or resolving, so we'll resolve the right one when you combine alerts in a single topic.