# Alerting

The **Alerting** app in Tessell Observability app family lets you define the conditions under which alerts are triggered, and where notifications are sent when those conditions are met. Alerting is organized into three layers that work together: Notification Channels, Notification Policies, and Alert Policies.

***

## Prerequisites

* Ensure that you have registered your cloud subscription and added your network details in the Subscriptions application under Governance.\
  For more information on cloud subscriptions, see [Subscriptions](https://docs.tessell.com/tessell/governance/subscriptions).

***

## Overview

### How alerting works

Tessell alerting uses a three-tier model:

1. **Notification Channels** define individual destinations — such as a Slack workspace, a webhook endpoint, or an email address — where alert notifications are delivered.
2. **Notification Policies** group one or more Notification Channels under a single named policy, making it easy to reuse a set of destinations across multiple alerts.
3. **Alert Policies** define the monitoring rules — which metrics to watch, at what thresholds, and with what severity — and associate those rules with one or more subscriptions and a Notification Policy.

When a metric breaches a configured threshold in the customer environment, Tessell evaluates the alert condition, generates an alert, and routes the notification to all channels defined in the associated Notification Policy.

### Alert severity levels

Each metric rule can be configured with up to three severity levels:

| Severity     | Description                                                                    |
| ------------ | ------------------------------------------------------------------------------ |
| **Critical** | Highest priority. Indicates a serious condition requiring immediate attention. |
| **High**     | Elevated priority. Indicates a condition that may escalate if not addressed.   |
| **Warning**  | Informational. Indicates a condition approaching a threshold of concern.       |

### Metric types

Alert Policies are scoped to one of two metric types:

* **DB Metrics** — Database-level metrics such as primary instance count, connections, and query performance. Metrics are organized by unit type (count, percent, seconds, state).
* **OS Metrics** — Operating system metrics for DB compute nodes, such as CPU usage, memory, and node availability. Metrics are organized by unit type (boolean, count, percent, state).

**Figure 1 - Alerting dashboard**

![](https://3421475909-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FXNKiIctERJykfezXfUU2%2Fuploads%2Fgit-blob-9cc9abba3db2aef3417f3daec5152c03ee7d8b56%2Falerting-dashboard.png?alt=media)

***

## Notification Channels

A Notification Channel represents a single destination for alert notifications. Tessell supports three channel types: Slack, Webhook, and Email.

### Add a Slack channel

**Before you begin:** Obtain the Slack API URL (incoming webhook URL) for the target Slack channel.

1. In the left navigation, select the **Alerting** icon.
2. On the **Alerting** page, select the **Notification Channels** tab.
3. On the **Slack** card, select **Add New**.
4. In the **Add Channel – Slack** dialog, complete the following fields:

   | Field                  | Description                                              |
   | ---------------------- | -------------------------------------------------------- |
   | **Display Name**       | A friendly name to identify this channel within Tessell. |
   | **Slack Channel Name** | The name or identifier of the target Slack channel.      |
   | **Slack API URL**      | The incoming webhook URL for the Slack channel.          |
5. Optionally, select **Send Test Notification** to verify the configuration.
6. Select **Add Channel**.

**Figure 2 - Add Slack channel dialog**

![](https://3421475909-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FXNKiIctERJykfezXfUU2%2Fuploads%2Fgit-blob-2bf47a9ded8823aeccb46cb0aae02f6eceecfd5b%2Fadd-slack-channel.png?alt=media)

### Add a Webhook channel

**Before you begin:** Obtain the endpoint URL and authentication credentials for the target service.

1. In the left navigation, select the **Alerting** icon.
2. On the **Alerting** page, select the **Notification Channels** tab.
3. On the **Web-hooks** card, select **Add New**.
4. In the **Add Channel – Webhook** dialog, complete the following fields:

   | Field            | Description                                              |
   | ---------------- | -------------------------------------------------------- |
   | **Display Name** | A friendly name to identify this channel within Tessell. |
   | **Endpoint URL** | The HTTPS endpoint URL that will receive alert payloads. |
5. Under **Headers**, select the authentication method:
   * **HTTP Basic Auth** — Enter the authorization value and token.
   * **API Keys** — Enter the API key credentials.
6. Optionally, select **Send Test Notification** to verify the configuration.
7. Select **Add Channel**.

**Figure 3 - Add Webhook channel dialog**

![](https://3421475909-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FXNKiIctERJykfezXfUU2%2Fuploads%2Fgit-blob-a3784145cb2bd16c4dcb45daaccb7f1d76f2f737%2Fadd-webhook-channel.png?alt=media)

### Add an Email channel

1. In the left navigation, select the **Alerting** icon.
2. On the **Alerting** page, select the **Notification Channels** tab.
3. On the **Email** card, select **Add New**.
4. In the **Add Email** dialog, complete the following fields:

   | Field            | Description                                                  |
   | ---------------- | ------------------------------------------------------------ |
   | **Display Name** | A friendly name to identify this channel within Tessell.     |
   | **Email id**     | The email address to which alert notifications will be sent. |
5. Select **Add Channel**.

**Figure 4 - Add Email channel dialog**

![](https://3421475909-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FXNKiIctERJykfezXfUU2%2Fuploads%2Fgit-blob-5df85c340ccc63d7e9388d88b073fed9349c5cad%2Fadd-email-channel.png?alt=media)

***

## Notification Policies

A Notification Policy is a named group of one or more Notification Channels. By associating an Alert Policy with a Notification Policy rather than individual channels, you can manage notification destinations centrally and reuse them across multiple alert configurations.

Each **Notification Policy** registered with Tessell has policy name, total number of channels, channel details, last modified details, and the policy creator.

### Create a Notification Policy

1. In the left navigation, select the **Alerting** icon.
2. On the **Alerting** page, select the **Notification Policies** tab.
3. Select **Create Policy**.
4. On the **Create Notification Policy** page, under **Policy Configuration**, specify a **Policy Name**.
5. Under **Notification Configuration**, in the **Select Notification Channels** picker:

   a. Browse or search for channels by name.\
   b. Select a channel type (Slack or Email) to expand the list of available channels.\
   c. Select the checkbox next to each channel to include.\
   d. Select **Add** to confirm your selection.
6. Select **Create Policy**.

**Figure 5 - Create Notification Policy page**

![](https://3421475909-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FXNKiIctERJykfezXfUU2%2Fuploads%2Fgit-blob-83564c4a5e5573b694596e819f278d67b2e18a86%2Fcreate-notification-policy.png?alt=media)

***

## Alert Policies

An Alert Policy defines the conditions under which alerts are triggered for resources within a subscription. Each policy is scoped to either DB Metrics or OS Metrics, and is linked to a Notification Policy that determines where alerts are sent.

Each **Alert Policy** registered with Tessell has policy name, policy status, total number of metrics, metric details, creation date, and the policy creator.

### Create an Alert Policy

1. In the left navigation, select the **Alerting** icon.
2. On the **Alerting** page, select the **Alert Policies** tab.
3. Select **Create Policy**.
4. On the **Create Alert Policy** page, under **Policy Details**, complete the following:

   | Field           | Description                                                                                                                |
   | --------------- | -------------------------------------------------------------------------------------------------------------------------- |
   | **Policy Name** | A unique name for this alert policy.                                                                                       |
   | **Metric Type** | Select **DB Metrics** to monitor database metrics, or **OS Metrics** to monitor operating system metrics on compute nodes. |
5. Under the metrics section (Database (DB) Metrics or Operating System (OS) Metrics, depending on your selection), configure at least one metric rule:

   a. In the **Add New Metric** dropdown, select a metric. Metrics are organized by unit type. A description of the selected metric is shown on the right.

   b. When you select a metric, the metric configuration panel appears. Set the following in the metric configuration panel:

   | Field                           | Description                                                                                                                                                    |
   | ------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
   | **Severity**                    | Select one or more severity levels (Critical, High, Warning) and set a threshold value for each.                                                               |
   | **Trigger Condition**           | Define when the alert fires — for example, when the metric equals the threshold, or when a status persists for a period of time within an evaluation interval. |
   | **Evaluation Window (in mins)** | The time window over which the metric is evaluated. Default is 5 minutes.                                                                                      |

   c. Select **Add** to save the metric rule.

   d. Repeat for additional metrics as needed.
6. Under **Alert Annotations**, configure the metadata to include in each alert notification. Annotations provide contextual information that helps recipients quickly identify and triage the alert. For more information, see [Alert Annotations](#alert-annotations).
7. Under **Target Subscription**, select the subscription to which this policy applies.
8. Under **Notification Configuration**, select **Select Notification Policy** and choose the policy that defines where alerts are sent. Expand any policy to preview its included channels.
9. Select **Create Alert Policy**.

> **Note:** Each subscription can be associated with only one Alert Policy per metric type (DB or OS) at a time.

### Alert Annotations

Alert Annotations let you enrich alert notifications with structured metadata, so that every notification carries the context needed for rapid triage and seamless integration with incident management systems. You configure annotations as part of the Alert Policy creation flow.

#### Default Annotations

Tessell automatically includes a set of default annotations with every alert. These are system-defined and cannot be removed. Default annotations include:

| Annotation                  | Description                                                             |
| --------------------------- | ----------------------------------------------------------------------- |
| **tenant\_name**            | The name of the tenant that owns the monitored resource.                |
| **service\_instance\_role** | The role of the service instance (for example, primary or replica).     |
| **threshold**               | The threshold value that was breached.                                  |
| **alert\_profile\_name**    | The name of the alert policy that triggered the alert.                  |
| **tenant\_id**              | The unique identifier of the tenant.                                    |
| **service\_name**           | The name of the database service associated with the alert.             |
| **alertname**               | The name of the specific alert that was triggered.                      |
| **service\_instance\_id**   | The unique identifier of the service instance.                          |
| **severity**                | The severity level of the triggered alert (Critical, High, or Warning). |

#### Custom Annotations

In addition to the default annotations, you can define custom annotations to embed additional metadata into alert notifications. Custom annotations are useful for adding environment tags, identifying impacted services, or including any other information relevant to your operational workflows.

Each custom annotation consists of the following:

| Field            | Description                                                                                                                                                                                                   |
| ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Display Name** | A label for the annotation (maximum 40 characters). This appears as the metadata key in alert notifications.                                                                                                  |
| **Type**         | Select **Dynamic** to use a system variable that resolves at alert time, or **Static** to set a fixed value.                                                                                                  |
| **Value**        | For Dynamic annotations, select a variable from the dropdown. Variables are organized by category (Tenant, Database Services, Metrics, and Alert Profiles). For Static annotations, enter a fixed text value. |

To add a custom annotation:

1. In the **Alert Annotations** section of the Create Alert Policy page, under **Custom Annotations**, locate the **Add Annotation** row.
2. Enter a **Display Name** for the annotation.
3. Select the annotation type: **Dynamic** or **Static**.
4. Specify the value:
   * For **Dynamic**, select a variable from the dropdown (for example, `$server.id` from the Tenant category, or `$dbservice.impacted.names` from Database Services).
   * For **Static**, enter a fixed text value (for example, `PROD`).
5. Select **+ Add** to save the annotation.
6. Repeat for additional annotations as needed.

Added custom annotations appear in a table showing the Display Name, Value, and Category. You can remove an annotation by selecting the delete icon next to it.

**Example custom annotations:**

| Display Name             | Value                     | Category          |
| ------------------------ | ------------------------- | ----------------- |
| impacted\_services       | $server.id                | Tenant            |
| environment              | PROD                      | Static            |
| numeric\_tenant\_id      | $numeric\_tenant\_id      | Tenant            |
| impacted\_service\_names | $dbservice.impacted.names | Database Services |

#### Preview Annotations

To verify how your annotations will appear in alert notifications before saving the policy, select the **Preview** link in the upper-right corner of the Alert Annotations section. The preview renders a sample alert notification with both default and custom annotations populated.

**Figure 6 - Create Alert Policy page**

![](https://3421475909-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FXNKiIctERJykfezXfUU2%2Fuploads%2Fgit-blob-4185d5f46a3a42bab783168cda1b821d03a32e8a%2Fcreate-alert-policy.jpg?alt=media)