How To: List the System Alerts for a Kubernetes Cluster

After deploying the RealTheory collector to a Kubernetes cluster (See Installing RealTheory in a Cluster), the cluster and its associated objects, including any System Alerts, become available in the RealTheory API within minutes.

Get System Alerts

To get the list of available system alerts for a given cluster, call the method below on the Alert endpoint.

GET <endpoint>/theory/api/v1/alert/systemalerts/search/query/{query}?index={index}&count={count}&order={order}&direction={direction}

where

• {endpoint} - is the URL of the RealTheory SaaS portal for your account.
• {query} - is a query used to limit the results to alerts within a given cluster.
• {index} - is the starting index of the alerts to include in the list. This parameter is optional and defaults to 0 if not specified.
• {count} - is maximum number of alerts to return. This parameter is optional and defaults to 100 if not specified.
• {order} - is a string containing the name of the attribute to sort on. This parameter is optional.
• {direction} - is a number that controls the direction of the sort. A value equal to or greater than 0 indicates ascending order; a value less than 0 indicates descending order. This parameter is optional and defaults to 0 if not specified.

Note: All calls to RealTheory APIs must be authenticated and authorized. See Authentication for more details.

Example

GET theory/api/v1/alert/systemalerts/search/query/metadata.clusterName=realtheory-cluster-01?index=0&count=100&order=level&direction=1

which returns a list of the first 100 system alerts associated with cluster realtheory-cluster-01, sorted by level in ascending order.

Notes:

• As cluster names are not guaranteed to be unique across all environments, it is strongly recommended to use metadata.clusterId field instead of the methadata.clusterName in order to uniquely identify a cluster in the above query.
• The query should be URL encoded; otherwise, the request might be rejected.

Output

Returns an array of SystemAlert objects:

[
    {
        "type": "System",
        "metadata": {
            "objectType": "Deployment",
            "objectId": "1280b518-ab88-4387-a735-5162bc9a22d0",
            "objectName": "queue",
            "clusterId": "f99e055f-b35d-208b-63b1-f4222b80151a",
            "clusterName": "realtheory-cluster-01",
            "nodeId": "00000000-0000-0000-0000-000000000000",
            "nodeName": "",
            "customResourceDefinitionId": "00000000-0000-0000-0000-000000000000",
            "customResourceDefinitionName": "",
            "namespaceId": "0b3c50fe-5116-44bd-b229-543251be4e74",
            "namespaceName": "web-app",
            ...
        },
        "advisoryId": "00000000-0000-0000-0000-000000000000",
        "id": "2b2e816d-d76b-4565-88a0-c83413b4a01b",
        "key": "MetricsPerformanceCheck::1280b518ab884387a7355162bc9a22d0::memory_percentage::gt",
        "title": "The deployment's overall memory usage was above a specific threshold.",
        "message": "The 'Deployment High Memory Usage Check' has identified a high impact performance concern on deployment 'queue'.",
        "details": "Memory Percentage - Threshold: 95%, Actual: 119.74%",
        "comments": "",
        "level": "Critical",
        "category": "Performance",
        "source": "Deployment High Memory Usage Check (performance check)",
        "sourceId": "67847808-28f2-4f47-b8ce-cc888b34f7b2",
        "url": "",
        "state": "Read",
        "created": "2024-06-30T19:44:34.0367823Z",
        "updated": "2024-09-04T18:56:37.0746637Z",
        "postponedUntil": null,
        "expires": "2024-09-04T19:01:35.6003209Z"
    },
    {
        "type": "System",
        "metadata": {
            "objectType": "Deployment",
            "objectId": "ea4b4de0-51da-4020-acde-87cb2da015ca",
            "objectName": "console",
            "clusterId": "f99e055f-b35d-208b-63b1-f4222b80151a",
            "clusterName": "realtheory-cluster-01",
            "nodeId": "00000000-0000-0000-0000-000000000000",
            "nodeName": "",
            "customResourceDefinitionId": "00000000-0000-0000-0000-000000000000",
            "customResourceDefinitionName": "",
            "namespaceId": "0b3c50fe-5116-44bd-b229-543251be4e74",
            "namespaceName": "web-app",
            "serviceId": "00000000-0000-0000-0000-000000000000",
            "serviceName": "",
            "replicaSetId": "00000000-0000-0000-0000-000000000000",
            "replicaSetName": "",
            "deploymentId": "ea4b4de0-51da-4020-acde-87cb2da015ca",
            "deploymentName": "console",
            ...
        },
        "advisoryId": "00000000-0000-0000-0000-000000000000",
        "id": "2edf4012-09a4-21a3-eebe-ed4bee2cfbce",
        "key": "DeploymentPodsReadyHealthCheck::ea4b4de051da4020acde87cb2da015ca",
        "title": "The deployment did not have the minimum number of ready pods.",
        "message": "The 'Deployment Pods Ready Check' has identified a very high impact health concern on deployment 'console'.",
        "details": "Pods Ready Percentage - Threshold: 80%, Actual: 0%",
        "comments": "",
        "level": "Critical",
        "category": "Health",
        "source": "Deployment Pods Ready Check (health check)",
        "sourceId": "e298f7be-e6be-4f46-9786-8e10f4a987ee",
        "url": "",
        "state": "Read",
        "created": "2024-06-30T19:44:59.1252518Z",
        "updated": "2024-09-04T18:58:38.3484048Z",
        "postponedUntil": null,
        "expires": "2024-09-04T19:03:36.6203679Z"
    }

Note: Content has been omitted from the JSON response above for brevity.