New alert generated as Note Added to existing alert

notifications

#1

Hello All,
We use a combination of OpsGenie and Kapacitor.
Kapacitor sends API alerts to OpsGenie and Opsgeniw sends email notifications to all.
This is working fine.

Current Behaviour:
When a new alert is generated from Kapacitor an email comes from OpsGenie describing the alert with a subject line of the alert.
So if another alert comes from the same alert describing the change in state. OpsGenie sends a note added mail with the same subject line instead of sending the mail with the new subject line.

Expected behaviour:
I want the new alert to be generated with its own subject instead of having the subject of the previous alert.

I know this can be managed in the settings of Note added in integration section.
I am new to OpsGenie Can Any one please guide me on how this can be managed.


#2

Hi,

Unfortunately, we don’t know a lot about the Kapacitor integration, as it is not created by us and is using our generic API. That means we are not in control of the logic - it’s the piece of code on the Kapacitor side.

That being said, I think I can share some OpsGenie mechanics with you to help you understand better why that could be happening:

In OpsGenie, we have a feature called Deduplication. Every time an alert is being created, OpsGenie will check, if there is an alert already open with the same ‘alias’ value. If yes - the count is increased instead of a new alert. Deduplication should be used for the very same alerts, hence the content is not updated. Think of the Alias as an ID - and usually it is an ID from the other system. Let me share our creation flow to visualize this:

What you are experiencing is definitely this - somehow, Kapacitor is sending the “new” alert with the very same alias as the “old” alert, hence the count is increase and the message is not updated.

As for the added Notes : the only update you can capture during the deduplication is a silent note to the alert. This is not governed by the Add Note rule in the integration but by the Create Alert rule. That has a Note: field which will - normally - add a note during the creation process of the alert. When the deduplication happens, OpsGenie adds this note parameter from the new payload to the existing open alert, while increasing the count of the alert.

I’m almost sure this is what you are seeing. In short - Kapacitor perhaps categorized those alerts as the “same” and send it with the very same alias. Probably the builder of the integration is sending some changing numbers in the Note parameter which is added as a silent update to the original open alert. It sounds to me like a designed behavior though. If you would like to change this, you can maybe send the alerts with a more precise alias, or simply remove the alias dynamic field from the advanced settings of the integration. The latter solution will remove deduplication completely from the logic and every singe create request will create an alert. Not recommended, but can be a solution in your specific case.

If you would like to understand the Deduplication+Note case, you may find a longer article here: