TPO.Component Lost:
This incident is related to TPO Sensor TPOHotBackup that sends Component Lost event.
Severity
|
Reason
|
Title
|
Attempts
|
Description
|
Warning
|
Whenever TPO sends Component Lost event
|
{% if group_id %}TPO serving Group ID {{ group_id }}{% else %}Passive TPO{% endif %} has been lost
|
3
|
{{event_name}} on TPO {{device_identifier}} in Cluster {{tpo_cluster}}.<br>The TPO was {% if group_id %}serving Group ID {{ group_id }}{% else %}Passive{% endif %}.<br>{% if new_group_location %}The new TPO that is serving the group is {{new_group_location}}.{%endif%}
|
Normal
|
Whenever TPO sends Component restored event
|
Cleared
|
{{event_name}}on {{tpo_cluster}}.<br>{%if group_id %}Service for Group ID {{ group_id }} has been restored.{% endif %}<br>{% if device_identifier%}TPO {{ device_identifier }} is back online {% endif %}
|
TPO.Certificates
This incident is related to TPO Sensor that Validate Certificates and sends information that TPO Certificate will expire in next 90 days/30days/expired
Severity
|
Reason
|
Title
|
Attempts
|
Description
|
Critical
|
expired
|
{{ certificate Type }} on {{ hostname }} is expired
|
1
|
{{ certificateType }} on {{ hostname }} will expire in the next 30/90/expired days/Valid.<br/>
Certificate Type: {{ certificateType }}<br/>
Certificate Validity: {{ certificateValidity }}<br/>
Certificate Subject: {{ certificateSubject }}<br/>
Certificate Issuer: {{ certificateIssuer }}<br/>
Certificate Version: {{ certificateVersion }}<br/>
Certificate NotBefore: {{ certificateNotBefore }}<br/>
Certificate NotAfter: {{ certificateNotAfter }}<br/>
Certificate SN: {{certificateSN}}<br/>
Certificate Domain: {{ certificateDomain }}
|
Critical
|
expire in next 30 days
|
{{ certificateType }} on {{ hostname }} will expire in the next 30 days
|
1
|
Warning
|
expire in next 90 days
|
{{ certificateType }} on {{ hostname }} will expire in the next 90 days
|
1
|
Normal
|
valid more than 90 days
|
{{ certificateType }} on {{ hostname }} is Valid
|
Cleared
|
TPO.Failover:
Note: This Alarm is disabled by default
This incident is related to TPO Failover recieved via SNMP trap
Please note that this alert is for a failover that has already happened. This alert is intended for the user that they need to normalize the cluster
Severity
|
Reason
|
Title
|
Attempts
|
Description
|
Warning
|
Whenever the SNMP trap is recieved on the system
|
TPO failover happened in cluster {{ tpo_cluster }}
|
1
|
The {{takeover_tpo}} TPO is taken over in {{tpo_cluster}} cluster
|
TPO.Free Memory:
This alarm monitors free memory on TPOs. When the free memory is less than 500 MB then an alarm will be raised, and a log file collected.
Severity
|
Reason
|
Title
|
Attempts
|
Description
|
Warning
|
No time out reported for the tpo and the Free memory for the tpo is less than 500MB.
|
Free memory on TPO is less than 500MB
|
1
|
TPO {{device_identifier}} with address {{ ip_address }} is low on memory. Free memory is currently {{ free_memory }} kb.
|
Normal
|
When Free memory for the tpo is more than 50MB
|
Free memory on TPO is now greater than 500MB.
|
Cleared
|
TPO {{device_identifier}} memory is now at {{ free_memory }} kb
|
TPO.GroupID Changed:
This incident is related to TPO Sensor TPOHotBackup that sends group changed event
Severity
|
Reason
|
Title
|
Attempts
|
Description
|
Warning
|
Whenever there is a group change event
|
TPO Group ID changed in {{ tpo_cluster }} cluster
|
1
|
The TPO {{device_identifier}} of {{ tpo_cluster }} is {{reason}}
|
Normal
|
When TPO Group ID change is recovered
|
TPO Group ID change - recovered
|
Cleared
|
The {{device_identifier}} TPO group ID recovered.
|
TPO.GroupIDMisconfiguration:
This incident is related to TPO Sensor TPOHotBackup that sends group id configuration incorrect event
Severity
|
Reason
|
Title
|
Attempts
|
Description
|
Warning
|
Whenever there is misconfiguration in TPO Clusters Group IDs.
|
TPO Cluster Group ID Misconfiguration
|
3
|
{{device_identifier}} TPO is active with a Group ID({{current_group_id}}) which is not configured in the {{tpo_cluster}} cluster
|
Normal
|
Whenever TPO Cluster Group ID configuration is restored
|
TPO Cluster Group ID configuration restored
|
Cleared
|
{{device_identifier}} TPO is active with a Group ID({{current_group_id}}) which is configured in the {{tpo_cluster}} cluster
|
TPO.HeartbeatFailureCount:
This alarm monitors tpo heartbeat failure count by monitoring the keep alive missed counts
Severity
|
Reason
|
Title
|
Attempts
|
Description
|
Warning
|
Whenever there is no time out reported for the TPO and Status of TPO.missed_keep_alive_count is greater than zero
|
TPO has heartbeat failure count
|
1
|
TPO {{device_identifier}} heartbeat failure count : {{ missed_keep_alive_count }}.
|
TPO.Registration Node Lost:
This incident is related to PbxRegistrationState that sends PBX Node Lost On TPO event. This alarm uses the PbxRegistration sensor to detect registration issues on TPOs
Severity
|
Reason
|
Title
|
Attempts
|
Description
|
Warning
|
Whenever TPO sends an events that it has lost PBX node for extension
|
TPO has lost PBX node for extension
|
6
|
TPO {{device_identifier}} in Cluster {{tpo_cluster}} has failed to register the following lines on the call server at {{ip_address}}:
{% for line in lines %}
{{ line|pprint }}
{%endfor%}
|
Normal
|
Whenever the Sensor PbxRegistrationState sends a PBX Node Restored On TPO event
|
PBX node has been restored for extension on TPO
|
Cleared
|
{% if extension %}
Service has been restored for extension {{ extension }} on TPO {{device_identifier}}.
{% else %}
TPO {{device_identifier}} in Cluster {{tpo_cluster}} has registered with the call server at {{ip_address}}
{% endif %}
|
TPO.Registration Node Lost:
This incident is related to PbxRegistrationState that sends PBX Service Lost On TPO event. This alarm uses the PbxRegistration sensor to detect registration issues on TPOs
Severity
|
Reason
|
Title
|
Attempts
|
Description
|
Critical
|
Whenever TPO fails to register line(s) on the PBX
|
TPO has failed to register line(s) on the PBX
|
6
|
Extension {{ extension }} is no longer available. TPO {{device_identifier}} in Cluster {{tpo_cluster}} has failed to register the following lines:
{% for line in lines %}
{{ line|pprint }}
{%endfor%}
|
Normal
|
Whenever the Sensor PbxRegistrationState sends a PBX Service Restored On TPO event
|
Service has been restored for extension on TPO
|
Cleared
|
{% if extension %}
Service has been restored for extension {{ extension }} on TPO {{device_identifier}}.
{% else %}
TPO {{device_identifier}} in Cluster {{tpo_cluster}} has registered with the call server at {{ip_address}}
{% endif %}
|
TPO.Place Error:
This alarm will be raised if the TPO Place state is in Error
Severity
|
Reason
|
Title
|
Attempts
|
Description
|
Critical
|
Whenever the status of TPO Place is “STARTED” and the current state is in “ERROR
|
TPO place {{name}} state is Error
|
1
|
TPO place {{name}} state is Error<br/>
TPO cluster: {{tpo_cluster.name}}<br/>
Group id: {{tpo_group.tpo_group_id}}<br/>
TPO: {{tpo.tpo_name}}<br/>
Place type: {{place_type}}<br/><br/>
{% if pw_slots %}{% for pw_slot in pw_slots %}{% if pw_slot.state_current=='error' %}Slot: {{ pw_slot.slot }}<br/>
Current state: {{ pw_slot.state_current }}<br/>
Error code: {{ pw_slot.error_code }}<br/>
Error message: {{ pw_slot.error_message }}<br/><br/>
{%endif%}{%endfor%}{%else%}Current state: {{state_current}}<br/>
Error code: {{error_code}}<br/>
Error message: {{error_message}}<br/>
{% endif %}
|
Normal
|
Whenever the status of TPO Place is “STARTED” and the current state is in “STARTED
|
TPO place {{name}} state is Started
|
Cleared
|
TPO place {{name}} state is Started<br/>
nTPO cluster: {{tpo_cluster.name}}<br/>
Group id: {{tpo_group.tpo_group_id}}<br/>
TPO: {{tpo.tpo_name}}<br/>
Place type: {{place_type}}<br/>
Current state: {{state_current}}<br/>
{% if pw_slots %}{% for pw_slot in pw_slots %}{% if pw_slot.state_current=='error' %}Slot: {{ pw_slot.slot }}<br/>
Current state: {{ pw_slot.state_current }}<br/>
Error code: {{ pw_slot.error_code }}<br/>
Error message: {{ pw_slot.error_message }}<br/><br/>
{%endif%}{%endfor%}{%endif%}
|
PrimaryRecordingSession:
This alarm monitors Recording session.
Severity
|
Reason
|
Title
|
Attempts
|
Description
|
Warning
|
Whenever the primary Voice recorder connected to the tpo is down and no time out reported for the tpo. The recording server ID should not be null to raise this alarm.
|
Voice recording session on tpo {{device_identifier}} is down
|
3
|
{{device_identifier}} voice recording session on {{recording_server.ip_address_primary}} is down
|
Normal
|
When the primary Voice recorder connected to the tpo is UP
|
Voice recording session on tpo {{device_identifier}} is up
|
Cleared
|
{{device_identifier}} voice recording session on {{recording_server.ip_address_primary}} is up
|
SecondaryRecordingSession:
This alarm monitors Recording session.
Severity
|
Reason
|
Title
|
Attempts
|
Description
|
Warning
|
Whenever the Secondary Voice recorder connected to the tpo is down and no time out reported for the tpo. The recording server ID should not be null to raise this alarm.
|
Voice recording session on tpo {{device_identifier}} is down
|
3
|
{{device_identifier}} voice recording session on {{recording_server.ip_address_secondary}} is down
|
Normal
|
When the Secondary Voice recorder connected to the tpo is UP.
|
Voice recording session on tpo {{device_identifier}} is up
|
Cleared
|
{{device_identifier}} voice recording session on {{recording_server.ip_address_secondary}} is up
|
TPO. Recorder State:
This alarm monitors TPO Recorder Status.
Severity
|
Reason
|
Title
|
Attempts
|
Description
|
Critical
|
|
Voice recording is unavailable on TPO
|
3
|
Voice recording is down on TPO {{ device_identifier }}. Configured to use the following Voice Recorders: {{recording_server.ip_address_primary}} {{recording_server.ip_address_secondary}}
|
Normal
|
When the primary Voice recorder connected to the TPO is UP
|
Voice recording is restored on TPO
|
Cleared
|
Voice recording is restored on TPO {{ device_identifier }}
|
TPO. RoleChanged:
This alarm monitors TPO role change Status.
Severity
|
Reason
|
Title
|
Attempts
|
Description
|
Warning
|
Whenever the sensor TPOHotBackup sends role changed event.
|
TPO Role changed in {{ tpo_cluster }} cluster
|
1
|
The TPO {{device_identifier}} of {{ tpo_cluster }} is {{reason}}
|
Normal
|
Whenever the Sensor TPOHotBackup sends role change recovered event.
|
TPO Role change - recovered
|
Cleared
|
The {{device_identifier}} TPO role change recovered.
|
TPO. RTPTimeOut:
This alarm monitors RTP timeout at TPO. This alarm is split into two namely, TPO.RTPMasterCallTimeOut and TPO.RTPLocalCallTimeOut.
Severity
|
Reason
|
Title
|
Attempts
|
Description
|
Warning
|
Whenever a SNMP Trap is received with Eventname/OID rtpTimeout for the type TPO.
|
{{type}} RTP Local Call Timed out
|
1
|
{{type}} {{device_identifier}} RTP Local Call Timed out. Below are the details <br/>
Zone = {{zone}}, <br/>
Call ID = {{rtpTimeoutCallID}}, <br/>
Turret User = {{rtpTimeoutTurretUser}}, <br/>
Turret Host = {{rtpTimeoutTurretHost}}, <br/>
Timestamp = {{rtpTimeoutTimestamp}} <br/>
Timeout Index = {{rtpTimeoutIndex}} <br/>
Call Type = {{rtpTimeoutCallType}}
|
TPO. Split Brain:
A Split Brain is where multiple TPOs in the same Cluster both claim to be serving the same Group ID. This alarm monitors Split Brain functionality of TPO.
Severity
|
Reason
|
Title
|
Attempts
|
Description
|
Warning
|
Whenever the TPO sends Split Brain event as a trap
|
Split Brain on {{device_identifier}} TPO.
|
1
|
Split Brain on {{device_identifier}} TPO in {{tpo_cluster}} Cluster, {{zone}} Zone.
|
TPOGroup.Service Lost:
This alarm is to figure out whether all groupID's are actually run by a TPO, if one is missing this means that there are no more backup nodes available to handover groupID's job.
Severity
|
Reason
|
Title
|
Attempts
|
Description
|
Critical
|
Whenever the sensor TPOHotBackup sends Service Lost event
|
Service of Group ID {{ group_id }} has been lost
|
3
|
{{event_name}} on {{tpo_cluster}}.{% if passive_tpos %} Passive TPOs {{ passive_tpos }} are available in cluster {{ tpo_cluster }} but service of Group ID {{ group_id }} has been lost.{% else %} No TPOs available to serve Group ID {{ group_id }} and service has been lost.{%endif%}
|
Normal
|
Whenever the sensor TPOHotBackup sends Service restored event
|
Service of Group ID {{ group_id }} has been restored
|
Cleared
|
{{event_name}} on {{tpo_cluster}}. {% if group_id %}Service for Group ID {{ group_id }} has been restored.{% endif %}{% if device_identifier %}TPO {{ device_identifier }} is back online {% endif %}
|
TPO.Rebooted:
Note: This Alarm is disabled by default
This alarm is to identify if TPO reboots
Severity
|
Reason
|
Title
|
Attempts
|
Description
|
Warning
|
As and when TPO reboots
|
{{device_identifier}} TPO rebooted
|
1
|
{{device_identifier}} TPO in Cluster {{tpo_cluster}} rebooted.
|
TPO.recorderLostKeepalive:
This alarm is to monitor which recorder lost the keep alive connection as it receives a trap from the tpo as and when the connection is lost
Severity
|
Reason
|
Title
|
Attempts
|
Description
|
Warning
|
As and when the recorder connection is lost
|
Keep alive connection lost to the recorder {{device_identifier}} TPO.
|
1
|
Recording Keep Alive connection lost for {{device_identifier}} TPO in {{tpo_cluster}} Cluster, {{zone}} Zone.
|
|