Understanding Alarms

The Orchestrator and the appliances have separate alarm summaries and alarm tables.

Silver Peak appliances "push" all their alarms to the Orchestrator database, which then updates the Appliance Alarms table.

Each entry represents one current condition that may require human intervention. Because alarms are conditions, they may come and go without management involvement.

Whereas merely acknowledging most alarms does not clear them, some alarm conditions are set up to be self-clearing when you acknowledge them. For example, if you remove a hard disk drive, it generates an alarm; once you’ve replaced it and it has finished rebuilding itself, the alarm clears.

Categories of Alarms

Alarms have one of four severity levels: Critical, Major, Minor, and Warning. Only Critical and Major alarms are service-affecting.

n	Critical alarms require immediate attention, and reflect conditions that affect an appliance or the loss of a broad category of service.

n	Major alarms reflect conditions which should be addressed in the next 24 hours. An example would be an unexpected traffic class error.

n	Minor alarms can be addressed at your convenience. An example of a minor alarm would be a user not having changed their account’s default password, or a degraded disk.

n	Warnings inform you of conditions that may become problems over time. For example, a software version mismatch.

Types of Appliance Alarms

The appliance can raise alarms based on issues with tunnels, software, equipment, and Threshold Crossing Alerts (TCAs). The latter are visible on the appliance but managed by the Orchestrator.

Although Appliance Manager (the WebUI) doesn’t display Alarm Type ID (Hex) codes, the data is available for applications that can do their own filtering, such as SNMP.

Silver Peak Appliance Alarms  

Subsystem	Alarm Type ID (Hex)	Alarm Severity	Alarm Text
Tunnel	00010009	CRITICAL	An unexpected GRE packet was detected from tunnel peer. RESOLUTION: Check for tunnel encapsulation mismatch.
	00010003	CRITICAL	Tunnel keepalive version mismatch RESOLUTION: Tunnel peers are running incompatible software versions. • Normal during a software upgrade. • Run the same or compatible software releases among the tunnel peers.
	00010008	CRITICAL	Tunnel local IP address not owned by this appliance. RESOLUTION: Delete the tunnel and re-create it with a valid IP address.
	00010001	CRITICAL	Tunnel state is Down RESOLUTION: Cannot reach tunnel peer. • Check tunnel configuration [Admin state, Source IP/Dest IP, IPsec] • Check network connectivity.
	00010007	MAJOR	Duplicate license detected in peer (only applies to virtual appliance) RESOLUTION: Install unique license on all virtual appliances. To check and/or change license: • In WebUI: Administration > License & Registration • In Orchestrator: Administration > Licenses
	0001000a	MAJOR	Software version mismatch between peers results in reduced functionality. RESOLUTION: Tunnel peers are not running the same release of software. This results in reduced functionality. Run the same or compatible software releases among the tunnel peers.
	00010000	MAJOR	Tunnel remote ID is misconfigured RESOLUTION: System ID is not unique. • Virtual Appliance: Was the same license key used? • Physical Appliance: Change System ID in the rare case of a duplicate ID (CLI command: system id < >)
	00010005	MINOR	Tunnel software version mismatch RESOLUTION: Tunnel are not running the same release of software. They will function, but with reduced functionality. • Normal during an upgrade. • Run the same software version to eliminate the alarm and fully optimize.
Software	0004001c	CRITICAL	EC license not granted RESOLUTION: Please obtain additional EC (EdgeConnect) licenses.
	0004000c	CRITICAL	Invalid virtual appliance license. RESOLUTION: Enter a new license key on the <System Page> to proceed.
	00040016	CRITICAL	Software capability license has expired. RESOLUTION: You must have HTTPS connectivity to internet to renew the licensing token.
	00040003	CRITICAL	The licensing for this virtual appliance has expired. [For VX series only]1 RESOLUTION: Enter a new license.
	00040004	CRITICAL	There is no license installed on this virtual appliance. [For VX series only]a RESOLUTION: Enter a valid license.
	00040005	MAJOR	A disk self-test has been run on the appliance. RESOLUTION: Reboot the appliance. Traffic will not be optimized until this is performed.
	00040013	MAJOR	A peer name has been specified in the route-map configuration matching no existing remote peer RESOLUTION: Correct route-map entry or build tunnel.
	00040019	MAJOR	Application deleted on portal RESOLUTION: Contact Customer Service.
	0004000d	MAJOR	Dual wan-next-hop topology is no longer supported. RESOLUTION: Reconfigure appliance as single bridge with one next-hop, or dual bridge with two IP addresses and two next-hops.
	00040010	MAJOR	Major inconsistency among tunnel traffic class settings found during upgrade. RESOLUTION: Review the WAN shaper traffic class settings.
	0004001b	MAJOR	Portal registration data incorrect RESOLUTION: Please provide valid portal account registration information.
	00040002	MAJOR	Significant change in time of day has occurred, and might compromise statistics. Please contact TAC. RESOLUTION: Appliance statistics could be missing for a substantial period of time. Contact Customer Service.
	00040015	MAJOR	Software capability license needs to be renewed before it expires. RESOLUTION: Software will automatically renew the licensing token as long as it has HTTPS connectivity to the internet.
Software (cont.)	00040001	MAJOR	System is low on resources RESOLUTION: Contact Customer Service.
	00040011	MAJOR	Tunnel IP header disable setting was discarded during upgrade. RESOLUTION: Review the optimization map header compression settings.
	0004000a	MAJOR	Virtual appliance license expires on mm/dd/yyy. [15-day warning] RESOLUTION: Enter a new license key on the <System Page> to avoid loss of optimization or potential traffic disruption.
	0004001a	MINOR	Performance limited by maximum Boost bandwidth RESOLUTION: Recommend subscribing to more Boost bandwidth.
	00040012	WARNING	A very large range has been configured for a local subnet. RESOLUTION: Please confirm that you intended to configure such a large local subnet.
	00040014	WARNING	Interface shaper max bandwidth exceeds system max bandwidth RESOLUTION: Review the interface shaper max bandwidth settings. Please make sure it doesn't exceed system max bandwidth.
	0004000f	WARNING	Minor inconsistency among tunnel traffic class settings found during upgrade. RESOLUTION: Review the WAN shaper traffic class settings.
	0004000e	WARNING	Setting default system next-hop to VLAN next-hop no longer necessary. RESOLUTION: Use the VLAN IP address as tunnel source endpoints instead of bvi0.
	00040017	WARNING	Silver Peak portal is unreachable. RESOLUTION: Appliance cannot connect to Silver Peak portal using HTTPS. This connectivity is needed for internet applications classification.
	00040018	WARNING	Silver Peak portal is unreachable. RESOLUTION: Appliance cannot connect to Silver Peak portal using HTTPS Websockets.
	00040009	WARNING	The NTP server is unreachable. RESOLUTION: Check the appliance’s NTP server IP and version configuration: • Can the appliance reach the NTP server? • Is UDP port 123 open between the appliance’s mgmt0 IP and the NTP server?
	00040007	WARNING	The SSL certificate is not yet valid. RESOLUTION: The SSL certificate has a future start date. It will correct itself when the future date becomes current. Otherwise, install a certificate that is current.
Software (cont.)	00040008	WARNING	The SSL certificate has expired. RESOLUTION: Reinstall a valid SSL certificate that is current.
	00040006	WARNING	The SSL private key is invalid. RESOLUTION: The key is not an RSA standard key that meets the minimum requirement of 1024 bits. Regenerate a key that meets this minimum requirement.
	0004000b	WARNING	Virtual appliance license expires on mm/dd/yyy. [45-day warning] RESOLUTION: Enter a new license key on the <System Page> to avoid loss of optimization or potential traffic disruption.
Equipment	0003002b	CRITICAL	Bridge creation failed RESOLUTION: Check log messages for more details on the failure.
	00030029	CRITICAL	Bridge loop is detected RESOLUTION: Make sure bridge ports are connected to different virtual switches and restart the appliance. Traffic will not be optimized until this is resolved.
	00030007	CRITICAL	Encryption card hardware failure RESOLUTION: Contact Customer Service.
	00030003	CRITICAL	Fan failure detected RESOLUTION: Contact Customer Service.
	00030024	CRITICAL	Insufficient configured memory size for this virtual appliance RESOLUTION: Assign more memory to the virtual machine, and restart the appliance. Traffic will not be optimized until this is resolved.
	00030025	CRITICAL	Insufficient configured processor count for this virtual appliance RESOLUTION: Assign more processors to the virtual machine, and restart the appliance. Traffic will not be optimized until this is resolved.
	00030026	CRITICAL	Insufficient configured disk storage for this virtual appliance RESOLUTION: Assign more storage to the virtual machine, and restart the appliance. Traffic will not be optimized until this is resolved.
	00030005	CRITICAL	LAN/WAN fail-to-wire card failure RESOLUTION: Contact Customer Service.
	0003002a	CRITICAL	Network interface is unassigned RESOLUTION: Assign the network interface to an existing MAC address, and then restart the appliance. Or, if the network interface isn't being used, then set its admin state to down.
	00030021	CRITICAL	NIC interface failure RESOLUTION: Contact Customer Service.
Equipment (cont.)	00030004	CRITICAL	System is in Bypass mode RESOLUTION: Normal with factory default configuration, during reboot, and if user has put the appliance in Bypass mode. Contact Customer Service if the condition persists.
	0003001d	MAJOR	Bonding members have different speed/duplex RESOLUTION: Check interface speed/duplex settings and negotiated values on wan0/wan1 and lan0/lan1 etherchannel groups.
	0003001c	MAJOR	[Flow redirection] cluster peer is down RESOLUTION: • Check flow redirection configuration on all applicable appliances. • Check L3/L4 connectivity between the peers. • Open TCP and UDP ports 4164 between the cluster peer IPs if they are blocked.
	00030017	MAJOR	Disk removed by operator RESOLUTION: Normal during disk replacement. Insert disk using UI/Orchestrator. Contact Customer Service if insertion fails.
	00030001	MAJOR	Disk is failed RESOLUTION: Contact Customer Service to replace disk.
	00030015	MAJOR	Disk is not in service RESOLUTION: • Check to see if the disk is properly seated. • Contact Customer service for further assistance.
	0003000b	MAJOR	Interface is half duplex RESOLUTION: Check speed/duplex settings on the router/switch port.
	0003000c	MAJOR	Interface speed is 10 Mbps RESOLUTION: • Check speed/duplex settings. • Use a 100/1000 Mbps port on the router/switch.
	00030027	MAJOR	Interfaces have different MTUs. [LAN0 and WAN0]. RESOLUTION: Check interface MTU settings on lan0/wan0(pairwise) on dual bridge mode and lan0/lan1/wan0/wan1... on single bridge mode.
	00030028	MAJOR	Interfaces have different MTUs. [LAN1 and WAN1]. RESOLUTION: Check interface MTU settings on lan1/wan1 or tlan1/twan1 interfaces.
Equipment (cont.)	00030022	MAJOR	LAN next-hop unreachable2 RESOLUTION: Check appliance configuration: • LAN–side next-hop IP • Appliance IP / Mask • VLAN IP / Mask • VLAN ID
	0003001a	MAJOR	LAN/WAN interface has been shut down due to link propagation of paired interface RESOLUTION: Check cables and connectivity. For example, if lan0 is shut down, check why wan0 is down. Applicable only to in-line (bridge) mode.
	00030018	MAJOR	LAN/WAN interfaces have different admin states RESOLUTION: Check interface admin configuration for lan0/wan0 (and lan1/wan1). Applicable only to in-line mode.
	00030019	MAJOR	LAN/WAN interfaces have different link carrier states RESOLUTION: Check interface configured speed settings and current values (an0/wan0, lan1/wan1). Applicable only to in-line mode.
	0003000a	MAJOR	Management interface link down RESOLUTION: • Check cables. • Check interface admin status on the router.
	00030009	MAJOR	Network interface link down RESOLUTION: Is the system in Bypass mode? • Check cables. • Check interface admin status on the router.
	00030020	MAJOR	Power supply not connected, not powered, or failed RESOLUTION: • Connect to a power outlet. • Check power cable connectivity.
	0003002c	MAJOR	System optimization disabled RESOLUTION: Turn on system optimization.
	00030023	MAJOR	Unexpected system restart RESOLUTION: Power issues? Was the appliance shutdown ungracefully? Contact Customer Service if the shutdown was not planned.
	00030012	MAJOR	VRRP instance is down RESOLUTION: Check the interface. Is the link down?
Equipment (cont.)	00030014	MAJOR	WAN next-hop router discovered on a LAN port (box is in backwards) RESOLUTION: • Check WAN next-hop IP address. • Check lan0 and wan0 cabling (in-line mode only). • If it cannot be resolved, call Customer Service.
	00030011	MAJOR	WAN next-hop unreachableb RESOLUTION: • Check cables on Silver Peak appliance and router. • Check IP/mask on Silver Peak appliance and router. Next-hop should be only a single IP hop away. • To troubleshoot, use: show cdp neighbor, show arp, and ping -I <appliance IP> <next-hop IP>.
	0003001e	MAJOR	WCCP adjacency(ies) down RESOLUTION: Cannot establish WCCP neighbor: • Check WCCP configuration on appliance and router. • Verify reachability. • Enable debugging on router: debug ip wccp packet
	0003001f	MAJOR	WCCP assignment table mismatch RESOLUTION: Check WCCP mask/hash assignment configuration on all Silver Peak appliances and ensure that they match.
	00030002	MINOR	Disk is degraded RESOLUTION: Wait for disk to recover. If it does not recover, contact Customer Service.
	00030016	MINOR	Disk is rebuilding RESOLUTION: Normal. If rebuilding is unsuccessful, contact Customer Service.
	0003001b	MINOR	Disk SMART threshold exceeded RESOLUTION: Contact Customer Service to replace disk.
	0003002d	MINOR	Non-optimal configured memory size for this virtual appliance RESOLUTION: Assign more memory to the virtual machine and restart the appliance. Traffic will be sub-optimal until this is resolved.
	0003002e	MINOR	Non-optimal configured processor count for this virtual appliance RESOLUTION: Assign more processors to the virtual machine and restart the appliance. Traffic will be sub-optimal until this is resolved.
Equipment (cont.)	0003002f	MINOR	Non-optimal configured disk storage for this virtual appliance RESOLUTION: Assign more storage to the virtual machine and restart the appliance. Traffic will be sub-optimal until this is resolved.
	00030008	WARNING	Network interface admin down RESOLUTION: Check Silver Peak interface configuration.
	00030013	WARNING	VRRP state changed from Master to Backup RESOLUTION: VRRP state has changed from Master to Backup. • Check VRRP Master for uptime. • Check VRRP Master for connectivity.
Threshold Crossing Alerts (TCAs)	00050001	WARNING	The average WAN–side transmit throughput of X Mbps over the last minute [exceeded, fell below] the threshold of Y Mbps RESOLUTION: User configured. Check bandwidth reports for tunnel bandwidth.
	00050002	WARNING	The average LAN–side receive throughput of X Mbps over the last minute [exceeded, fell below] the threshold of Y Mbps RESOLUTION: User configured. Check bandwidth reports.
	00050003	WARNING	The total number of X optimized flows at the end of the last minute [exceeded, fell below] the threshold of Y RESOLUTION: User configured. Check flow and real-time connection reports.
	00050004	WARNING	The total number of X flows at the end of the last minute [exceeded, fell below] the threshold of Y RESOLUTION: User configured. Check flow and real-time connection reports.
	00050005	WARNING	The file system utilization of X% at the end of the last minute [exceeded, fell below] the threshold of Y RESOLUTION: Contact Customer Service.
	00050006	WARNING	The peak latency of X during the last minute [exceeded, fell below] the threshold of Y RESOLUTION: User configured. • Check Latency Reports. If latency is too high, check routing between the appliances and QoS policy on upstream routers. • Check tunnel DSCP marking. If latency persists, contact ISP and Silver Peak support.
Threshold Crossing Alerts (TCAs) (cont.)	00050007	WARNING	The average pre-FEC loss of X% over the last minute [exceeded, fell below] the threshold of Y% RESOLUTION: User configured. • Check Loss Reports. • Check for loss between Silver Peak appliances (interface counters on upstream routers). • Use network bandwidth measurement tools such as iperf to measure loss. • Contact ISP (Internet Service Provider).
	00050008	WARNING	The average post-FEC loss of X% over the last minute [exceeded, fell below] the threshold of Y% RESOLUTION: User configured. • Check Loss Reports. • Check for loss between Silver Peak appliances (interface counters on upstream routers). • Use network bandwidth measurement tools such as iperf to measure loss. • Enable/Adjust Silver Peak Forward Error Correction (FEC). • Contact ISP (Internet Service Provider).
	00050009	WARNING	The average pre-POC out-of-order packets of X% over the last minute [exceeded, fell below] the threshold of Y% RESOLUTION: User configured. • Check Out-of-Order Packets Reports. Normal in a network with multiple paths and different QoS queues. Normal in a dual-homed router or 4-port in-line [bridge] configuration. • Contact Customer Service if out-of-order packets are not 100% corrected.
	0005000a	WARNING	The average post-POC out-of-order packets of X% over the last minute [exceeded, fell below] the threshold of Y% RESOLUTION: User configured. • Check Out-of-Order Packets Reports. Normal in a network with multiple paths and different QoS queues. Normal in a dual-homed router or 4-port in-line [bridge] configuration. • Contact Customer Service if out-of-order packets are not 100% corrected.
	0005000b	WARNING	The average tunnel utilization of X% over the last minute [exceeded, fell below] the threshold of Y% RESOLUTION: User configured. Check bandwidth reports for tunnel bandwidth utilization.
Threshold Crossing Alerts (TCAs) (cont.)	0005000c	WARNING	The average tunnel reduction of X% over the last minute [exceeded, fell below] the threshold of Y% RESOLUTION: User configured. • Check bandwidth reports for deduplication. • Check if the traffic is pre-compressed or encrypted.
	0005000d	WARNING	The total number of flows <num-of-flows> is approaching the capacity of this appliance. Once the capacity is exceeded, new flows will be <dropped|bypassed>. RESOLUTION: If this condition persists, a larger appliance will be necessary to fully optimize all flows.

---

1

The VX appliances are a family of virtual appliances, comprised of the VX-n000 software, an appropriately paired hypervisor and server, and a valid software license.

2

If there is either a LAN Next-Hop Unreachable or WAN Next-Hop Unreachable alarm, resolve the alarm(s) immediately by configuring the gateway(s) to respond to ICMP pings from the Silver Peak appliance IP Address.

Please send comments or suggestions regarding user documentation to techpubs@silver-peak.com.