TPO smooth migration

Top  Previous  Next
IT&Operations

 

TableCornerTopLeftB

IPT_Empty

IPT_Empty

TableCornerTopRightB

IPT_Empty

Important_icon

The following procedure can only be applied when upgrading from a Package Day/Maintenance Package to another Package Day/Maintenance Package within the same Major/Minor Release

IPT_Empty

TableCornerBottomLeftB

IPT_Empty

IPT_Empty

TableCornerBottomRightB

 

Within a 24/7 command and control environment, service interruption for maintenance reason is never easy.

Obtaining a maintenance window in such environment is a very difficult process that jeopardize the service provided to external world and must in any case be reduced as much as possible.

 

TSS new version can be installed in parallel of others TSS version without service disruption and anyway TSS is not in the critical path of day to day users activities.

Turrets can be upgraded group by group or even unit by unit allowing another group/unit to deal with the traffic handled by the turret rebooting for upgrade purpose.

However for central real time components like TPO this is not as simple even with a redundant TPO cluster can handle TPO lost almost instantly.

 

The procedure here under explain how to deal with TPO upgrade activities in such critical environment.

This procedure is valid starting from a 9.1 installed version and after. As it relies on TPO hot backup capabilities, end users can still experience a few seconds glitch necessary to switch from one TPO to another but this would be very light compared to a service interruption.

 

This procedure is made in 3 main steps: preparation, passive TPO nodes upgrade, active TPO nodes upgrade

Preparation phase

 

1.Duplicate the Zone that contains the TPOs (original zone will be called OldZone, copy will be called NewZone)

 

clip0008

 

2.In NewZone, set the TPO bootstrap/firmware to the new version you want to install

 

clip0009

Passive TPO upgrade

 

3.Go to Device Management > TPO Clusters > Cluster_To_Upgrade

 

4.Go to TPO Cluster tab

 

5.Click on a passive node

 

6.In Zone Group, select NewZone, then save the change by clicking on Update

 

7.Go to System > Sessions > Device Sessions

8.Click on the shutdown button next to the TPO from step 5, then on OK clip0010From this step 8 to step 10, this passive TPO is not operational as it is upgrading and thus will not be able to recover any active TPO during this upgrade process.

 

9.Go to Device Management > TPO Clusters > Cluster_To_Upgrade

 

10.Wait for the TPO to be upgraded, and its current role to be passive again. Passive TPO coming back need some time to "learn" from active TPOs before being fully operational. On fully loaded TPO, this can take 30 mins. Administrator can insure that a passive TPO coming back from scratch is fully operational seeking in TPO syslog after those codes (those code must be there for each active nodes the passive is supposed to receive information as soon it is reconnected). a, c, e,g are purely informational. b,d,f,h must be there to consider the node as operational.

 

a.HBA!I0001 - The passive node receives the Start signaling of the calls re-transmission information from the active node.

 

b.HBA!I0002 - The passive node receives the End signaling of the calls re-transmission information from the active node.

 

c.HBA!I0003 - The passive node receives the Start signaling of the DDI Slots bridging re-transmission information from the active node.

 

d.HBA!I0004 - The passive node receives the End signaling of the DDI Slots bridging re-transmission information from the active node.

 

e.HBA!I0005 - The passive node receives the Start signaling of the DDI line subscriptions re-transmission information from the active node.

 

f.HBA!I0006 - The passive node receives the End signaling of the DDI line subscriptions re-transmission information from the active node.

 

g.HBA!I0007 - The passive node receives the Start signaling of the SIP registrations re-transmission information from the active node.

 

h.HBA!I0008 - The passive node receives the End signaling of the SIP registrations re-transmission information from the active node.

 

11.Repeat steps 5 to 10 with all passive TPO

 

Active TPO Upgrade

12.At this point passive TPO are upgraded and ready to take ownership of any active TPO. Go to Device Management > TPO Clusters > Cluster_To_Upgrade

 

13.Go to TPO Cluster tab

 

14.Select an active node

 

15.In Zone Group, select NewZone, then save the change by clicking on Update

 

16.Go to System > Sessions > Device Sessions

17.Click on the shutdown button next to the TPO from step 14, then on OK clip0010

 

18.Wait for the TPO to come back, and its current role to be passive. Passive TPO coming back need some time to "learn" from active TPOs before being fully operational. On fully loaded TPO, this can take 30 mins. Administrator can insure that a passive TPO coming back from scratch is fully operational seeking in TPO syslog after those codes (those code must be there for each active nodes the passive is supposed to receive information as soon it is reconnected). a, c, e,g are purely informational. b,d,f,h must be there to consider the node as operational.

 

a.HBA!I0001 - The passive node receives the Start signaling of the calls re-transmission information from the active node.

 

b.HBA!I0002 - The passive node receives the End signaling of the calls re-transmission information from the active node.

 

c.HBA!I0003 - The passive node receives the Start signaling of the DDI Slots bridging re-transmission information from the active node.

 

d.HBA!I0004 - The passive node receives the End signaling of the DDI Slots bridging re-transmission information from the active node.

 

e.HBA!I0005 - The passive node receives the Start signaling of the DDI line subscriptions re-transmission information from the active node.

 

f.HBA!I0006 - The passive node receives the End signaling of the DDI line subscriptions re-transmission information from the active node.

 

g.HBA!I0007 - The passive node receives the Start signaling of the SIP registrations re-transmission information from the active node.

 

h.HBA!I0008 - The passive node receives the End signaling of the SIP registrations re-transmission information from the active node.

 

19.Repeat steps 14 to 28 with all active TPO

 

 

At this point, the cluster is fully upgraded and operational however initial active nodes are now passive and initial passive nodes are now active. Administrator can optionally proceed to another active node reset in order to bring back active node at their initial assignation. Notice that again, the active node becoming passive will then need time to "learn" the situation when coming back alive.