Key highlights:
- A network hardening programme to eliminate common causes of failure such as power outages and backhaul failures.
- Close and professional vendor management to coordinate and manage the work of multiple suppliers to the NSW Telco Authority.
- Operational processes run to the strict ITIL standard to drive best practice and continuous service improvement.
- Major incident support to maintain critical communications during the largest emergency events.
Running one of the world’s largest critical communications networks
The Australian state of New South Wales (NSW) has one of the world’s largest public safety radio networks, covering 357,000 square kilometres. 50 emergency services agencies, including ambulance, police, and fire and rescue forces, as well as transport, energy and environment service providers, rely on the network, using around 42,000 terminals to stay in touch.
Keeping the Government Radio Network (GRN) accessible at all times is a big task, one that the NSW Telco Authority (responsible for the overall coordination of radio telecommunication services for the NSW Government) has entrusted to BAI. In August 2013, BAI took over the operations and maintenance of the GRN, which is based on the P25 trunked radio standard to provide voice communications, narrowband data services and remote radio configuration to its users.
The Network Operations Control Centre (NOCC) is the hub of the GRN, where BAI controls and monitors the network, activates individual services, locates faults and restores services. Support for users is available through a helpdesk operating 24 hours a day, seven days a week. The network currently involves BAI managing over 200 sites, which handle on average some 35,000 voice calls and over 100,000 data calls daily.
Typically, the NOCC will see more than 35,000 system alarms (‘events’) per day, of which about 750 are critical alarms. In addition, some 3,000 events are logged per day from the IP/MPLS overlay network. We handle approx 100 requests for changes to terminal information and averaging over 150 changes to the boarder network every month.
Strong management tools and processes
BAI is considered a key strategic partner by the NSW Telco Authority. In this role, BAI is responsible for managing several other suppliers that have contracts with the NSW Telco Authority including those providing network infrastructure and power. The relationships with all vendors have developed so well that today, BAI needs to spend only minimal time on vendor management, enabling it to apply more resources on maintaining network quality.
BAI also follows ITIL (Information Technology Infrastructure Library) a set of processes, procedures, tasks and checklists for service management that underpins ISO27001 and supports consistent and high-quality service delivery in major projects with multiple stakeholders. The key ITIL processes for BAI are incident, problem, change and service management, which deliver a structured framework for best practice, good behaviour and continuous service improvement. The framework enables us to demonstrate compliance and measure improvement. This was a step change in the quality of process and procedures used to run and operate the Critical communications network resulting in high network availability and improved reporting to the NSW Telco Authority.
Maintaining the highest network availability
The NSW Telco Authority’s decision to award BAI the GRN contract followed a fierce competitive tendering process. Immediately following contract award BAI earned the strong reputation as a company that reacts positively, responds instantly and takes whatever action is needed to maintain services. This was a key factor in the decision to extend the original contract term (3+1+1) in 2017 by a further three years.
When BAI took over the GRN it immediately focused on improving its availability by upgrading base station power systems and adding redundant remote links to reduce the risk of service downtime. Most battery systems on sites have been replaced to support 10 hours of service, with many extending well beyond 24 hours of running time. Previously, when a power outage occurred at a site, its battery back-up would typically fail within an hour, taking the site off-line. BAI has also ensured that every site can accept an emergency generator that can be plugged in quickly if required. In addition, BAI has supported an upgrade to the network’s backhaul by implementing an IP/MPLS overlay with 195 MPLS routers and rolling out 180 LTE routers at nearly every site to provide redundancy in the links. Each site now has immediate fall back to a mobile carrier.
Reacting to major incidents
BAI’s dedication is often most visible during major incidents, principally large area fires and flooding. Large fires where the GRN network has no coverage can be a challenge but one that BAI solves with Cells on Wheels, which are mobile sites that can provide temporary coverage where needed. Floods can often cause power failures that take sites offline. In April 2015 several GRN locations were affected by floods and BAI deployed 14 generators within a few hours and they remained in place for nine days.
Future partnership
Currently the GRN network is being extended under the NSW Telco Authority’s Critical Communications Enhancement Program (CCEP) to consolidate several legacy radio networks and increase the shared coverage available to agencies to more than 80% of the state (an increase from what is currently only a third of the state’s geographical area). Coverage in urban areas will improve from 96% to close to 100%. The CCEP will provide seamless communications with talkgroups across the state for all agencies, which is currently not possible.
BAI is heavily involved in the integration and testing of new sites into the network and responsible for operational acceptance on behalf of the NSW Telco Authority. In 2018 the first phase of the CCEP rollout statewide commenced, upgrading almost 90 sites across the north coast region of NSW covering an area of approximately 46,000 km2 .
Having worked proactively to substantially improve the network’s availability and prepare it for the future, BAI is well placed to support the GRN in the coming years as it expands and provides new capabilities to the emergency services.