Service Activation Testing (SAT) standards have been in place for many years. The classical RFC2544, for instance, was already defined in 1999 and its successor Y.1564 was standardized in 2011. Since then they certainly have been used for verification of network performance and quality, but it is now with the recent focus on user experience that the usage of these test methodologies has exploded. Service providers want to proactively ensure that the connectivity to a business site or base station is in order and expectations are met before going live. In this blog, we introduce the key methodologies for SAT and show some use cases.
Understanding Service Activation Testing
SAT is a series of tests to verify the performance, functionality, and reliability of a network service. These tests are typically performed on a live network for site verification. The goal is to identify and rectify any issues before the service is transferred to production.
There are multiple drivers for SAT. Solving network performance issues is the most obvious one, but nowadays proactively verifying business services and mobile site roll-outs are also becoming increasingly important. Also, regulatory bodies are requiring more and more reporting of KPIs related to user experience, and some of those KPIs are related to SAT.
Finally, there are some replacement use cases where an end-of-life solution needs to be replaced with a more modern solution or manual RFC2544 testing solutions with handheld devices are replaced with automated end-to-end test solutions. The drivers are summarized below:
Key Testing Methodologies
RFC 2544 is the classic version and is widely used for testing packet-switched data networks. It provides a framework for measuring network performance metrics such as throughput, packet loss, and latency:
- Throughput tests measure the maximum data rate that can be sustained between two network endpoints.
- Latency tests evaluate the delay experienced by packets as they traverse the network.
- Frame loss tests assess the percentage of packets lost during transmission.
- Back-to-back frames measure the burst tolerance of the network
The strength of RFC2544 is that it is a well-established and widely supported standard and provides a comprehensive set of tests for basic network performance. The limitations are that it may not be sufficient for complex network scenarios, especially those with multiple traffic flows and Quality of Service (QoS) requirements. Its use can also be considered time-consuming to configure and implement.
Y.1564 is an ITU-T standard for testing IP and Ethernet networks. It provides a comprehensive set of tests to assess the performance of networks, including both voice and data services. Key tests include:
- Configuration tests: Check the performance of each individual flow, including latency, jitter and frame loss ratio KPIs as well as sent and received data rates.
- Performance verification test: Recheck the performance of the aggregate flows by looking at latency, jitter, and frame loss ratio as well as sent and received data rates.
The strengths of Y.1564 are that it has a comprehensive suite of tests for Ethernet and IP networks and supports a wide range of service types, including voice, data, and video. It can be used to test complex service mix scenarios, including those with multiple traffic flows and varying QoS requirements. Y.1564 can be more complex to configure and implement than RFC 2544 unless the supporting management system has extensive support for templating or similar techniques to hide the complexity.
iPerf3 is a popular network performance testing tool that measures the maximum achievable bandwidth on TCP and UDP. The key tests of iPerf3 are:
- Bandwidth Tests: Measure the maximum achievable data transfer rate.
- Latency Tests: Evaluate the round-trip time of packets.
- Packet loss/retransmits: Measure the amounts of packets lost (UDP tests) or the amount of retransmits (TCP tests).
The strengths of iPerf3 are that it is easy to use and configure, and provides accurate and reliable performance measurements. It can be used for both basic and advanced network test scenarios. The iPerf3 test is not as comprehensive as RFC 2544 or Y.1564 and is not suitable for testing complex network scenarios with multiple traffic flows and QoS requirements.
Below is a summary of the different service activation tests and throughput tests:
Final Notes on RFC2544 and Y.1564
RFC2544 is the classic throughput testing protocol. It was originally designed for testing device throughput, but it turned out that the protocol was also very good for testing throughput in real networks. However, the lack for true SLA verification was the protocol’s drawback. It has been replaced by Y.1564, which has very good functionality for verifying SLAs, both at the individual flow level and for running all necessary flows in parallel to verify the performance for the aggregated flows. See below for a comparison of the two protocols.
Conclusion
SAT is a critical component of network service deployment and maintenance. By using a combination of RFC 2544, Y.1564, and iPerf3, network engineers can comprehensively evaluate the performance and reliability of their networks.
A well-executed SAT test can help to:
- Identify and resolve network issues before they impact end-users.
- Optimize network performance by identifying bottlenecks and inefficiencies.
- Ensure new services are successfully deployed and meet performance expectations.
- Validate network changes and ensure that they do not negatively impact service quality.
By understanding the strengths and limitations of each test methodology and tailoring the test approach to specific network requirements, network engineers can effectively utilize SAT to deliver high-quality network services with excellent user experience.