Toufic Kourbeh

How to measure real CX -Customer Experience- across and in wireless technologies?

Disclaimer: The opinions and ideas expressed in this article are my own and do not necessarily represent those of my employer.

The Internet has become a tool used across the globe to communicate, learn, teach and conduct businesses. The Wireless for Communities model (W4C) empowered a holistic approach to communication worldwide by empowering communities everywhere with digital skills and tools. By building wireless networks managed by the communities to secure the community’s benefits for optimal internet connectivity. Most wireless connectivity technologies like UMTS, LTE, and 5G NR are expensive mainly because of the licensed bandwidth. Communities needed something faster, cheaper, and more dynamic. Thus new carrier-grade services emerged using the unlicensed radio spectrum. The most notable technology is a non-3GPP technology based on IEEE 802.11 equipment we all know as Wi-Fi. Wi-Fi uses free, unlicensed RF bandwidth, which makes it inexpensive. Therefore under-served communities worldwide are now connected and empowered at the same time using this cheap technology. Mobile phone data offloading motivated many service providers to build their massive Wi-Fi networks. However, some challenges remain. While the technical issues of deploying 802.11 wireless networks and mesh wireless networks have been mainly solved, there remains the need for significant work on identifying quality and customer experience and mobility challenges, especially when compared with the other 3GPP technologies.

Many RF planning and analysis tools for Wi-Fi from companies such as iBwave, Air magnet, ASSET, and AirCom have become more and more efficient and sophisticated in building enhanced RF designs and deployment strategies. However, these tools are still expensive and dependent on the human element for surveying, deployment, and testing. Moreover, there is a missing element: the performance monitoring for RF across multiple technologies in the same network and what the customer is experiencing, not to mention detecting mobility issues after deployment. While some vendors like Ericsson, Nokia, Cisco, and many others have provided tools and solutions to monitor and control their Radio Access nodes, due to the limitation of some technologies such as 802.11, the user equipment cannot provide QoS information on Radio Links for the downlink on the UE side to the AP (unlike other wireless technologies like LTE, UMTS, GSM, and others) which create a gap in assessing the customer experience in our upcoming world of wireless convergence.

I think we all agree that the most crucial factor in the entire network lifecycle is customer experience (Cx) in any wireless network, whether in public venues such as stadiums, shopping malls, cafes, or the carrier-grade WiFi networks /and I will be focusing on WiFi/, “slow network” is a common complaint that dents the customer satisfaction of the service. At such sites, where several competing providers of WiFi services, a “slow network” rating hurts the NPC approval rates of a business, service, or brand value.

While ITU-T developed MOS for wired networks, some vendors tried to use Wireless-MOS as an indicator. The main problem is that no one agreed on what Wireless MOS means. Attempts to define WMOS -by some vendors- based on Physical layer status for a session or connection or even the number of retransmissions on Layer 4 is not adequate. ITU-T failed to put an early clear definition for wireless MOS, which made it mission impossible to define it properly in such an uncontrolled environment and space. While 3GPP filled the gaps in UMTS, LTE, and 5GNR, 802.11 technology still has a big gap to fill.

So, how does one look at wireless technology from the viewpoint of optimal customer experience? How does one examine the customer end-to-end viewpoint along the entire network connectivity lifecycle (3GPP and non-3GPP)? How does one measure the customer satisfaction of a service?

Let me put this question differently. What is the best way to measure customer experience across multiple wireless technologies such as Wi-Fi, CBRS, LTE, and 5G to compare them or score an overall user experience?

What is the best way to measure customer experience across multiple wireless technologies such as Wi-Fi, CBRS, LTE, and 5G to compare them or score an overall user experience?

Google’s Site, Reliability Engineering book, released in 2016, the authors mentioned the factors of the site reliability, laying the foundations for a clear definition of Cx. I will be using that concept in my approach by prescribing the four golden parameters that Google uses as the cornerstones of our Cx, which also made me think that we can use the same concept to measure any network performance wired or wireless from the customer point of view, these cornerstones are latency, traffic, errors, and saturation.

I will be starting by implementing this concept to measure Cx for the Wi-Fi network. Then it can be expanded to other networks (LTE and CBRS). so I will be focusing on the customer touchpoints along any wireless network lifecycle

1. Network discovery and availability: Assuming the service is present and available, network discovery depends heavily on the protocols defined in the UE (User Equipment). Service providers use design tools standards calibrated to work with various ranges of user equipment, such as the iBwave. The robust network design will include best practices of UE hardware and protocols standards as Apple iPhones and Samsung Notes at the network end.

2. Connection: Wi-Fi alliance or WFA designed open and secure network mechanisms to connect to Wi-Fi networks. At the same time, most commercial networks use an available network with a captive portal authentication. A transformation occurs in wireless network implementations with a shift towards Passpoint 2.0 (or sometimes called Hotspot 2.0) as a secure and reliable method. The WFA introduced the WPA3-Enterprise security protocol in June 2018, which offers the equivalent of a 192-bit cryptographic strength, utilizing Simultaneous Authentication of Equals (SAE) handshake as defined in the IEEE802.11s standard.

3. Onboarding: Onboarding users begin with a probe request to establish L2 connectivity with the network and end with the first data packet received from the internet. This passes through simple to complicated nodes, and of course, the business captive portal if exists.

4. Saturation: Several factors affect Wi-Fi service saturation. The conditions of the RF, different HW components specs and capabilities, and the servers that provide the service. A high interference from an RF source (like SNR: Signal – to – Noise ratio and CCI: CO-Channel interference) can negatively impact the modulation schemes and consequently affect the quality of the physical layer. The Access point / Switch /Gateway condition affects the transport layer and, therefore, the service, for example, firmware bugs, CPU, and memory utilization of the Access point. And the precise AAA / DNS / DHCP resource planning can positively or negatively impact the service delays. Bandwidth problems cause network backbone delays. Also, on the service provider side, wiring and cabling issues can have a tremendous impact. Finally, the status of the destination application hosts provides the application such as a website or email client. Any of these components can impact the saturation and cause a poor user experience.

The challenge we are addressing in this article is to define the best indicator that can describe all of the challenges at different components and stages. For that, what we need is an agent that can save time, count, measure, collect and send us this information, taking into consideration various User Equipment (UE) environments used by the user such as computers with different Operating systems, mobile phones, tablets, and even IoT devices.

A software agent is built and installed on the User Equipment or UEs (such as Windows OS, Linux OS, macOS, Android, and iOS) or a code injected in a specific free or commercial APP to collect a set of measurements considered; as our “Probe.” A probe may also include specially built IoT devices running a light version of Linux OS.

The probe will be collecting measurements from our wireless networks based on the four golden rules of site reliability. These values can be anything we determine it is vital to our user for. The next table will be a data model for what a probe could collect from every session:

What is defined as the measurement in the table above is a mere example! The flexibility of the formula allows the addition of any required measurement. Additional different counters, timers, or metrics can be determined by the customer experience teams for specific applications or protocols such as Adaptive video bitrate, RTP, and Concealment metrics for Voice. Please notice that these measurements should be geographically and time-stamped.

The next step is to define the appropriate satisfactory values for every measurement collected by the probe. This process is very flexible. For example, a latency of 50ms might be acceptable for a user watching a movie but not for a user using a VOIP application such as Skype, WhatsApp, or IMO.

Once that process is done, we define an “artificial weight” for each of the measurements we are collecting from the probe. This weight will rank the values based on importance to our calculation. To explain this further, I will use the following example.

In Wi-Fi networks, We prefer a 5GHz band over the 2.4GHz band. The 5GHz band has more non-overlapping channels than the 2.4GHz band, and it is known that due to power, SNR and Spectrum, the 5GHz band has fewer interference problems than the 2.4GHz.

Now that we have our measurements collected and ready, we need to process these values to get our metric point for every session. This is a relatively straightforward process; by implementing common statistical principles, we can identify a score for every session the UE is generating.

Data collected from the probes can also generate a heatmap representing Cx. Such heat maps can mean either a service we measure a detailed or the overall user experience:

This is the first step to build an idea about a user experiencing the Wi-Fi network. Assuming we could run this data model for Wi-Fi, CBRS, LTE, and 5GNR network, we can get an overall ratio for each technology and understand how our customers think of our network.

The method of weighting and rating measured values is a proven way to measure Cx, quantify and justify network resources allocated to each user and build a practical customer happiness index based on the real-life network deployment in a cost-effective method that serves the business goals. Data accuracy and the calculation is significant in understanding Cx, a service that is invaluable in determining if the offered service is desirable or not!