As concepts like intent-based networking, automation networks are progressed, need to monitor networks in near real-time and state becomes increasingly important. If you look at Gartner study for Intent-based networking, they speak about four steps. Translation and validation, automatic provisioning, state awareness and assurance. This is where streaming telemetry comes in to play as it provides near real time visibility of network which was not possible with legacy technologies such as SNMP.
One of the fundamentals of an autonomous network would be to sense its environment. Just like how a self-driving car would sense its statistics and surroundings to drive on its own. Similarly, an intent-based network or an autonomous network would need to know about near real-time statistics about its state based on which it can determine if the behavior is as per the intent or deviated. If any deviations are found, network will rearrange its properties to ensure network assurance to the initial intent. Whilst autonomous network is a more futuristic application, on current context digital transformation demands near real time awareness about network state by different stakeholders including DevOps teams to faster provisioning and troubleshooting.
Traditionally SNMP is being used for network monitoring. But as more and more real time data about the state is required, SNMP becomes an inefficient protocol to use in modern day networks.
Most often SNMP based monitoring is also referred as ‘pull’ based monitoring. That is because, SNMP requires devices to be polled in order to retrieve current statistics of a network. Smaller the polling interval, more the amount of statistics you could collect. However, there is a trade-off. SNMP polling is resource intensive, when you reduce the polling interval, it increases the CPU utilization of network devices quite heavily. Therefore, typically the polling interval is kept at a minute or more interval. Typically, within a 5-minute range. This is a limitation when you need to monitor real time state as it only gives the summarized statistics related to the most recent polling interval. For instance, you won’t be able to view exact network state in a per second or less interval. A burst of traffic, packet drops, flipped routes within an interval of few seconds won’t be visible with this technique.
At the same time, SNMP requires OIDs to be included on the monitoring tool to monitor relevant parameters of a devices. And these OIDs must be manually added, creating limitations in terms of scaling especially when the networks are large. Further, network monitoring tools will also have resource concerns when you keep adding more and more devices and more and more OIDs. This becomes a limitation as it impacts the performance of the SNMP based monitoring tool.
Streaming telemetry aims to overcome these limitations and as the name suggest ‘stream’ network state continuously through model driven methodologies. With streaming telemetry network devices stream its state data which can be listened by an application. Streaming telemetry is known as a ‘push’ based monitoring mechanism for above reasons. YANG based data models are being used with streaming telemetry and can be communicated with an application using NETCONF protocol.
Another advantage of streaming telemetry is its ability to stream data ‘on-change’ instead of streaming them continuously. This helps to identify an issue as and when it takes place rather than watching it on a continuous basis.
Your monitoring application or for that matter any application or device that can make use of operational state data, can subscribe once for ‘continuous’ data or ‘on-change’ data and the network device will continue to stream operational data as requested without the need for the monitoring application to keep on requesting.
Almost all leading networking vendors have started using streaming telemetry and the implementation of streaming telemetry on networking devices, data formats used are dependent on vendors. However, most of techniques are standardized and it’s the selection of technique by the vendor that differs.
Maturity and adaptability of streaming telemetry will eventually play a critical role in autonomous networks and its not very far from where we are today. Overall streaming telemetry is well positioned as a much more effective and efficient network monitoring technology which will hopefully be a mainstream in times to come.