Algorithm precisely quantifies flow of information in complex networks

Networks are systems comprised of two or more connected devices, biological organisms or other components, which typically share information with each other. Understanding how information moves between these connected components, also known as nodes, could help to advance research focusing on numerous topics, ranging from artificial intelligence (AI) to neuroscience.

To measure the directional flow of information in systems, scientists typically rely on a mathematical construct known as transfer entropy, which essentially quantifies the rate at which information is transmitted from one node to another. Yet most strategies for calculating transfer entropy developed so far rely on approximations, which significantly limits their accuracy and reliability.

Researchers at AMOLF, a fundamental physics institute in the Netherlands, recently developed a computational algorithm that can precisely quantify transfer entropy in a wide range of complex networks. Their algorithm, introduced in a paper published in Physical Review Letters, opens new exciting possibilities for the study of information transfer in both biological and engineered networks.

“Our daily life relies on the smooth functioning of a myriad of complex networks,” Avishek Das, co-author of the paper, told Phys.org. “Typical examples range over many scales: from the internet, financial markets, ecosystems, the human brain, to the thousands of mutually reacting chemicals inside a single biological cell. The unifying feature of these networks is that they process external signals to give intelligent outputs.”

A key objective of the recent research by Das and his colleague Pieter Rein ten Wolde was to devise a reliable method to understand and control information processing in complex systems. A first step toward this goal is to reliably measure transfer entropy.

“This directional flow of information could not be measured in general network models until now without unpredictable errors,” said Das. “Our paper introduces a computational algorithm, TE-PWS, to quantify it exactly for the first time.”

Quantifying the rate at which information is transferred from one node to another essentially entails counting simultaneous fluctuations in the two nodes and framing them as a function of time. A key difficulty encountered when attempting to do this is that fluctuations in nodes are too rare to be captured by conventional simulation strategies.

“Our breakthrough came from borrowing a method commonly used in statistical physics called importance sampling, which makes rare fluctuations occur more frequently in simulations,” explained Das. “TE-PWS uses this to count rare fluctuations accurately, giving the exact transfer entropy for any model.”

A notable advantage of the algorithm developed by Das and ten Wolde is that it can be applied to a wide range of networks. In fact, the algorithm works in the presence of an arbitrary amount of nonlinearity and feedback in the network, successfully quantifying transfer entropy in instances where all other methods fail.

“In our study, we used TE-PWS to show that strong feedback can counterintuitively amplify the feedforward information transfer to faraway nodes in a network,” said Das. “We also found that TE-PWS uses either comparable or less computer time than other methods, making it both accurate and cheap.”

In initial tests, the computing technique for measuring transfer entropy was found to be highly accurate, outperforming other methods developed in the past. Since this technique does not rely on any approximations, it yields ground-truth results, which are essential when trying to rigorously test the accuracy of physics, network science or neuroscience theories.

“We used our exact technique to test the accuracy of other methods, and find that they are often inaccurate, and, moreover, less efficient,” said Das. “Without the exact answer from TE-PWS, we would never know how large of an error the other methods make. To date, TE-PWS is the only reliable method for a general network.”

The recent work by Das and ten Wolde could soon pave the way for new studies investigating the transfer of information in AI systems, communication systems, financial networks, ecological systems and biological neural networks. As the algorithm they developed is both precise and computationally light, it could be applied to a wide range of complex and large networks.

“We now plan to use TE-PWS to measure the information processing inside bacterial cells through chemical signaling networks,” added Das. “Even though bacteria are simple organisms, they perform sophisticated computations like taking integrals and derivatives and finding optima. TE-PWS will help us understand how their signaling networks can do this efficiently.”

© 2025 Science X Network