The culmination of about two decades of research, IBM’s NorthPole artificial intelligence (AI) accelerator not only represents a major milestone in computing but it may also pave the way for more sustainable AI applications in the future.
According to IBM, NorthPole seamlessly integrates processing units and memory on a single chip, thus eliminating the need for bulky cooling systems and enhancing the technology’s energy efficiency. A digital AI chip for neural inference based on the computation pathways of the human brain, NorthPole takes a different approach to standard chip design, where data is shuffled between memory, processing and other elements.
“Architecturally, NorthPole blurs the boundary between compute and memory,” said lead researcher Dharmendra Modha. “At the level of individual cores, NorthPole appears as memory-near-compute and from outside the chip, at the level of input-output, it appears as an active memory.” This makes NorthPole easy to integrate in systems and significantly reduces load on the host machine, IBM points out.
Eliminating the von Neumann Bottleneck
IBM Research’s new development is significant for several reasons, not the least of which being that it turns traditional semiconductor manufacturing on end. Since the birth of the semiconductor industry, computer chips have primarily followed the same basic structure, with processing units and memory discretely storing the information to be processed.
“While this structure has allowed for simpler designs that have been able to scale well over the decades, it’s created what’s called the von Neumann bottleneck,” IBM explains, “where it takes time and energy to continually shuffle data back and forth between memory, processing and any other devices within a chip.”
Going a step further, Science explains that the processor-centric computer environment typically discerns “memory” from “computer.” Inspired by the organic brain and optimized for inorganic silicon, on the other hand, NorthPole is a neural inference architecture that blurs this boundary by eliminating off-chip memory, intertwining compute with memory on-chip, and appearing externally as an active memory chip.
“NorthPole is a low-precision, massively parallel, densely interconnected, energy-efficient and spatial computing architecture with a co-optimized, high-utilization programming model,” the publication adds. “NorthPole outperforms all prevalent architectures, even those that use more-advanced technology processes.”
Energy Efficient & Inspired by the Human Brain
In IBM Research Shows Off New NorthPole Neural Accelerator, NAND Research’s Steve McDowell explains the profound impact that NorthPole may have on the computer chip industry. The accelerator’s architecture integrates processing and memory on the same chip and minimizes data transfer between components, he writes, resulting in higher energy efficiency, lower latency and improved performance for AI-inference tasks.
According to McDowell, the IBM project was “shrouded in secrecy until recently” and represents a significant milestone in the quest for energy-efficient computing inspired by the human brain.
“This chip is designed to be efficient, easy to integrate into systems and suitable for a wide range of AI applications,” McDowell adds. NorthPole can also process data efficiently without the need for bulky liquid cooling systems, making it suitable for deployment in compact spaces. “Ongoing research efforts aim to explore further innovations and advancements in chip processing technologies,” he writes, “promising even greater efficiency and performance gains.”
