TetraMem
Game-Changing Technology for AI Computing

Recent breakthroughs in data and algorithms have ignited transformation in the AI computing hardware realm, bringing revolutions to various industries. Healthcare, autonomous vehicles, smart cities, and cybersecurity industries are widely embracing these innovations, but not without their share of challenges.

As Dr. Glenn Ning Ge, CEO of TetraMem says, “Relying on traditional computing methodologies involving separate processing and memory unit leads to von Neumann bottlenecks and memory walls, where the system spends significant time and energy moving data for the matrix-based deep-learning applications. This issue is further aggravated by the surge in data-centric applications, which can result in reduced computation capabilities.”

AI models, particularly deep neural networks, require massive computational power for vector-matrix multiplication accumulation operations. The exponential rise in computing needs calls for huge power consumption at data centers, putting a significant financial toll on most businesses. Simultaneously, intensive computation leads to excessive heat generation, and efficiently dissipating heat is challenging in compact devices without active cooling systems. Scalability and technology node limits also add to existing woes where physical limitations and quantum effects pose a serious challenge to further miniaturization.

Part of the issue lies in hardware-software co-optimization, which is often hindered by the hardships associated with developing compilers, runtime systems, and hardware architectures that work harmoniously.

“Addressing these challenges and developing an environment for AI applications to make a real difference hinges on the combination of innovation in fundamental new material science, emerging devices, semiconductor manufacturing, system architecture, and algorithm design,” says Dr. Ge.

That’s where TetraMem emerges as a leader.

A pioneer in analog in-memory computing, TetraMem is at the forefront of this transformative shift in the AI computing paradigm. Born out of the collective vision of a team with deep expertise and experience in semiconductor design, AI applications, and emerging memory technologies, it delivers an efficient crossbar array-based neural processing unit (NPU) that uses analog in-memory computing technology. Under the leadership of Ge, who holds a PhD and three master's degrees and brings 20 years of industry experience and approximately 800 patent filings globally to his role, the team has developed the first-of-its-kind, purpose-built solution. It is inspired by the human brain’s efficiency and uses the power of emerging nonvolatile memory devices to perform computation directly within the physical memory unit, accelerating AI algorithms. This architectural shift promises unparalleled, several order-of-magnitude improvement in energy efficiency and performance, especially in edge applications where real-time processing is paramount.

The ‘TetraMem Way’ of Spearheading Innovation

TetraMem’s groundbreaking technology offers scalability and energy efficiency beyond any market-leading analog technologies. The TetraMem’s NPU delivers 20–100 TOPS/W at INT8 using mature technology nodes, like 65 nm and 22 nm, and the team expects to gain 300 TOPS/W at more advanced nodes with crossbar sizes scaling up and some of other technical innovations like ADC renovation. The solution’s memory-based computation ability eliminates weight-data movement and elevates the performance of AI or ML workloads. Coupled with massive parallel processing, it guarantees higher throughput than traditional architectures. The technology also enables non-volatile memory devices, where the neural network models are pre-loaded onto the chip to allow data fetching directly from sensors, without weighing down the memory.

TetraMem ingrains all these features into its suite of solutions, comprising an analog in-memory computing chip, IP and customizable design solutions, and software toolkit and support. This analog in-memory computing chip is designed to accelerate AI algorithms, particularly for edge devices. Given its architecture, the chipset consumes significantly less power and provides faster processing times. Currently delivered as the TetraMem® MX100 SoC (system on-chip), an evaluation kit designed for customers to test its capabilities, the chipset is a gateway for numerous groundbreaking innovations that transform how AI models are used. Academic and industry partners and researchers are exploring the earlystage development and research of analog IMC using TetraMem’s hardware and SDK software, giving them the opportunity to delve into this emerging field.

“Our platform isn’t just about the hardware—it’s a six-dimension co-design and co-optimization,” says Dr. Ge. “We consider the entire AI computing ecosystem when we have a chance to re-design the new AI computing system, from materials and devices to algorithms and software, ensuring optimized performance across the board where the emerging computing paradigm meets up with emerging devices”

TetraMem’s hardware solutions are complemented by its comprehensive software development kit: TetraMem Instinct™ software, which facilitates the seamless development and deployment of applications using IMC technology. It also offers standard IP solutions tailored to specific needs, recognizing that different applications have varying requirements. This supports seamless integration of IMC technology into a diverse range of applications, including high-throughput, low-power-consumption devices for eyeball tracking, face detection, and motion detection. The firm’s offerings in AIMC can also be harnessed to accelerate AI algorithms crucial for real-time data processing in medical applications. Rapid analysis of patient data, medical images, or even real-time monitoring can benefit from the computational efficiency and energy savings offered by TetraMem’s solutions.

“This is ‘The Transistor Moment’ for AI computing after decades of von Neumann and transistor resonances,” says Dr. Ge.

Strategic Partnership to Build AI Accelerator Chip

Backed by industry leaders and solid alliances with major semiconductor companies, TetraMem is positioned to validate the technology’s potential for large-scale commercial deployments. For instance, its strategic partnership with Andes Technology, a supplier of high-efficiency, low-power RISC-V processor cores, allows for delivery of a fast, highly efficient AI inference chip. The fusion of Andes’ high-performance RISC-V Vector CPU with TetraMem’s IMC architecture through Andes Custom Extension (ACE) enables the best performance. This amplifies the strengths of both companies, resulting in fast, energy-efficient AI inference that surpasses all limitations of traditional computing approaches and transcends the ‘memory wall’ and ‘end of Moore’s Law’ constraints.

The joint collaboration aims to create a powerful chip that improves energy efficiency by at least an order of magnitude. Optimizing computations and eliminating the transfer of weight data will significantly extend the battery life of edge devices and impose a near-zero impact on thermal budgets.

TetraMem is also venturing to enhance its technology’s potential for flexibility and scalability. It stays true to this mission by designing AI accelerator chips with versatility and scalability from 22 nm to 7 nm and beyond, making it easy to integrate into numerous AI-powered products and applications across multiple industries. The team has demonstrated that its compute memristor can be scaled down to 2 nm and below, carving a roadmap to many future-proof solutions.

Envisioning a Human- Centered World of AI

A team of engineers and domain specialists is at the heart of TetraMem’s excellence, giving it a competitive spirit to drive advancements and stay ahead of the curve in today’s tech landscape teeming with innovators. Their extensive experience and rigorous R&D process enable the firm to constantly push the envelope to ensure that its technology remains at the forefront.

The blend of deep expertise, comprehensive solutions, and a client-centric approach sets TetraMem apart. An unwavering commitment to innovation and excellence ensures that it does not just meet, but exceeds client and partner expectations. For instance, Dr. Ge himself holds a PhD and three master's degrees including a MBA from The University of Michigan Ross School of Business.

“The tech landscape is fast and everevolving, and resting on one’s laurels is not an option. We encourage a culture of continuous learning and innovation, guaranteeing we’re always ahead of the curve,” says Dr. Ge.

A testament to TetraMem’s clientcentricity and continuous innovation is its ability to be attuned to market demands and emerging trends. Through regular interactions with potential clients, tech analysts, and industry leaders, it gauges real-time market needs and actively seeks feedback to understand the challenges businesses face. Coupled with a crossfunctional collaboration approach, the team ensures that its solutions are technologically advanced and a direct response to real-world challenges. Rigorous testing to assess real-world efficiency and applicability is also a priority for TetraMem.

The team focuses on agile principles, continuous development, and regular model updates, all with a commitment to sustainability. Its IMC solutions are energy-efficient, addressing environmental concerns and operational costs associated with data processing.

“We believe that technology must serve humanity. Our goal is not just to enhance machines by adding artificial intelligence but to uplift humanity through emotional intelligence. Empathy and compassion are core values we hold dear, and this is important to machines and robots. Therefore, we strive to infuse humanity into machines and robots,” says Ning Ge. “Our technology enables a world of AI that acts and feels more human.”

Stepping into the Future of Efficient AI Models

The horizon is luminous for TetraMem as the upcoming era of AI computing, spanning the edge and cloud, comes into focus. Its IMC technology is poised to address the pressing challenges of energy efficiency and power constraints in a landscape that demands highly power-conscious computational strategies. Continuing its streak to steer ahead of technological innovations, the firm is building more efficient processors for CPUs and GPUs, capable of running AR, VR, and data centers with less power consumption. The goal is to deliver cuttingedge technologies that operate 10 times more efficiently than a GPU running at 500 watts with less than 50-watt power consumption.

TetraMem envisions unveiling the AI accelerator chip and making engineering samples and development kits for the new 22-nm TetraMem MX series chip available to the public by the end of 2024. The firm is also joining hands with numerous tech leaders and research entities to co-develop environmentally friendly AI solutions, shaping the narrative around green AI.

It’s safe to say that TetraMem is more than just a silicon chip vendor. It is a partner for companies looking to harness the next wave of computing advancements. All of its solutions are crafted with the future in mind, ensuring that its partners are always ahead in the rapidly evolving technological landscape.