Designers' Forum

Designers’ Forum is conceived as a unique program that shares the design experience and solutions of real product developments among semiconductor designers and EDA academia/developers. The topics discussed in this forum include Next-Generation AI Semiconductor Design, Heterogeneous Integration and Chiplet Design, AI/ML(Machine Learning) for EDA – Current Status and Future Perspectives from Diverse Views, Advanced EDA using AI/ML at Synopsys.

Sessions:

(2F: DF-1) Next-Generation AI Semiconductor Design
(3F: DF-2) Heterogeneous Integration and Chiplet Design
(5E: DF-3) AI/ML for Chip Design and EDA – Current Status and Future Perspectives from Diverse Views
(1F: DF-4) Advanced EDA using AI/ML at Synopsys

Session 2F: DF-1 (January 23, 13:30 - 15:10)

[Next-Generation AI Semiconductor Design]

In this session, during first half, we will invite two AI chip startup companies headquartered in Korea, FuriosaAI and SAPEON, both of whom have advanced AI semiconductor design technology and have shown exceptional performance in MLPerf benchmarks. The first talk will introduce high performance AI chips developed by FuriosaAI for building the programmable, high performance and energy-efficient AI chip for ChatGPT. The second talk will show how to enable AI innovation through SAPEON’s Zero-Touch AI inference system, powered by state-of-the-art semiconductor technology, provides an AI model inference SDK and a cloud-based inference serving platform. Then, during second half, we will invite two memory giant in Korea, Samsung and SK Hynix how to handle a current hot issue of LLM (large language models). The first talk will not only look at current HBM DRAM technologies but also discuss the next generation of HBM DRAM technologies. Also, the recently developed HBM-PIM will be examined, and the next generation of DRAM-PIM will be discussed. The second talk will introduce shorter latency and operating cost effective LLM accelerator using AiM, SK hynix’s PIM, how to resolve the current issue of not providing sufficient performance and energy efficiency.

• 1. Building the programmable, high performance and energy-efficient AI chip for ChatGPT
  

  Joon Ho Baek (CEO, FuriosaAI, Korea)

  With the advent of ChatGPT and generative AI models, the demand for deep learning inference in data centers is exploding. While energy efficiency is important to reduce TCO (total cost of ownership), high performance is also essential to serve large models in production. Hyperscalers, on the other hand, emphasized the importance of programmability and flexibility for inference accelerators to track DNN progress. This talk will introduce high performance AI chips developed by FuriosaAI, designed to tackle all these challenge.

• 2. Enabling AI Innovation through Zero-touch SAPEON AI Inference System
  

  Soojung Ryu (CEO, SAPEON, Korea)

  SAPEON, a leading player in the AI semiconductor industry, has achieved remarkable success in bringing server-grade semiconductors to market through its X220 platform. These semiconductors have gained wide recognition for their exceptional performance in MLPerf benchmarks. SAPEON's Zero-Touch AI Inference System, powered by state-of-the-art semiconductor technology, provides an AI model inference SDK and a cloud-based Inference Serving Platform. This comprehensive solution enables Customer Engineers to perform AI model inference on NPUs with minimal involvement. SAPEON is actively engaged in collaborations with key stakeholders in the AI industry, playing a pivotal role in driving AI innovation forward as we prepare for the widespread adoption of AI inference using our cutting-edge X330 platform.

• 3. Processing-in-Memory in Generative AI Era
  

  Kyomin Sohn (Master, Samsung Electronics, Korea)

  With the advancement of neural networks, particularly the emergence of LLM(large language models), a solution to address memory bottlenecks and improve system energy efficiency is required strongly. Currently, HBM DRAM is the only memory solution to meet high bandwidth requirements. In this talk, we will look at HBM DRAM that are currently actively used and discuss the next generation of HBM DRAM and what technologies are needed. However, the memory bottleneck caused by the Von Neumann architecture is no exception in the case of HBM, so we look at the PIM technology that is actively discussed to overcome this limitation. The concept and implementation cases of the recently developed HBM-PIM will be examined, and the next generation of DRAM-PIM will be discussed.

• 4. AiMX: Cost-effective LLM accelerator using AiM (SK hynix’s PIM)
  

  Euicheol Lim (Fellow, SK hynix, Korea)

  AI chatbot service has been opening up the mainstream market for AI services. But problems seem to exist with considerably higher operating costs and substantially longer service latency. As the LLM size continued to increase, memory intensive function takes up most of the service operation. That’s why even latest GPU system does not provide sufficient performance and energy efficiency. To resolve it, we are introducing shorter latency and operating cost effective LLM accelerator using AiM (SK hynix’s PIM) We’d like to introduce how to reduce service latency and decrease energy consumption through AiM, as well as explain the architecture of AiMX, an accelerator using AiM. Please come and see for yourself that AiM is no longer a future technology, but can be deployed to the existing system right now.

Session 3F: DF-2 (January 23, 15:30 - 17:10)

[Heterogeneous Integration and Chiplet Design]

As semiconductor scaling technology approaches its limits, Heterogeneous Integration technology, which integrates various electronic subsystems (chiplets) into a mega-scale SoC in a 3D structure, is gaining attention. While heterogeneous Integration is expected to overcome the technical and economic challenges faced by current semiconductor technology, its design must consider various factors not addressed by traditional 2D SoC design methodologies. In this session, we will extensively explore points of consideration related to heterogeneous integration and chiplet design. The first presentations will provide a comprehensive overview of the technology detail of heterogeneous integration including the process issues related to 3D Integration. The second presentation will focus on thermal issues related to 3D Integration and introduce the applications where 3D integration can be effectively employed. The third and fourth presentations will cover the establishment and modeling of electrical interconnect systems between heterogeneous chiplets based on unified standards.

• 1. Co-Design Considerations of Heterogeneous Integrated Packaging
  

  Gu-Sung Kim (Professor, Kangnam University, Korea)

  Modern Semiconductor technology is a position representing competitiveness among countries. Due to the preoccupation of semiconductor front-end technology by a few countries, most of the countries that were left out of ranks are strengthening support for semiconductor back-end technology, such as China, Japan, Southern Asia, and Europe. Unlike the eight major semiconductor processes, it is difficult to understand the flow of the entire technology in the semiconductor back-end process due to its diversity and variability. Heterogeneous Integration is new era technology, integration of separately manufactured into a higher-level assembly that provides functional improvements. The Presenter explains the semiconductor Back-End Process and Technology, from the assembly technology mentioned in the ITRS last version to the Heterogeneous Integration technology in the IEEE EPS HIR, in connection with Moore’s Law. In addition, present what is considerable design concepts including electrical, mechanical, and thermal simulations in this area.

• 2. Don’t Close Your Eyes on Temperature: System Level Thermal Perspectives of 3D Stacked Chips
  

  Sung Woo Chung (Professor, Korea University, Korea)

  With the limited process technology scaling, heterogeneous integration becomes a viable solution to elevate system performance. For heterogeneous integration, 3D stacking is one of the attractive solutions since it leads to small area. However, 3D stacking inevitably causes higher on-chip temperature due to high power density, which negatively affects processing units and DRAM as follows: 1) When on-chip temperature of the processing units such as CPU, GPU, and NPU reaches threshold temperature, DTM(Dynamic Thermal Management) is invoked to reduce power consumption (which eventually sustains on-chip temperature). For DTM, frequency and (or) voltage is decreased leading to system performance degradation. 2) When on-chip temperature of DRAM (HBM) goes over threshold temperature, DRAM should be refreshed more frequently to safely store data, resulting in refresh energy increase and system performance degradation. In this talk, experimental results on Intel Lakefield (first TSV-based CPU) and AMD 5800X3D (first C2C-based CPU; actually, last level cache is stacked) are presented. Additionally, additional refresh overhead due to high on-chip temperature (mainly dissipated from a processing unit through silicon via) in HBM is presented.

• 3. Introducing UCIe: The Global Chiplet Interconnect Standard
  

  Youngbin Kwon (Engineer, Samsung Electronics, Korea)

  As the chiplet business gains momentum in the HPC (High-Performance Computing) market, several chiplet interconnect standards have emerged and vied for global standardization. Throughout this process, two standards, UCIe (Universal Chiplet Interconnect Express) and BoW (Bunch of Wire), have persevered. UCIe, in particular, has garnered significant attention as a potential global standard, primarily due to its robust consortium backing. Additionally, unlike other standards, UCIe offers support for widely used protocols in the HPC field, such as PCIe and CXL. This talk aims to introduce the key characteristics of UCIe, delve into an overview of the UCIe Transceiver Structure, and highlight its major electrical parameters. Furthermore, this talk will briefly touch upon the current development trends of IP vendors within the UCIe ecosystem.

• 4. Addressing Modeling and Simulation Challenges in Chiplet Interfaces
  

  Jaeha Kim (Professor, Seoul National University, Korea)

  High-speed die-to-die interfaces are essential in enabling chiplets, the emerging building blocks for heterogeneous integration. Interestingly, many chiplet interface standards including Universal Chiplet Interconnect Express (UCIe) are evolving in ways so that the analog circuits become standardized blocks, and the digital finite-state machines (FSMs) provide complex functionalities. While such architecture can improve design efficiency and portability, it presents new challenges for verifying the overall system functionalities. This talk will use an example of modeling both the analog circuits and digital FSMs of a UCIe physical layer in SystemVerilog and discuss how one can combine the analog and digital approaches to functional verification.

Session 5E: DF-3 (January 24, 13:00 – 14:40)

[AI/ML for Chip Design and EDA – Current Status and Future Perspectives from Diverse Views]

AI/ML for chip design and EDA has received tremendous interests both of academia and industry in recent years. However, as it is natural to take time for the adoption of new technology, there seems gaps between pure research results at academia and real adoptions at industry. In this session, we will review current status and future perspectives for AI/ML for chip design and EDA from diverse views. The first talk, as overall visionary talk, will cover recent advancement/breakthroughs and share future perspectives. The second talk will introduce how engineers can leverage AI/ML solutions in chip design from EDA industry’s viewpoint. Then, as the third talk, Samsung (Memory) will present how AI/ML are reshaping semiconductor memory design and manufacturing. Finally, AI/ML researches for computational lithography will be reviewed as the best practice academia research for industry applications, as the final talk.

• 1. AI for Chip Design & EDA: Everything, Everywhere, All at Once
  

  David Z. Pan (Professor, Univ. of Texas at Austin, USA)

  AI for chip design and EDA has received tremendous interests from both academia and industry in recent years. It touches everything that chip designers care about, from power/performance/area (PPA) to cost/yield, turn-around-time, security, among others. It is everywhere, in all levels of design abstractions, testing, verification, DFM, mask synthesis, for digital as well as some aspects of analog/mixed-signal/RF designs as well. It has also been used to tweak the overall design flow and hyper-parameter tuning, etc., but not yet all at once, e.g., generative AI from design specification all the way to layouts, in a correct-by-construction manner. In this talk, I will cover some recent advancement/breakthroughs in AI for chip design/EDA and share my perspectives.

• 2. How Engineers can Leverage AI Solutions in Chip Design
  

  Erick Chao (Senior Software Architect, Cadence Design Systems, Taiwan)

  Integrating AI solutions into chip design can indeed offer significant benefits in terms of optimizing performance, power, area and productivity. This integration can be approached from multiple angles, including those of EDA (Electronic Design Automation) research and development and the end user's perspective. An overview of how engineers can leverage AI solutions in chip design will be introduced.

• 3. AI/ML Empowered Semiconductor Memory Design: An Industry Vision
  

  Hyojin Choi (Principal Engineer, Samsung Electronics)

  In this talk, we delved into the transformative realm of AI/ML empowered semiconductor memory design and manufacturing, with a keen focus on memory products within the semiconductor industry. We navigate through the synergistic integration of artificial intelligence and machine learning in streamlining the design and manufacturing processes of semiconductor memory. Our vision embraces a future where AI/ML technologies seamlessly harmonize, efficiency, and product quality. Join us as we present a visionary perspective on how AI/ML technologies are reshaping semiconductor memory design and manufacturing, propelling the industry towards an era of unparalleled advancements and sustainable growth.

• 4. ML for Computational Lithography: What Will Work and What Will Not?
  

  Youngsoo Shin (Professor, Korea Advanced Institute of Science and Technology, Korea)

  ML has extensively been studied for computational lithography processes including OPC, assist features, lithography modeling, hotspot, and test patterns. This talk will review some of these while focusing on the best practice for industrial applications, e.g. hybrid ML and standard algorithmic approach, synthesis of test data for higher coverage, etc.

Session 1F: DF-4 (January 23, 10:45-11:35)

[Advanced EDA using AI/ML at Synopsys]

Synopsys.ai™ is the industry’s first electronic design automation (EDA) solution suite to use the power of AI from system architecture through to manufacturing. Synopsys.ai suite quickly handles design complexity and takes over repetitive tasks such as design space exploration, verification coverage and regression analytics, and test program generation, while helping to optimize power, performance, and area. This frees up engineers to focus on chip quality and differentiation. AI capabilities can help teams quickly migrate their chip designs from foundry to foundry or from process node to process node. Synopsys.ai empowers engineers to get the right chip with the right specs to market faster. In this session you will learn more about how these AI-driven solutions can accelerate the design of your chip through two actual implementation cases.

• 1. AI-Driven Solution for DFT Optimization
  

  Soochang Park (EDA Group, Synopsys)

  In the industry of semiconductor design, configuring optimal specifications for a design is becoming challenge due to numerous inter-dependent design parameters. Specifically, in design for testability (DFT), it is more demanding to predict the quality of test pattern in advance since it is generated through automatic test pattern generation (ATPG) step after DFT IP is implemented. Thus, such flow innately requires long iteration run-time and computing resources for designers. In frond-end of design implementation step, ways to improve ATPG quality of results (QoR) can be adjusting DFT specifications and applying automatic test-point insertion (TPI). However, DFT specifications should be considered within design limitations and hierarchical design guidelines, and TPI solutions might increase area overhead. To find the recipe of DFT for optimal ATPG results while considering the design circumstances, the ML applied solution of Synopsys is suggested as a promising solution to automates the process and eventually to replaces human resources. In conducted experiments, integrating two different steps from DFT insertion to ATPG is enabled, so that the ML solution can automate the flow and learn the relation of DFT recipe and ATPG QoR. The experimental results not only show outstanding results in terms of ATPG QoR, but also successfully show that the constraints of synthesis can be reflected in accordance with the user’s intention.

• 2. Optimization of PDN and DTCO using Synopsys Machine Learning Framework
  

  Kyoung-In Cho (EDA Group, Synopsys)

  In the realm of sub-nanometer technology nodes, the semiconductor industry faces the challenge of fulfilling increasingly complex application demands while adhering to stringent power, performance, and area (PPA) requirements. A major hurdle is the mismatch between metal pitch and cell height reduction, leading to heightened routing congestion and hindering effective chip size reduction. Although adding more metal layers can mitigate this issue, it substantially raises production costs and expands the design space, especially when considering IR-drop during physical implementations. Design-Technology Co-Optimization (DTCO) emerges as a vital strategy to bypass physical scaling limits and enhance transistor density, performance, and power efficiency. However, it significantly broadens the design scope in physical implementations, as chip designers must integrate technology-related variables with existing design parameters in the early technology stage. To address these challenges, we advocate for the integration of Machine Learning (ML) to identify and optimize technical parameters. ML demonstrates exceptional potential in fine-tuning complex parameters. Specifically, Synopsys DSO.ai (Design Space Optimization AI), a pioneer in applying ML within the Electronic Design Automation (EDA) sector, shows promising results. Our experiments using Synopsys DSO.ai successfully identify an optimal metal pitch that minimizes IR-drop impact and efficiently determine suitable parameters for DTCO.

• Kyumyung Choi (Seoul National University)
• Changho Han (Kumoh National Institute of Technology)