Jisu Kwon (Integrated Ph.D. Student)
 |
Ph.D Candidate. |
Repository Commit History
 |
Introduction
My research focuses on realizing AI at the edge, specifically addressing the challenge of executing neural network computations encompassing training and inference on MCUs. I have conducted full-stack research aimed at improving the software for AI applications optimized for target MCU architectures and designing hardware architectures capable of efficiently executing embedded software, including ARM core-based MCU HW/FW/SW/OS:
ARM-core compatible physical chip & accelerator design.
AMBA connectable RTL IP development & top integration.
Efficient MCU firmware update technique development (STM32, Nordic NRF).
Efficient OS-driven firmware design using automotive MCU (Infineon TriCore).
Efficient chip verification platform using RTL simulator & system emulator-mixed framework.
Full Bio Sketch
Mr. Kwon received my B.S. degree in Electronics Engineering at Kyungpook National University, Republic of Korea, in 2019 with early graduation. Currently, I am an integrated Ph.D. student in the School of Electronic and Electrical Engineering at Kyungpook National University, Republic of Korea. I am the first student enrolled in the BS/MS/Ph.D.-integrated course and I am currently working towards obtaining my Ph.D. degree in the AI-embedded System-S2oftware-on-Chip Lab.
My research focus lies in realizing AI at the edge, specifically addressing the challenge of executing neural network computations encompassing training and inference on MCUs. Applying general-purpose neural networks as is to resource-constrained MCU-based edge environments is not feasible. Particularly in tightly coupled MCU architectures where hardware and software must be considered together, a design that accounts for both is necessary. Therefore, I have conducted full-stack research aimed at improving software for AI applications optimized for target MCU architectures and designing hardware architectures capable of efficiently executing this software. Consequently, my research interest encompasses both software for MCUs (firmware) and hardware processor design.
In this context, I have participated in various papers and projects, including: 1) Proposed efficient firmware update techniques considering the MCU flash memory structure. 2) Designed various AMBA-connected ARM peripheral IPs used in MCUs and ICs, leading projects to chip fabrication. 3) Published papers proposing various techniques for efficient execution of TinyML applications on MCUs. 4) Published papers proposing a platform for emulating the interaction between accelerators and software-driven cores.
Furthermore, I have obtained three patents and have two others currently under review. I am highly proficient and experienced in C/C, Verilog, MATLAB, Tcl, and Shell Script languages.
Research Topic
Partial Firmware Update Technique
In applications where firmware updates are frequent, such as edge devices in the Internet of Things (IoT) networks with embedded systems, the update process is considered a significant role in improving device performance. This research proposed a user insensible sliding firmware update technique based on a function block that reduces flash memory usage by handling only part of function blocks instead of the entire firmware, and device's pause time by allowing the user to use the device between function block updates. The proposed technique was evaluated using the target board with the actual ARM core. As a result, the flash memory usage overhead was reduced by 63.64%, paused minimum time was reduced by 79.98% at once update process, and energy consumption reduced by 78.78%. We show that user insensible sliding firmware update has great potential for implementing IoT networks that change behavior quickly by being aware of the environment.
Chip Simulation Virtualization
Iterative register-transfer level (RTL) simulation is essential for the edge processor design, but the RTL simulation speed is significantly slower in a system where various RTL models are complicatedly integrated. In this paper, we propose a novel metamorphic edge processor simulation framework that partitions the software part and virtualizes it in the system emulator to eject from full RTL simulation. The system emulator, which is written in a high-level language, and the Verilog simulation have different abstraction levels, thus the Verilog procedural interface (VPI) module is plugged into the Verilog simulator to connect with the virtual layer interface. In the system emulator, a Verilog RTL simulation session corresponding to a specific parameter set can be dynamically loaded at runtime to provide metamorphism by flexible partial parameter-driven RTL model replacement. We applied the proposed framework to finite impulse response (FIR) filter, and it is successfully demonstrated and achieved simulation speedup for given parameters.
Lightweight AI Acceleration on Edge Devices
My research interests in AI on the embedded system by hardware & software co-design for lightweight edge AI. Typical inference requires only read operation to access static pre-trained weight parameters. On the other hand, training needs to use both read and write operations to update weight during backpropagation. In the case of MCU, on-device training faces two challenges; low-cost read-write accessible on-chip SRAM is too small to store the weight, and the relatively large flash memory has a write access constraint. Furthermore, considering conventional network size, flash memory is inevitable to realize on-device training on tiny MCU. To cope with the MCU memory constraints, we need to design AI software algorithms considering MCU hardware characteristics. We propose the gradient delta-based runtime adaptive partial update for weight stored in a flash memory according to the MCU's memory structure. Based on column-wise gradient delta computation at the training runtime, the weight to update is partially copied to SRAM from the flash memory specific page. The figure on the right shows where my research area is located. An approach that considers both hardware and software, such as the one I propose, can enable lightweight AI applications to be deployed on resource-frugal devices with minimal resources.
Neural Network-based Hardware Compensation
Typically, circuits had to be designed at a high cost to prevent irregular and random noise. This paper combines a low-cost designed part with a lightweight compensation technique, instead of designing a noise-tolerant circuit at a high cost. A technique that uses a compensate program in an embedded system has been applied to the ADC case study for compensating ADC output as ideal. The proposed technique implemented in embedded systems can compensate for deterministic noise operating on static hardware (ADC) as a minimal resource. The embedded system compensation technique can be applied to ADCs and various hardware that include human uninterpretable deterministic noise.
Publications
Authored Books
Low-Power Digital System Design - Practical Approach
 |
Mr. Kwon contributed his effort on wriring this book as co-author. This book covers low-power system-on-chip design methodologies and implementation techniques, from gate-level to architectural approach and software-hardware interfaces
|
Journal Publications (KCI 5, SCOPUS 1, SCI 7)
Jisu Kwon, Jeonghun Cho, and Daejin Park. Efficient Flash Memory Access Power Reduction Techniques for IoT-Driven Rare-Event Logging Application (KCI) IEMEK Journal of Embedded Systems and Applications, 14(2):87-96, 2019.
Jisu Kwon and Daejin Park. Acceleration of ECC Computation for Robust Massive Data Reception under GPU-based Embedded Systems (KCI) Journal of the Korea Institute of Information and Communication Engineering, 24(7):956-962, 2020.
Jisu Kwon, Moon Gi Seok, and Daejin Park. GPU-Based ECC Decode Unit for Efficient Massive Data Reception Acceleration (SCOPUS) Journal of Information Processing Systems, 16(6):1359-1371, 2020.
Jisu Kwon and Daejin Park. Velocity and Distance Estimation-based Sensing Data Collection Interval Control Technique for Vehicle Data-Processing Overhead Reduction (KCI) Journal of the Korea Institute of Information and Communication Engineering, 24(12):1697-1703, 2020.
Jongheon Baek, Jiwoong Jung, Minsung Kim, Jisu Kwon, and Daejin Park. Low-Power Metamorphic MCU using Partial Firmware Update Method for Irregular Target Systems Control (KCI) Journal of the Korea Institute of Information and Communication Engineering, 25, 2021.
Jisu Kwon, Moon Gi Seok, and Daejin Park. Low-Power Fast Partial Firmware Update Technique of On-Chip Flash Memory for Reliable Embedded IoT Microcontroller (SCI) IEICE Transactions on Electronics, 2021.
Jisu Kwon and Daejin Park. Hardware/Software Co-design for TinyML Voice-Recognition Application on Resource Frugal Edge Devices (SCI) Applied Sciences, 11(22):11073-11087, 2021.
Jisu Kwon and Daejin Park. Efficient Sensor Processing Technique using Kalman Filter-based Velocity Prediction in Large-Scale Vehicle IoT Application (SCI) IEEE Access, 2022.
Jisu Kwon and Daejin Park. Collaborative Streamlined On-Chip Software Architecture on Heterogenous Multi-Cores for Low-Power Reactive Control in Automotive Embedded Processors (KCI) IEMEK Journal of Embedded Systems and Applications, 17(12), 2022.
Seungmin Lee, Jisu Kwon, and Daejin Park. Runtime Tracking-based Replication of On-Chip Embedded Software using Transfer Function Learning for Dust Particle Sensor Systems (SCI) IEEE Access, 2023.
Seungmin Lee, Jisu Kwon, and Daejin Park. Optimized Replication of ADC-based Particle Counting Algorithm with Reconfigurable Multi-Variables in Pseudo-Supervised Digital Twining of Reference Dust Sensor Systems (SCI) Sensors, 2023.
Jisu Kwon and Daejin Park. Efficient Partial Weight Update Techniques for Lightweight On-Device Learning on Tiny Flash-Embedded MCUs (SCI) IEEE Embedded Systems Letters, 2023.
Jisu Kwon and Daejin Park. Sliding-Window-based Fast and Lightweight ADC Pseudo-Randomness Compensation Technique for Low-Cost ADC (SCI) (Under Revision) In IEICE Electronics Express, 2024.
Conference Publications (Intl. 12)
Jisu Kwon and Daejin Park. Efficient Flash Memory Access Power Reduction Techniques for IoT-Driven Rare-Event Logging Application In 2019 IEEE COOLChips, 2019.
J. Kwon, J. Cho, and D. Park. Function Block-Based Robust Firmware Update Technique for Additional Flash-Area/Energy-Consumption Overhead Reduction In 2019 International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS), pages 1-2, 2019.
J. Kwon and D. Park. Implementation of Computation-Efficient Sensor Network for Kalman Filter-Based Intelligent Position-Aware Application In 2020 International Conference on Artificial Intelligence in Information and Communication (ICAIIC), pages 565-568, 2020.
Jisu Kwon, Moon Gi Seok, and Daejin Park. User Insensible Sliding Firmware Update Technique for Flash-Area/Time-Cost Reduction toward Low-Power Embedded Software Replacement In 2020 IEEE COOLChips, 2020.
Minsung Kim, Jongheon Baek, Jiwoong Jung, Jisu Kwon, and Daejin Park. Segmented Polynomial Approximation for Controlled System Characteristic Estimation on Lightweight Edge Device In IEEE ICCE-Asia 2020, 2020.
Jisu Kwon and Daejin Park. Toward Data-Adaptable TinyML using Model Partial Replacement for Resource Frugal Edge Device In HPC-Asia 2021, 2021.
Jisu Kwon, Sejong Oh, and Daejin Park. Metamorphic Edge Processors Using Flexible Runtime Partial Replacement of Software-Embedded Verilog RTL Models In IEEE ISCAS 2021, 2021.
Jisu Kwon and Daejin Park. Lightweighted AI-based Inference Unit using Deterministic Randomness Compenstation Techniques for Long-Term ADC Resolution Enhancement In IEEE ISOCC 2022, 2022.
Jisu Kwon, Moon Gi Seok, and Daejin Park. Neural Network-based Approximate Quality Prediction for Parameter Exploration in Industrial Manufacturing In IEEE ISPACS 2022, 2022.
Jisu Kwon, Heuijee Yun, and Daejin Park. Dynamic MAC Unit Pruning Techniques in Runtime RTL Simulation for Area-Accuracy Efficient Implementation of Neural Network Accelerator In IEEE MWSCAS 2023, 2023.
Jisu Kwon and Daejin Park. Efficient Partial Weight Update Techniques for Lightweight On-Device Learning on Tiny Flash-Embedded MCUs In ACM International Conference on Embeeded Software (EMSOFT 2023), 2023.
Jisu Kwon and Daejin Park. Micro-Accelerator-in-the-Loop Framework for MCU Integrated Accelerator Peripheral Fast Prototyping In ACM International Conference on Embeeded Software (EMSOFT 2023), 2023.
Patents (8 Korea Patents)
Firmware Update Method using QR Code Image and Electronic Device Performing Same In Korea Patent and Tradmark Office, Korea Patent 10-2391306, 2022
Firmware Update Device and Update Method In Korea Patent and Tradmark Office, Korea Patent 10-2375290, 2022
Code Insertion Module and Method for Dividing Storage of Firmware Segment In Korea Patent and Tradmark Office, Korea Patent 10-2391312, 2022
Firmware system to control software malfunction and control method thereof In Korea Patent and Tradmark Office, Korea Patent, 10-2673433, 2023
Memory Connection System and Memory Connection Method for Acceleration of Operation In Korea Patent and Tradmark Office, June 2022. Korea Patent Pending
Artificial Intelligence Computing System and Method In Korea Patent and Tradmark Office, June 2022. Korea Patent Pending
Apparatus And Method to Update Weight for On-Device Learning Of Neural Network Model In Korea Patent and Tradmark Office, June 2022. Korea Patent Pending
System and Method for Simulating Micro Accelerator In Korea Patent and Tradmark Office, June 2022. Korea Patent Pending
Participation in International Conference
IEEE ASP-DAC 2019, Tyoko, Japan
IEEE COOLChips 2019, Yokohama, Japan
IEEE ISPACS 2019, Taipei, Taiwan
IEEE ICAIIC 2020, Fukuoka, Japan
IEEE COOLChips 2020, Tokyo, Japan
IEEE ICCE-Asia 2020, Busan, Korea
IEEE HPC-Asia 2021, Jeju, Korea
IEEE ISCAS 2021, Daegu, Korea
IEEE ISLPED 2022, Boston, USA
IEEE ISOCC 2022, Kangreung, Korea
IEEE ISPACS 2022, Penang, Malaysia
IEEE ASP-DAC 2023, Tokyo, Japan
IEEE MWSCAS 2023, Arizona, USA
IEEE/ACM EMSOFT 2023, Hamburg, Germany
Last Updated, 2024.12.13
)