SFSU Engineering Team Won Sony's Spresense Developer Challenge 2022 Grand Prize!

MIC Lab and ICE Lab Summer 2022 research group (front row left to right: Dr. Xiaorong Zhang, Calista Huang, Carol Sanders, Philip Liang, John Carlo Manual, and Dr. Zhuwei Qin; back row left to right: Jin Chul Rhim, Zhenyu Lin, Yuriah Lydon, Jimmy Lu, and Vincent Luo)

SFSU Engineering Team Won Sony's Spresense Developer Challenge 2022 Grand Prize!

The SFSU Engineering team won the Grand Prize at the highly competitive Spresense Developer Challenge 2022 with Sony which just concluded on Aug 30, 2022. The competition attracted 49 submissions with over 500 participants around the world.

The SFSU team was composed of a group of graduate, undergraduate, and high school student researchers who conducted research in the Mobile and Intelligent Computing Laboratory (MIC Lab) and the Intelligent Computing and Embedded Systems Laboratory (ICE Lab) in the School of Engineering in summer 2022. The student researchers are Jimmy Lu (Lowell High School), Philip Liang (graduate student in Electrical and Computer Engineering (ECE)), Zhenyu Lin (undergraduate student in Computer Science), Jin Chul Rhim (undergraduate student in Computer Engineering), and Amir Modan (SF State Scholar in the CompE/MSECE blended program).

The project advisors Prof. Zhuwei Qin (Assistant Professor in Computer Engineering and Director of MIC Lab) and Prof. Xiaorong Zhang (Associate Professor in Computer Engineering and Director of ICE Lab organized a collaborative research training program between the two labs in the summer of 2022, aiming to create an inclusive teamwork environment to engage students in cutting-edge interdisciplinary research. Together, they co-advised a diverse group of SFSU graduate and undergraduate students as well as community college and high school students on several different project topics in summer 2022 (image 1).

The winning project, entitled On-device EMG Pattern Recognition for Real-Time Bionic Arm Control by Deep Neural Network, developed an efficient deep learning algorithm to improve electromyographic (EMG)-based neural control for prosthetic arms. The computationally intensive deep learning and control algorithm, for the first time, was deployed by the team on the Sony Spresense 6-core microcontroller board, a high-performance and low-power edge computing device, for real-time processing (image 2). A 3D-printed prosthetic hand prototype was also developed to demonstrate the feasibility of a deep learning-based neural-machine interface for natural and intuitive prosthesis control (image 3). This project has produced promising preliminary results for a recently established collaborative project between the School of Engineering and Sony, which will provide funding and equipment resources for the MIC Lab and the ICE Lab to develop an efficient deep neural network for a neural-controlled bionic arm and to support engineering senior design projects and course projects.

See the SF State News article for more information about the project. 

System overview of the developed project.

Real-time bionic arm control by the deep neural network on Sony Spresense microcontroller board.