Yi, Zeji (易泽吉)

Nonlinear Covariance Control via Differential Dynamic Programming

Zeji Yi, Zhefeng Cao, Evangelos Theodorou, and Yongxin Chen
American Control Conference 2020
[arXiv]

We consider covariance control problems for nonlinear stochastic systems. Our objective is to find an optimal control strategy to steer the state from an initial distribution to a terminal one with specified mean and covariance. This problem is considerably more complicated than previous studies on covariance control for linear systems. We leverage a widely used technique - differential dynamic programming - in nonlinear optimal control to achieve our goal. In particular, we adopt the stochastic differential dynamic programming framework to handle the stochastic dynamics. Additionally, to enforce the terminal statistical constraints, we construct a Lagrangian and apply a primal-dual type algorithm. Several examples are presented to demonstrate the effectiveness of our framework.

High-dimensional Optimistic Safe Optimization with Projection to Distance-preserving, Quasi-physical Spaces

Zeji Yi, Yunyue Wei, Hongda Li, Yanan Sui
ICML 2022 ReALML Workshop
[arXiv]

Many real-life sampling problems are high-dimensional and require safety guarantee during optimization. Current safe exploration algorithms ensure safety by conservatively expand- ing the safe region, leading to inefficiency in large-scale input settings. In this paper, we propose a practical method, which utilizes auto-encoder to link physical prior of certain problems with index-based input space and also projects the original input space into a low-dimensional subspace. The low-dimensional space can be viewed as a quasi-conformal transformation of space with explicit physical meaning. An optimistic safe strategy to effi- ciently optimize the utility function is carried in the low-dimensional space then. We show in simulation that our method outperforms representative safe exploration algorithms while sacrificing little safety. Clinically, our proposed method also achieved better or competitive performance on two high-dimensional neural stimulation optimization tasks comparing to human experts.

Improving sample efficiency of high dimensional bayesian optimization with MCMC on approximated posterior ratio

Zeji Yi, Yunyue wei, Cloris Cheng, Kaibo He, Yanan Sui
Submitted to ICML2023
[PDF]

A Remote Adaptive Upper-Limb Training Framework With Collaborative Robot

Jun Hong Lim, Zeji Yi, Kaibo He, Chen Hou, Yanan Sui
Submitted to ICRA2023

Adversarial learning of control parameters

Advisor: ChuChu Fan, Assistant Professor at Department of Aeronautics and Astronautics, MIT
Summer 2022 - Now

● Generating rare/unsafe cases with one or two order larger probability compared with the original environment
● Constructed a network based distribution for control parameters with reparameterization
● Optimized the control parameters based on interpolated density calculated from Liouville equation

High-Dim Bayesian Optimization with MCMC, Beijing, China

Advisor: Yanan Sui, Associate Professor at School of Aerospace, THU
Spring - Fall 2022

● Significantly improved the efficiency of Bayesian optimization on high dimensional space on synthetic functions and RL benchmarks compared to current SOTA algorithm
● Elaborated an MCMC-based candidate selecting method to reduce the computation cost
● Naturally provided paralleled solution for batched Bayesian optimization
● Gave Theoretical regret bound of the algorithm and guarantee the convergence

Coactive learning for dueling bandits, Beijing, China

Advisor: Yanan Sui, Associate Professor at School of Aerospace, THU, and Yisong Yue, Professor of Computing and Mathematical Sciences at CalTech
Summer 2022 - Now

● Proposed a new exact regret bound to replace the original asymptotic analysis of convergence
● Expanded the original coactive learning framework with gradient information for better interaction with human
● Integrated Kalman Filter and Langevin dynamics to sampling problem on Gaussian Process

Machine Learning assisted Spinal Cord Stimulation, Beijing, China

Advisor: Yanan Sui, Associate Professor at School of Aerospace, THU
Fall 2021 - Spring 2022

● Proposed and designed a data-driven System for Spinal Cord Stimulation (Data collecting, Pre-processing)
● Optimized Stimulation Parameter with Safety Bayesian Optimization under different constraints
● Constructed Pre-trained model and Auto-encoders to learn sufficient low-dimensional representation for high-dimensional hybrid inputs with regularization term and likelihood function
● Theoretical analysis of Bayesian optimization's convergence in latent space

Covariance Steering and Differential-Dynamic-Programming, Atlanta, GA, US

Advisor: Yongxin Chen, Assistant Professor at School of Aerospace Engineering, GaTech
July 2020 - December 2020

● Proposed a covariance steering method for nonlinear stochastic system
● Solved optimization problem with hard terminal constraint in primal dual algorithm
● Designed the optimal control policy with open-loop and closed-loop control by differential dynamic programming method designed for stochastic dynamic systems
● Developed differential-dynamic programming in belief space, eliminated the uncertainty in propagation and gave a more transparent explanation for the system

Grasping Objects with Robot-arm Carried by Quadrotors, Beijing, China | Project Leader

Advisor: Geng Lu, Assistant Professor at School of Control Science, THU
Aug 2017 - Spring 2020

● Built a dynamical model for the highly coupled arm-quad system with multi rigid body dynamics and gained the forward and inverse dynamics solution
● Motion Planning considering the coupling effect by the functional minimum snap method
● Fused the robot arm and quadrotor's controller with the dynamics solution and eliminated the disturbance of quadrotor caused by the robot arm by the feedforward control
● Object detection with RGBD camera and located the quadrotor with indoor location system

Interactive Scenario in Autonomous Vehicle, Berkeley, CA, US | Research Assistant

Advisor: Masayoshi Tomizuka, Professor at School of Mechnical Engineering, UCB
July 2018 - Sep 2018

● Independently constructed a sequential strategy capable of handling multi interactive agents under information symmetric and asymmetric conditions based on Monte-Carlo method
● Independently realized and optimized a multi-layer inference algorithm in interactive motion planning using A* search and provided a more efficiency heuristic cost

Building and Controlling Quadruped Robot, Beijing, China | Research Assistant

Advisor: Ou Ma, Professor at School of Aerospace, THU
Aug 2017 - Aug 2018

● Designed and optimized the configuration of the robot's legs, lowered the maximum torque during walking by 20% and expanded the workspace of each leg by 30%
● Determined the motion of the legs to achieve the step by forcing periodic swinging sinusoidal centroid orbit while maintaining the robot's stability