Learning from demonstration videos
Ongoing research into extracting objects, actions and reusable skills from unstructured human demonstrations, connecting with multimodal grounding and skill learning.
RESEARCH · PROJECTS · EXPLORATIONS
Work Timeline.
Research, engineering projects and earlier experiments, from 2017 to today.
2026
Research
DR-TSLM
Grounding “this” and “here” in speech and gestures, so robots can identify what to move and where it belongs.
CIS-RAM 2026 →Industry collaboration
Cable Assembly & Contact-Guided Insertion
Robotic cable insertion through visual registration and force feedback, in collaboration with Desay SV Singapore.
Research & engineering
Vision-Servoed Microdevice Injection
A hardware-agnostic vision-servoed system for automated, precise injection of implantable microdevices.
ICRA 2026 →
2025
Engineering
Desk Organization with Visual Feedback
Connecting visual-language perception and arm control to find, grasp and organize objects on a cluttered desk.
Publication
LLM-augmented hierarchical reinforcement learning for human-like decision-making of autonomous driving
Expert Systems with Applications · 294, 128736
Publication
Large language model-enabled cognitive agent for self-aware manufacturing
Journal of Manufacturing Systems · 82, 1213–1226
Publication
A Cognitive Digital Twin for Industry 5.0 Based on a Large Language Model Agent
IEEE ICHMS · 223–228
2024
Engineering
Mobile Manipulation & Language Interfaces
Integrating a robot arm, mobile base and vision sensors into a platform for natural-language-guided manipulation.
Publication
Personalized Disassembly Sequence Planning for a Human–Robot Hybrid Disassembly Cell
IEEE Transactions on Industrial Informatics · 20(9), 11372–11383
Research
Long-Horizon Skill Learning
Turning instructions and interaction history into reusable skills for longer, compositional robot tasks.
CIS-RAM 2024 · 14–19 · Best Paper →
2022
Publication
Template-Based Category-Agnostic Instance Detection for Robotic Manipulation
IEEE Robotics and Automation Letters · 7(4), 12451–12458
Research & engineering
Human–Robot Interaction: Proximity & Admittance
Proximity-based slowing and stopping, FCL self-collision checks, and an admittance controller that turns obstacle distance into avoidance motion.
2021
Research & engineering
Human–Robot Interaction: Perception & Handover
Skeleton-based collision avoidance with Nuitrack and MoveIt, GGCNN grasping and handover, and position-based visual servoing on a UR3e.
2020
2019
Robotics
Ground-to-Air Robot Coordination
Ground-to-air robot coordination based on visual navigation.
Publication
An Modeling Processing Method for Video Games Based on Deep Reinforcement Learning
IEEE ITAIC · Chongqing, China
2018
Engineering
Hexapod Robot
A Raspberry Pi and STM32 hexapod with 18 servos, ROS interfaces and multimodal sensing for walking, obstacle avoidance and uneven terrain.
Machine learning
Deep Reinforcement Learning for Video Games
Deep reinforcement learning for video games; related work published at IEEE ITAIC 2019.
Human–machine interaction
EEG-Controlled Teleoperated Car
EEG-based teleoperation with visual signal stimulation.
2017
Further explorations
Visual-guided unloading & field integration
Engineering work on unloading order, trajectories and suction-based handling, with factory-site integration and demonstration experience in 2023.
NAV Clothes
A wearable navigation prototype integrating Raspberry Pi, ToF sensing, voice interaction, stereo cameras and ORB-SLAM3.