Hejia Zhang

Email / Google Scholar / GitHub / Twitter / LinkedIn

We utilize pre-trained large language models to generate robot actions for fabric smoothing and folding tasks.

This work is an extended version of our previous work on multi-robot geometric task-and-motion planning (MR-GTAMP). We conduct an application study on the roof-bolting task, which is an essential operation within the underground mining cycle. We conduct additional scalability evaluation experiments. We substantially expand the description of the previous MR-GTAMP framework.

We enable scalable robot data collection by assisting human teleoperators with a learned policy. Our approach estimates its uncertainty over future actions to determine when to request user input. In real world user studies we demonstrate that our system enables more efficient teleoperation with reduced mental load and up to four robots in parallel.

We address multi-robot geometric task-and-motion planning (MR-GTAMP) problems in synchronous, monotone setups. The key insight is that pre-computed manipulation capabilities of individual robots can be used to guide multi-robot planning.

Previous work has shown that the space of human manipulation actions has a linguistic, hierarchical structure that relates actions to manipulated objects and tools. Building upon this theory of language for action, we propose a system for understanding and executing demonstrated action sequences from full-length, real-world cooking videos on the web.

We present a system for knowledge acquisition of collaborative manipulation action plans that outputs commands to the robot in the form of visual sentence.

This is an extended verion of our previous work on sim-to-real transfer. We show an algorithm which allows our sim-to-real method to perform long-horizon tasks never seen in simulation, by intelligently sequencing short-horizon latent skills.

We present a novel solution to the problem of simulation-to-real transfer, which builds on recent advances in robot skill decomposition.

We propose augmenting scenario generation systems with surrogate models that predict both human and robot behaviors. In the shared control teleoperation domain and a more complex shared workspace collaboration task, we show that surrogate assisted scenario generation efficiently synthesizes diverse datasets of challenging scenarios.

We propose a “generate-then-repair” framework for automatic generation of playable levels adhering to specific styles.