News
|
- [Jul. 2024]:Our paper Maniwhere is released.
- [Jul. 2024]:Our paper DiffTactile won best paper award in Noosphere workshop at RSS 2024.
- [Jul. 2024]:Our paper Robo-ABC is accepted by ECCV 2024.
- [May. 2024]:Our paper 3D Diffusion Policy is accepted by RSS 2024.
- [Jan. 2024]:One paper ArrayBot is accepted by ICRA 2024.
- [Jan. 2024]:Two papers are accepted by ICLR 2024, one is spotlight.
- [Sep. 2023]:One paper is accepted by NeurIPS 2023.
- [Jun. 2023]:One paper Flexible Handover is accepted by IROS 2023.
- [Apr. 2023]:One paper IOT-CL is accepted by ICML 2023.
|
Publications
* indicates equal contributions. Representative papers are highlighted.
|
|
Learning to Manipulate Anywhere: A Visual Generalizable Framework For Reinforcement Learning
Zhecheng Yuan*,
Tianming Wei*,
Shuiqi Cheng,
Gu Zhang,
Yuanpei Chen,
and Huazhe Xu
Arxiv, 2024
project page /
paper /
twitter
In this paper, we propose Maniwhere, a generalizable framework tailored for visual reinforcement learning, enabling the trained robot policies to generalize across a combination of multiple visual disturbance types.
|
|
3D Diffusion Policy: Generalizable Visuomotor Policy Learning via Simple 3D Representations
Yanjie Ze*,
Gu Zhang*,
Kangning Zhang,
Chenyuan Hu,
Muhan Wang
and Huazhe Xu
Robotics: Science and Systems (RSS), 2024
project page /
paper /
code /
twitter
In this paper, we present 3D Diffusion Policy (DP3), a novel visual imitation learning approach that incorporates the power of 3D visual representations
into diffusion policies, a class of conditional action generative
models.
|
|
DIFFTACTILE: A Physics-based Differentiable Tactile Simulator for Contact-rich Robotic Manipulation
Zilin Si*,
Gu Zhang*,
Qingwei Ben*,
Branden Romero,
Xian Zhou,
Chao Liu
and Chuang Gan
International Conference on Learning Representations (ICLR), 2024
Best Paper Award, RSS Noosphere (Tactile Sensing) Worshop, 2024
project page /
paper /
code /
twitter
In this paper, we introduce DIFFTACTILE, a physics-based and fully differentiable tactile simulation system designed to enhance robotic manipulation with dense and physically-accurate tactile feedback.
|
|
Thin-Shell Object Manipulations With Differentiable Physics Simulations
Yian Wang*,
Juntian Zheng*,
Zhehuan Chen,
Xian Zhou,
Gu Zhang,
Chao Liu,
and Chuang Gan
International Conference on Learning Representations (ICLR), 2024 [Spotlight]
project page /
paper /
code
In this paper, we introduce ThinShellLab, a fully differentiable simulation platform tailored for diverse thin-shell material interactions with varying properties.
|
|
Robo-ABC: Affordance Generalization Beyond Categories via Semantic Correspondence for Robot Manipulation
Yuanchen Ju*,
Kaizhe Hu*,
Guowei Zhang,
Gu Zhang,
Mingrun Jiang,
and Huazhe Xu
European Conference on Computer Vision (ECCV), 2024
project page /
paper /
code
In this paper, we present Robo-ABC, a framework through which robots can generalize to manipulate out-of-category objects in a zero-shot manner without any manual annotation, additional training, part segmentation, pre-coded knowledge, or viewpoint restrictions.
|
|
ArrayBot: Reinforcement Learning for Generalizable Distributed Manipulation through Touch
Zhengrong Xue*,
Han Zhang*,
Jingwen Chen,
Zhengmao He,
Yuanchen Ju,
Changyi Lin,
Gu Zhang,
and Huazhe Xu
International Conference on Robotics and Automation(ICRA), 2024
project page /
paper /
code
In this paper, we present ArrayBot, a distributed manipulation system consisting of a 16×16 array of vertically sliding pillars integrated with tactile sensors, which can simultaneously support, perceive, and manipulate the tabletop objects.
|
|
Flexible Handover with Real-Time Robust Dynamic Grasp Trajectory Generation
Gu Zhang, Hao-shu Fang, Hongjie Fang and Cewu Lu
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2023 [Oral]
paper
In this paper, we propose an approach for effective and robust flexible handover, which enables the robot to grasp moving objects with flexible motion trajectories with a high success rate.
|
|
Understanding and Generalizing Contrastive Learning from the Inverse Optimal Transport Perspective
Liangliang Shi,
Gu Zhang,
Haoyu Zhen,
and Junchi Yan
International Conference on Machine Learning (ICML), 2023
paper
In this paper, we aim to understand CL with a collective point set matching perspective and formulate CL as a form of inverse optimal transport (IOT).
|
|
Relative Entropic Optimal Transport: a (Prior-aware) Matching Perspective to (Unbalanced) Classification
Liangliang Shi,
Haoyu Zhen,
Gu Zhang,
and Junchi Yan
Conference on Neural Information Processing Systems (NeurIPS), 2023
paper
In this paper, we propose a new variant of optimal transport, called Relative Entropic Optimal Transport (RE-OT) and verify its effectiveness for inhancing visual learning.
|
Selected Awards and Honors
|
- 2024: Best Paper Award in Noosphere workshop at RSS 2024
- 2024: Best Bachelor Thesis Award of Shanghai Jiao Tong University
- 2024: Shanghai Outstanding Graduates
- 2024: Zhiyuan Outstanding Scholarship (Top 30 winnners in Zhiyuan Honor College, ¥20000 RMB)
- 2023: SenseTime Scholarship (Top30 undergraduate AI researchers nationwide, ¥20000 RMB)
- 2023: Shanghai Scholarship (Top 0.2% Shanghai, ¥8000 RMB)
- 2022: National Scholarship (Top 0.2% nationwide, ¥8000 RMB)
- 2022: Hanying Juhua Scholarship (Top 15 winners in Zhiyuan Honor College, ¥15000 RMB)
- 2021: National Scholarship (Top 0.2% nationwide, ¥8000 RMB)
- 2021, 2022: A-level Merit Scholarship (Top 1% SJTU, ¥1500 RMB)
- 2021, 2022: Merit Student (Top 5% SJTU)
- 2020, 2021, 2022, 2023: Zhiyuan Honor Scholarship (Top 5% SJTU, ¥5000 RMB)
|
Service
|
- Reviewer: CoRL 2024, RA-L 2024, IJCAI 2024
|
|