High-Performance Quadruped Robot Dog @ UniUbi

• Project Duration: 2026–Present
• Developed a quadruped platform capable of a 720° aerial flip, among the first of its kind worldwide.
• Enabled multi-robot swarm control with strong multi-agent collaboration for coordinated tasks.
• Featured in CCTV media coverage; demonstrated at major robotics events such as the World Robot Conference.

Self-Imitated Diffusion Policy for Efficient and Robust Visual Navigation

• Project Duration: 2026
• Propose SIDP, a self-imitated diffusion policy that selectively imitates high-reward trajectories sampled from itself, reducing reliance on the costly “generate-then-filter” inference pipeline.
• Introduce a reward-guided self-imitation mechanism that encourages consistently high-quality trajectory generation for efficient and robust visual navigation.

Goal-oriented Navigation Instruction Generation with Tour Video Priors

• Project Duration: 2026
• Introduce VideoNIG, a goal-oriented video-grounded navigation instruction generation task that produces step-by-step instructions from ego-centric tour videos, an initial observation, and a multimodal goal—without relying on maps or graphs.
• Propose a two-stage curriculum learning framework (Action Warmup + Complexity Progression) that improves long-horizon spatial reasoning and yields instructions executable by downstream VLN agents.

Adaptive-Length Tokenizer for Autoregressive Mask Generation

• Project Duration: 2025
• Propose ALTo, an adaptive-length mask tokenizer that, for the first time, enables the model to autonomously determine the number of mask tokens based on the complexity of the input mask.
• Develop ALToLLM, which integrates ALTo into a multimodal large language model (MLLM), enabling adaptive mask token generation for object segmentation tasks.

Learning Hierarchical Mask Tokens for Image Segmentation with Large Multimodal Model

• Project Duration: 2025
• Compresses a full mask into ≤32 discrete tokens, so inference reduces to the LLM’s native next-token prediction paradigm.
• Recovers masks without the original image, removing the extra segmentation decoder and simplifying the pipeline.
• Hierarchical Mask Tokens support progressive detail refinement for high-quality masks.

Text-to-Image Generation Based on Multimodal Large Models

• Project Duration: 2024
• This is an Any-to-Any Multimodal LLM project that supports using both images and text as conditions simultaneously.
• By leveraging the powerful MLLM, it achieves superior text encoding and offers better prompt following compared to open-source models.

Intelligent Image Editing Based on Multimodal Large Models

• Project Duration: 2024
• Enables various intelligent image editing tasks.
• Automatically generates Edit Type, Mask Prompt, and Output Image Prompt based on MLLM.

Multimodal-Driven Digital Human Model for Thousands of Users

• Project Duration: 2021-2023
• Supports customization of digital humans for multiple users (>1000) within a single model.
• Supports multiple input types, including voice, singing, and images.
• Extremely fast inference speed, utilizing RTX 4090 for 10x video synthesis speed.

Self-Supervised Multi-Speaker TTS Model

• Project Duration: 2022-2023
• Seamlessly integrates phoneme prediction, phoneme alignment, and vocoder to achieve true self-supervised training, leveraging the value of big data.
• Supports multi-speaker training and zero-shot voice cloning.
• Non-autoregressive design, enabling fast inference speeds.