FreeCloth: Free-form Generation Enhances Challenging Clothed Human Modeling

Arxiv 2024
Hang Ye 1,   Xiaoxuan Ma 1,   Hai Ci 1,   Wentao Zhu 1,   Yizhou Wang 1

1 Peking University


Achieving realistic animated human avatars requires accurate modeling of pose-dependent clothing deformations. Existing learning-based methods heavily rely on the Linear Blend Skinning (LBS) of minimally-clothed human models like SMPL to model deformation. However, these methods struggle to handle challenging loose clothing, such as long dresses, because the required canonicalization process is ill-defined where clothing is away from the body, leading to disjointed and fragmented results. To overcome this limitation, we propose a novel hybrid framework to model challenging clothed humans. The core idea is to use two separate strategies to model clothing regions close to the body and distant from the body. Specifically, we leverage LBS to model close regions that are heavily affected by body movements. More importantly, we further introduce a novel free-form generation to model more distant clothing regions that are less affected by body movements. This free-form generation module, conditioned on the underlying body, endows enhanced flexibility and expressive capabilities when modeling challenging loose clothing, such as skirts and dresses. Combining the strengths of LBS-based deformation and free-form generation, our hybrid framework effectively captures the intricate geometric details of loose clothing. Experimental results on the benchmark dataset featuring loose clothing demonstrate that our method achieves superior visual fidelity and realism, particularly on the most challenging cases.

Comparison with state-of-the-art

More Results

User Study

Perceptual study results. The looseness of the clothing increases from the top row to the bottom. Across all examples, 69.3% users prefer our model over the baselines. In addition, our model demonstrates overwhelming superiority, especially on the most challenging clothing.

Method Overview

Given an unclothed body and a specific garment type, our goal is to create a realistic clothed human. FreeCloth comprises two essential modules: (1) an LBS-based local deformation network to obtain pose-dependent deformed points that are close to the human body, and (2) a free-form generation module that focuses on generating the more distant clothing regions. By merging the deformed points and the generated points, we finally obtain the full clothed human point cloud.