PolyU Intelligent 3D Human Modelling Technology Projecting Body Shape and Size Accurately Within 10 Seconds

Shopping well-fitting clothes online or making bespoke garments can be done more easily with the intelligent 3D human modelling technology developed by The Hong Kong Polytechnic University (PolyU), which digitally reconstructs the shape and size of a person accurately from two full body photographs within 5-10 seconds. With online shopping gaining popularity, this PolyU innovation, developed by Dr Tracy P.Y. Mok, Associate Professor from the Institute of Textiles and Clothing at PolyU, and Dr Zhu Shuaiyin, PhD graduate of the same institute, can solve the ill-fitting problem and enhance the online shopping experience.

     (Photo: https://mma.prnewswire.com/media/665050/PolyU_3D_modelling.jpg )

Existing methods to achieve automatic shape modeling and dynamic poses deformation, including scan-based, image-based and example-based have known limitations. They involve expensive and bulky scanners and too much approximation, and therefore are of questionable accuracy.

Projecting body shape and size accurately in 5-10 seconds 

By integrating cutting-edge computer graphic and vision technology, this innovation allows a customised model in arbitrary dynamic poses to be created automatically within 5-10 seconds. The system can reconstruct the 3D shape and extract over 50 size measurements of different parts of a person, including the girth of bust, waist, hip, thigh, knee, calf and neck, as well as arm length, shoulder slope accurately. The discrepancies in tight-fitting and loose-fitting clothing are less than 1cm and 2cm respectively, which are precise enough to fulfil the specific requirements of the clothing industry for fashion applications and comparable to body scans.

Four milestones to digitize human models 

Four methods were developed to realise automatic shape modelling of individuals by:

   1. detecting body parts in tight fitting clothing from front-view and side-view 2D images and then construct them to 3D models;

   2. predicting under-the-clothes body profiles of the subjects based on input images where the body profiles are covered in arbitrary clothing, including tight-fitting, normal-fitting and loose-fitting clothing;

   3. applying cutting-edge deep learning technology to segmentise the human body image from the background. It also improves the robustness, efficiency and accuracy of shape modelling of individuals;

   4. applying the above modelling methods to develop a client-server system. It is a mobile application of the automatic shape customisation technology, starting from photo taking to model customization and size extraction:

https://itunes.apple.com/us/app/1measure/id1234853015?mt=8

https://play.google.com/store/apps/details?id=com.tozmart.imeasure&hl=zh_HK  

These methods are enabled by "big data analytics" where a large dataset with over 10,000 human scan models were used to analyse local 3D shape features and predict under-the-clothes body profiles. Different from other existing methods where 3D scans were analysed globally, PolyU's research team decomposes the data yielded from 3D scan into local features for data analysis. The team then predicts and assembles the 2D and 3D shapes of subjects based on 2D photographs. The resulting 3D models have precisely reconstructed both global and local shape characteristics of individuals, and accurate body measurements can be extracted from these customised models.

Benefiting the fashion industry

The digital reconstruction of human subjects can provide competitive advantages for the fashion industry. "With an accurate projection of the size and shape of the human body, these customised models will enhance online shopping experience, and stimulate growth in fashion online shopping," said Dr Mok.

"The output models can also enable customers to visualise try-on effects before purchases in online stores. This frees us from the limitations imposed by taking body measurements physically, helping customers to select the right size in online clothing purchases," added Dr Zhu.

Commercialisation of the technology 

With the support from Shanghai Technology Entrepreneurship Foundation-PolyU China Entrepreneurship Fund, Dr Zhu has set up the start-up, TOZI, in Shenzhen in 2017 to commercialise and further develop the 3D human modelling technology, echoing PolyU's commitment to foster culture that boosts entrepreneurship and inspires entrepreneurs. 

Based on the intelligent 3D human modelling technology, TOZI has collaborated with "Industry 4.0" factories to launch a brand-new customer-to-manufacturer (C2M) service for customers. It brings unmatched online shopping experience to end users by empowering them to order tailor-made clothing within just three minutes anywhere anytime. By a few clicks on smartphones, customers can measure their own shape and size, and order tailored made shirts, with a range of fabrics, collars, pockets, cuffs, front plackets to choose from.

Videos：

Introduction of the mobile application

https://www.youtube.com/watch?v=dTQxtaYKREU&feature=youtu.be

Introduction of the service launched by TOZI using the intelligent 3D human modelling technology (from the perspective of consumers)

https://www.youtube.com/watch?v=vnCMfTOHvek

Press Contact

Dr Tracy P.Y. Mok
Associate Professor, Institute of Textiles and Clothing, PolyU
Telephone: (852)-2766-4442
Email: tracy.mok@polyu.edu.hk

Dr Zhu Shuaiyin
PhD graduate, Institute of Textiles and Clothing, PolyU
Telephone: (852)-5266-4575
Email: sy.zhu@connect.polyu.hk