Virtual fitting room

Virtual fitting room

October 3, 2019

A virtual fitting room (VFR) for a network of European multi-brand retailers selects clothes by superimposing items of clothing onto a customer photo. Errors in fitting should be kept to a minimum, as inaccuracies may mean the return of goods and lost sales.

Various online fitting programs exist already. Their major downside is that you have to enter too much information about yourself. Besides, the quality of the fitting leaves much to be desired. The distinctive feature of the VFR is that most of the model's parameters are identified automatically from the photograph.

VFR offers:

  • Identification of model's parameters from the customer photograph;
  • Selection of the model and size of the chosen item based on the attributes of the model;
  • Selection and visualisation of full outfits (of various brands if desired) taking into account the customer's preferences (casual, business style, evening wear, etc.);
  • Fitting of clothes with the help of a mobile device, online store or stationary fitting room;
  • Self-service in store;
  • Ordering clothes without visiting a store;
  • Faster fitting and less manual work;
  • Less workload on consultants and merchandisers and partial reduction of personnel as a result;
  • Sales growth;
  • Captivation of clientele through technological innovation;
  • Increased competitiveness through the use of technology.

Structure

01 Mobile application VFR's main instrument which makes it possible to create a digital model, allowing customers to visualise trying on clothes and make an order.
02 Stationary fitting room A specialised outlet, consisting of a large upright LCD touch panel and a webcam. The machine speeds up and simplifies the fitting process, can call a consultant on request, and places orders.
03 Online store Remote selection and ordering of goods using a web browser.
04 Online fitting room Module for Bitrix CMS, which has part of the functionality of a mobile application: trying on clothes and placing an order. The customer can switch to a PC by opening the online store, continuing the fitting process from there.
05 Module for building a two-dimensional body model Digitises a model from a photograph.
06 Expert clothing matching system Helps customers to choose stylish clothes that match.
07 Consultant app Notifies consultants in the mobile application of calls from one of the fitting rooms.
08 Administrative unit Manages the virtual fitting room and customises its operations.
09 Backend
  • VFR logics;
  • REST API;
  • Database;
  • Saving user photos and clothing mockups;
  • Processing requests from fitting rooms and other components.
10 Integration
  • Bitrix;
  • Payment gateway of a banking company;
  • SMS gateway;
  • Social networks: Facebook, VK, Twitter;
  • AddThis services.

Body model

A customer looks at the screen, which displays a request to take a full-length portrait photograph. The screen displays instructions and markers indicating the desired perimeters of the body in order to obtain an image that can be later calculated by the algorithm.

When a person enters the video zone, the algorithm starts analysing the in-built webcam stream. The photograph is taken when the customer's body is in the correct pose and position relative to the lens.

The photo is then sent to a server where the model's contours are identified and a two-dimensional vector model is built.

  1. A photo of a human body is fed to the input. The photo should show the person against a contrasting background, and the clothes the person is wearing should be tight-fitting and preferably solid-coloured. Other people or objects are not allowed in the frame.
  2. The body contour is calculated using the OpenCV library.
  3. The resulting contour is validated. The person in the photo should be standing straight, the body should be positioned symmetrically, the legs should be shoulder-width apart, the feet should be facing forward, and the arms should be spread at a 45° angle from the body with the palms inward.
  4. The bends of the contour are used to determine the coordinates of the nodal points.
  5. The distances in pixels between the nodal points of the contour are calculated horizontally and vertically.
  6. Typical models for a man and a woman (Slava and Coco) are used as basic vector models.
  7. The correction of the nodal points along the coordinate axes (delta) is calculated to adjust the basic vector model for a man or woman.
  8. As the output we have an array of coordinates for the nodal points of the vector model.
Vector model
Vector model

Model adjustme

The two-dimensional model is displayed to the user in the form of a monochrome, semi-transparent filled contour superimposed over the original photo. There are tools for scaling the photos relative to the model for a more accurate fit. The contour of the model has nodal points, which can be moved by the user within set limits. Correction boundaries are set for every point.

The following nodal points are given.

01 Transverse neck diameter Located on the border of the neck on the left and the right. Nodal points can be moved horizontally. The line connecting the points moves vertically.
02 Arm length Located on the shoulder joint and the wrist. These can be moved horizontally and vertically.
03 Transverse chest diameter Located on the border of the body at chest level. These points can be moved horizontally. The line connecting the points moves vertically.
04 Transverse waist diameter Located on the border of the body at waist level. These points can be moved horizontally. The line connecting the points moves vertically.
05 Transverse hip diameter Located on the border of the body at hip level. These points can be moved horizontally. The line connecting the points moves vertically.
06 Inside leg length The points are located on the inside surface of the legs. The points can be moved in any direction.
07 Adjustment Circled points are displayed on the contours to change the angles of the arms and legs to better fit the model to the photo. Points of adjustment are located on the palms and feet. These points can be moved horizontally and vertically.
Body model with nodal points
Body model with nodal points

Moving a nodal point leads to a mirrored movement of the symmetrical point relative to the vertical axis. When capturing a nodal point, graphic elements are displayed that indicate acceptable directions and correction limits.

A user can specify one or more parameters: height, waist circumference etc. The other parameters are calculated automatically taking into account the proportions of various parts of the body and the anatomical characteristics of the human body in general. Since a two-dimensional projection is used, the calculations of circumferential measurements are inaccurate and may require manual adjustment of the parameters.

The model's parameters and an image set containing the model and the photograph are saved in the user's profile.

Fitting clothes

The catalogue is a standard online store interface with search and product view functionality, allowing customers to see the cost and availability of items. A key feature is the hint function, which shows the availability of clothes that are suitable in size and match the customer's desired style.

The selected clothes and accessories are combined into an ‘image’ that is superimposed on the body model. The clothes can be found at a stationary fitting room by scanning the relevant barcode.

The fitting screen displays the body model and, if desired, the customer's photo. A catalogue of goods of a specific type is opened by clicking on an ‘unclothed’ part of the model. A new item of clothing is placed on the corresponding body part. The size is calculated automatically, taking into account the compatibility of the new items with the previously selected ones.

Base photograph
Dress superimposed on photograph
Dress superimposed on a photograph
Raster image of dress superimposed on vector model
Raster image of dress superimposed on the Coco vector

The customer can use the «Share photo» button with the superimposed items from the image. The resulting image can be transferred from the mobile application to a stationary fitting room and vice versa using a QR code.

2D clothing models

Shop assistants or brand merchandisers can prepare virtual models of every item of clothing in advance.

  1. The item is photographed against a contrasting background.
  2. The item should be placed on a mannequin whose colour matches the background.
  3. The size of the mannequin should correspond to the size of the clothing item.
  4. The background is removed from the photo.
  5. 2D models are created for every size and colour of the item of clothing.
  6. It is impossible to extrapolate the proportions onto various sizes of one model. For example, in two dresses of different sizes, the ratio of the sleeve lengths to the waist circumference may not coincide.
  7. Manual measurements are taken for every model of every item of clothing.
Dresses on a mannequin
Dresses on a green background
Background correction

Outlook

A working version with 2D static body and clothing models was implemented. In future we plan to implement 3D models and augmented reality which will enable video processing from the camera facing the customer, allowing for instant fitting.