The virtual fitting experience is gradually becoming the new norm for those who buy online. Brands such as Amazon, Alibaba, Kering, and Nike invest in virtual fitting rooms to increase conversion and reduce return rates. It’s predicted that the global virtual fitting room market will grow from $4.03 billion in 2022 to $14.87 billion by 2029. The development of AR and AI technologies opens up many ways to implement this idea with different levels of complexity.
Interested in creating your own virtual dressing room? In this article, we will explain how to develop a virtual fitting solution and share our experience in working on such products to help you understand what to prepare for when starting such a project.
How Does Virtual Room Technology Work?
A virtual fitting room is a technology to try on items without physically having them on. It demonstrates an item by overlaying it on a live video feed of a user via smartphones or mirrors. The customer can check the size and fit of an item before they purchase it.
From the technological side, virtual fitting rooms usually utilize AI, AR, and computer vision.
Most virtual fitting rooms work using Augmented Reality. However, the capabilities of AR instruments can be limited when it comes to some cases. For example, while working with ARKit (Augmented Reality framework for Apple’s devices) we discovered that it has rendering limitations. The cause of this limitation may be the maintenance of the inference speed while neglecting the tracking accuracy, which might be critical for apps working in real time.
Another issue was the poor identification of body parts by the ARKit algorithms. Since the main goal of this algorithm is to identify the whole body, it doesn’t detect any key points if the processed image contains only a part of the body. An example of this issue is the case of a footwear fitting room when the algorithm is supposed to process only a person’s legs.
The conclusion was that virtual fitting room apps might require additional functionality along with the standard AR libraries. For cases when AR capabilities are not enough, you have to involve data scientists to develop a custom pose estimation model to detect key points on only one or two feet in the frame and operate in real-time.
Artificial Intelligence is a foundational technology for most virtual fitting rooms as it is used everywhere from machine learning to image recognition:
- Form-fitting algorithms are a type of AI that can create an accurate model of customers’ bodies. They utilize information gathered from consumers as well as items to match the product to the person.
- Image-recognition algorithms help to recognize objects within a picture to detect what a person looks like. With the help of machine learning, these algorithms become proficient at detecting the sizes of the shoppers and distinguishing fabric types and colors.
For example, solutions like Vue.ai offer an AI-based dressing room interface that allows customers to create their own looks by mixing and matching outfits on models that look similar to them. At the same time, AI that scans a shopper’s body or face can provide more customized recommendations and suggestions. However, ready-made solutions cannot always meet specific retailers’ needs, and that’s when custom app development comes into play.
Types of Virtual Fitting Room Solutions
Virtual fitting room solutions can be developed in different forms depending on many factors. Here are four types of virtual fitting room applications:
1. Wardrobe assistants and outfit planners
When it comes to virtual fitting room products, wardrobe assistants and outfit planners are the simplest ones. They allow customers to create stylish new outfits with clothes they already have.
A user takes photos of their clothes, and the clothes are segmented (partly automatically). Then, items in the wardrobe can be organized by filters, such as season, occasion, color, and brand. Style recommendations or recommendations for different weather conditions might be included as well.
From the business point of view, such products can be good for businesses to attract potential customers. Retailers can add built-in shops for extra items that will complement customers’ style — all in the same place.
2. In-store virtual dressing solutions
Smart mirrors are reflective screens that display a customer’s image when they stand in front of them. They also display other items, like clothing pieces or accessories, over the customer’s image to make it look as if the customer is wearing them. Depending on the features built into the solution, shoppers might also be able to check pricing, color variations, stock availability, and additional information about the product, like design details.
3. Online virtual room solutions
Some solutions work as digital mannequins providing the customer with a view of the product in different colors and sizes via 3D rendering. Typically, users can adjust the mannequin by selecting basic measurements like weight and height. Others give an opportunity to use a wider number of body measurements and shape selections to create a personalized body image. However, this option requires customers to manually measure their bodies which comes with the risk of human error and results that don’t really represent the user’s body.
Other types of virtual fitting room software use AR technology to generate the image of virtual clothing or the product’s effect on top of the customer’s body in real time through a mobile device. This helps shoppers visualize how the piece of clothes looks on them, but they still might have doubts about whether it fits well or not.
4. Body sizing solutions
Size recommendation tools are digital solutions that analyze each customer’s unique body and fit preferences to provide tailored guidance on which size a customer should purchase. Often, modern clothing fit recommendation tools are powered by artificial intelligence algorithms, which compare the customer’s input data against product data to determine which size would fit best.
These tools come in a variety of shapes and sizes, but, essentially, they remove the complexity around sizing and provide an efficient way for customers to find the right product for their bodies.
For example, photo-based size recommendation solutions use images of the customer’s body to calculate their measurements. While they may raise questions about data privacy, with the proper implementation, photo-based size recommendation products can deliver highly accurate results free from human error. Plus, such products look interesting and entertaining for the customers, which can benefit the retail brand as well.
How to Develop a Body Measurement App
Creating a virtual fitting room app involves several steps, from planning and design to development and testing. It’s necessary to understand that the workflow might change depending on the chosen type of product and the scope of features you want to have in your app. Here we focus on some technical aspects of the development process.
Such apps allow customers to try on clothes virtually and offer accurate body measurements to create more personalized experiences while buying clothes. This, in turn, can reduce costs and increase sales, engagement, and customer loyalty.
Here is a brief explanation of how they work:
- A user takes a picture or makes a video of themselves that should accurately replicate the prearranged A-pose and provides some characteristics such as height.
- Based on the photo, it’s possible to detect the necessary body points to find the size.
- With the help of generative models, it is possible then to match the necessary clothes and the existing photo with the additional generation of body parts that are not overlapped by the clothes.
Such an approach is used in some ready-to-use products on the market, but let’s take a look in detail at how we can deal with the body measurement part of the app. From our expertise, we can offer a combination of two ML models in order to achieve accurate results.
Firstly, the Pose Estimation model is necessary to detect body key points. An Image Segmentation model (more precisely Body Segmentation) will help with creating masks of the human body in pixels to be able to convert this information further to centimeters/meters.
However, according to our experience, it’s necessary to remember that the accuracy of body measurements for different body parts varies widely.
The complexity of measuring different body parts is presented in the table below.
Pose Estimation Models
To analyze the human’s position, pose estimation methods are necessary. There are two possible options for performing pose estimation tasks: 2D and 3D pose estimation models.
From our practical experience, 2D pose estimation models usually are not so accurate in comparison with 3D pose estimation models. Also, 3D pose estimation models detect body key points in (x,y,z) coordinates allowing evaluation of human poses in volume which is crucial for virtual fitting room solutions.
However, detecting key points isn’t enough to make correct measurements. So, at this stage, it’s time to use the Body Segmentation model.
Body Segmentation
The Body segmentation model will help with the detection of the body’s mask in the photos provided by users. The model will give the understanding of which pixel from the whole picture is from the object ‘body’ or not. It is quite a well-known task and a wide range of models will provide reliable output for this use case.
Considering that the model will provide results in pixels, we need to understand the real scale in order to convert pixels to centimeters or meters. This information is required, as we never know how users will take photos (including precise distance to the camera) even from three angles. So, it is definitely better to adjust the provided picture (pixels) per calculated scale, based on the user’s height (as the easiest metric for the user).
Body Measurements Calculations
The last step is developing a pipeline that is based on the results provided by the models, as we need to calculate the required body measurements.
The challenges of this step usually include:
- Dependency on the camera angle for proper key point detection. If the position of the user differs from the sample picture required for proper measurements, then there might be inaccuracies. If there is an overlapping of body parts, there might be inaccurate detection of key points.
- Background. If the background on the user’s photos is close in color to the clothes that the user is wearing, this can lead to improper body segmentation and as a consequence, improper body measurements.
When it comes to manual measurements, this information comes with its own challenges as well. Imagine that users will take three photos and they will pinpoint the key points to find the desired measurements according to the instruction.
There are some risks:
- Users must wear tight clothing or just underwear
- Thick clothing (even tight) will cause inaccuracies
- The environment must be well-lit
- Users must be distinguishable from the background (people will be able to find a body much better than AI anyway)
- The measurements quality may depend on phones’ camera quality
- Different users may measure differently
Since users will set points manually, any measurement is simple — just distance from point A to point B.
For better results, it’s possible to combine two types of measurements and measure some dimensions using AI and manual measurements for the rest. Also, it’s possible to allow people to correct AI measurements if needed.
How to Develop An App with 3D Models of Clothes and a Body
Another approach includes working with 3D models of bodies and clothes that are supposed to be processed.
Here is how they work in short:
- At first, we need to create a 3D mesh for the user from the video.
- From the preset of 3D reconstructions, we choose the necessary clothes.
- The clothes are scaled and placed on the defined landmarks on the 3D net of the user.
Working with cloth mesh (Source)
How to Develop an App Based on Latent Diffusions and Their Variations
Another technical approach that can be used in virtual fitting room development is based on latent diffusion and its variations. Here is how it works in practice.
In this case, we used a stable diffusion inpainting pipeline, set up on custom clothes. To set up the model, we needed to have more than 50 pictures of clothes from various angles, clothes masks, and text descriptions. Actually, you need to have only a dataset with clothes pictures as they are preprocessed to get masks and text descriptions during the development process. While setting up the model, we create the connections between our pieces of clothes and text descriptions.
On the input interface, we have:
- A picture upon which we will change the clothes.
- The mask where the change is happening.
- The text description that will match the clothes we want to add.
The choice of technical approach usually depends on your goal, resources, and limitations, as every virtual fitting room product is unique and requires a specific outlook.
How MobiDev Can Help You with Virtual Fitting Room Development
Creating a virtual fitting room app involves a combination of design, technology, and user-centered development. It’s important to prioritize user experience, as well as to ensure that the virtual try-on experience is accurate and visually appealing. Depending on your resources and technical expertise, you may also consider partnering with professionals or development agencies specializing in AR, VR, and app development.
Whatever model of virtual room you choose, the MobiDev team has the necessary expertise and experience to transform your idea into a cutting-edge virtual fitting room product. Our experts include AI engineers and AR developers who will choose the most suitable technologies to develop a product that meets your needs and requirements. We will start with a thorough analysis of your requirements and technical research to offer you the best solution for your business.
