Real-time area calculations, trying on décor elements, home repair estimates — these and more emerge from the power of smartphone AR measuring tools. Modern augmented reality technologies powered by LiDAR, AR SDKs and machine learning algorithms allow us to achieve this. But it requires specific AR and AI expertise that even technology-advanced companies may lack.
Understanding the process of developing an AR measuring app is critical to seeing your project through. Your team has chosen an idea, come up with a plan, and is ready to put it into action. So let’s take it to the next level and figure out how to implement such a project in the best possible way.
The Mechanics of AR Measurement Apps
Augmented reality measuring apps come in various shapes and sizes. They may simply be 3D rulers, allowing the user to measure the distance between two or more points. The measuring tools may also be a means to an end. For example, they might make it possible to virtually change the wallpaper of a room and give the user insights into how much wallpaper they should buy and how much it will cost.
AR measurement apps utilize the following elements:
- Smartphone cameras and sensors
- LiDAR for iOS
- Time of Flight (ToF) for Android
- ARKit and ARCore SDKs with rendering engine to render AR content on screens
- In some cases, custom machine learning algorithms
The Power of LiDAR for Developing Measuring Apps
AR SDKs have built-in features for measurement, but they are quite limited. For example, ARKit by default recognizes horizontal surfaces well but has trouble with vertical ones. The reason is that the horizontal surfaces are always perpendicular to the gravity vector, so we know their orientation and an AR SDK just has to find its vertical position, while vertical surfaces can be located closer/further, rotated at an angle around the gravity vector, etc. That’s where AR capabilities are not enough and developers need additional hardware power.
LiDAR is some of the most powerful hardware available for smartphones to understand the dimensions of their environment. LiDAR is a depth sensor. It measures the time it takes for light to bounce off of surfaces back to the sensor using pulsed infrared lasers to achieve this. Mesh creation with LiDAR is much more accurate because it provides more precise spatial information about the real-world environment.
LiDAR is available on most modern iPhones and iPads and provides great capabilities for developers to make fairly accurate virtual measurements. But there are some limitations to this technology:
- Lighting conditions: LiDAR uses its own light to capture the environment and make measurements. This means it can accomplish its goals in a variety of lighting conditions but may still be affected by some ambient light conditions.
- Surface reflectivity: mirrors and glass may impede the success of infrared measurements.
- Distance: the farther away the sensor is, the less accurate it will be (the optimal distance is 1-2 meters)
Therefore, it is impossible to completely rely only on hardware capabilities in AR measuring app development. Some cases require additional software solutions, in particular with the integration of AI development.
Rendering AR Content
AR SDKs with 3D engines (SceneKit, Metal, RealityKit, Sceneform, Unity, Unreal, OpenGL, etc.) are indispensable, if you plan to render any kind of AR content on the screen after taking measurements of space. The two most popular native SDKs available to develop AR content for smartphones are ARKit for Apple devices and ARCore for Android devices. These two SDKs are comparable in their capabilities and are generally only limited by the hardware of the device.
Typically, Apple devices have better AR performance due to their superior and more consistent hardware. In contrast, Android devices are typically less consistent, having a variety of different hardware specifications.
Also, iOS apps have LiDAR, and although Android has ToF as an analogue, this sensor is present on a very limited series of devices. In general, with the help of ARCore, it is impossible to create this kind of a mesh today as compared to development with the help of ARKit, which is why iOS most often becomes the main focus in the development of AR applications.
Challenges of Developing Augmented Reality Apps for Measurement
Before diving into the development process, let’s examine some challenges of developing an augmented reality app for measurement. This will help you understand what points to pay special attention to during development and what expectations to set for your development team.
Tracking
Measuring applications need to be able to appropriately track the object or space being measured. Without accurate tracking, measurements will be unreliable. This can be accounted for with software, but it’s also important to consider the human element. Instructing users with on-screen directions can help them use the application correctly so that they can get accurate and reliable results.
Accuracy
In our projects at MobiDev, we have achieved an AR measurement accuracy of 99.7%. However, this is subject to a number of factors and can vary depending on various conditions. For example, the lighting of objects being measured, human error, complexity of objects being measured, and more.
Indoor applications perform the best. However, outdoor environments and significantly large indoor spaces can be tricky. Since AR uses odometry to understand position, and infrared sensors can only see so far, users can run into trouble. However, outdoor spaces do have one advantage compared to indoor — lighting. During the day, the sun can light most spaces reliably.
Performance
Augmented reality measurement apps, like other AR applications, are resource intensive. They’ll use more power and drain batteries faster. In many cases they can slow down your device’s other features. If the application is demanding too many resources, it may even slow down the app itself.
The best way to deal with this problem is to optimize the app’s code as much as possible for the specific platform you’re developing on. Use efficient algorithms to process the camera feed and calculate measurements. You can also minimize unnecessary features and graphics that slow down the experience.
How to Build an AR Measurement App Step by Step
There are two parts to an AR measurement app: the AR feature, and then the infrastructure that makes the AR rendering possible. In what follows, we’ll describe a typical indoor measuring application. Keep in mind that depending on your objectives, your application may have different needs and steps.
Floor Scanning and Floor Plan Creation
Let’s imagine we want to design an app that can create virtual floor plans with accurate dimensions. The user will point their smartphone at floors, ceilings, and walls for the app to process.
ARKit has a built-in tool called RoomPlan that makes this task much easier. This tool allows developers to create AR room measurement apps that use augmented reality to scan and measure the dimensions of a room in real-time with LiDAR.
The user experience with a RoomPlan-enabled app is generally as follows:
- Open the app and select the room scanning feature.
- Aim the camera at the corner of the room.
- Move the device around to capture the room from different angles.
- The ARKit framework will use computer vision algorithms to analyze the images captured by the camera and create a 3D model of the room.
- The app will then use the 3D model to calculate the dimensions of the room, including the length, width, and height.
- The app may also allow you to add virtual objects, such as furniture or artwork, to the scene to see how they fit.
ARKit RoomPlan is a powerful tool, but it has some limitations you need to know more about in order to set the right expectations:
- Limited chip compatibility: ARKit RoomPlan is only available on Apple devices with an A12 Bionic chip or newer. This means that many iPhone and iPad models are incompatible.
- Lighting conditions: if the room isn’t lit well enough, it can be difficult to visually capture the room and display AR content.
- Reflective surfaces: these will disrupt the infrared sensors. For example, if you have a small mirror in the room, this is not a big problem, but if it is a room with giant mirrors like a gym or dance studio, then RoomPlan will unfortunately not help to scan everything correctly.
Adding Objects in AR
As in most complex AR solutions, rendering virtual objects is the ‘easy’ part. It relies heavily on the data and measurements that you use. This step is where you can display content like colored walls, a 3D ruler, furniture, and more.
3D visualization features like rulers allow users to better understand the measurements that they see on screen. Displaying this information in 3D space will require some creative thinking about UX and UI. The goal is to make sure that the application and its UI are easy to use, visually appealing, and easy to understand.
Estimation Module
This tool leverages algorithms and statistical techniques to analyze the data collected by the measurement module and generate an estimate. This allows you to perform complex calculations such as:
- build-out/renovation prices based on metrics like floor or wall area.
- costs for heating the house, knowing its area or volume, as well as the cost of gas
- optimal power of the air conditioner for the room
- the number of WiFi routers, knowing the area or volume
- approximate time of robot cleaning of the specified area with a vacuum cleaner, etc
For example, if your business sells wallpaper, a customer may measure the area of the wall of a room with your app. Based on the measurements they take, the app will use the estimation module to predict how much wallpaper they should buy and how much it will cost.
There are a number of ways to make your estimation model more effective:
- Error correction: small errors can be expected and accounted for. For example, if a user accidentally moves the device slightly during the measurement process, the estimation model can apply smoothing or filtering techniques to make up for it.
- Confidence intervals: this is a statistical technique used to make users aware of uncertainty. Providing a confidence interval to the user can help them make an informed decision about the result of the data they collected.
- Calibration: ensuring that the device is properly calibrated can improve the accuracy of data results. Prompting the user to calibrate the device prior to taking a measurement will help make results more consistent.
- Error messages: these messages can be an important tool for users to help them understand the cause of problems they may encounter. Specific error messages advise the user on how to improve their results. For example, if there isn’t enough light in the space, the app can alert the user and suggest that they turn on the lights.
With these features, the estimation module can provide more accurate insights to the user to help them make actionable decisions on the AR measurement data they collect.
The Scope of Proof of Concept (PoC)
If the project is complex and has many uncertainties, the best solution is to start with a proof of concept (PoC) to check the accuracy of the result under the given conditions. PoC will not be a full-fledged product that you can show to users, but it will be a confirmation that your idea is viable and worth investing in. Basically, it’s a small part of a minimum viable product (MVP). In most cases, a PoC can take around 1-3 weeks to create for augmented reality applications.
After this, your development team will be ready to proceed with MVP to add additional features to demonstrate the viability of the project. This might take several months and will involve connecting the MVP to backend databases, which might include information about material and service prices, and integration with your content management system (CMS). This will give you a more realistic representation of what your AR measurement application may look like later on.
Home Scanning: Case Study for AR Room Measurement App
There are many AR projects in MobiDev’s portfolio, and the lion’s share of them are related to measurements. Here’s an example of an AR room measurement app that our team worked on.
Our client was a United States-based construction and renovation firm. They needed a solution for iOS devices that would enable their customers to visualize refurbishment in augmented reality. Once measured and visualized, the client would be provided with an estimate for the renovation cost.
The clients were given the freedom to choose materials and objects from the catalog to help them configure their estimate to their budget and needs.
This project had a number of challenges that needed to be overcome, such as:
- Providing accurate measurements
- Developing the cost estimation feature
- Giving users the opportunity to mix different materials in augmented reality
Providing Accurate Measurements
Ensuring highly accurate measurements was essential. Too much variance in measurements could create a wrong idea about the price and materials needed by the client to accomplish their goals. At the time of this case study, Apple ARKit RoomPlan was not yet released. To tackle this challenge, MobiDev’s team developed a custom solution. The user would point the camera at a corner of the room, choose the walls, and the corners would be automatically detected. Since RoomPlan was released, we have updated the app to make it more accurate and easy to use.
Developing the Cost Estimation Feature
To provide users with a quote for the price of the materials and other objects selected, our team designed the app to calculate the area of walls and floors. It then subtracts the area of windows and doors using the shoelace formula (Gauss’s area formula).
In the next stages, we plan to integrate a database with a catalog of goods and prices with APIs. This will allow the app to correlate the received area with the prices of materials to get rough estimates.
Choosing Different Materials in AR
Users find it helpful to be able to compare different materials together simultaneously to make a decision. Making it so users could compare more than one material in this way was challenging, but we were able to create a custom solution to achieve our goal.
To accomplish this, our team developed a special algorithm and ARKit’s graph data structure. This allowed us to divide the floor area into an unlimited number of zones and set the appropriate material for each of them.
Note the tiling UI that allows users to mix materials in a grid in AR. You can find out more details about this case here.
Read also:
AR Solution for Home RenovationMobiDev: AR Experience to Move Your Project Forward
MobiDev has been creating AR projects since the launch of ARKit in 2017 and we have expertise with challenges of varying complexities. We have experience implementing not only 3D visualization but also incorporating it into complex systems capable of providing extensive functionality for a variety of business needs. Where existing AR tools have limitations, our AI experts get involved to create custom algorithms and breakthrough for success.
Now that you have made plans to move forward with your project, we’re here to help bring the AR expertise you need to the development stage. Check out our augmented reality development services and contact us today when your team is ready to make that next step into the future.
