Immediate introduction of AR system!

Immediate introduction of AR system!

We are developing an AR automatic ordering system (patent). It is possible to correspond to any industry, especially when selling products, it is a system that allows users to check the actual size with AR and automatically place an order. This time, I would like to introduce an article about AR apps. Click here for inquiries regarding development requests.

For web app developers to create mobile AR apps

The word "AR (augmented reality)" was known only to a small number of researchers and engineers, and a situation is emerging in which public awareness is gradually spreading. For example, there are an increasing number of familiar applications such as car navigation systems equipped with a function named "AR" and portable game machines also equipped with a function that allows you to experience AR.

Although it is becoming more common, actually saying "Let's make something using AR", "What to start with" "How to start" Even if you are interested, step on Isn't there some readers who don't?

The development team to which the author who is in charge of this series belongs is also a team belonging to SIer who can not imagine AR from its name, and also specializes in technologies closely related to AR. It's a group of people who aren't (for reference, most of the members came mainly from web development, and until a while ago, using Ruby on Rails etc. I was making a very normal web app).

In this series, we will share with you the experience that the authors actually tried after thinking about working on AR, and the hurdles for those who are thinking "Let's step into the world of AR from now on". I hope it can be lowered even a little.

Over the next few times, I will explain how to create apps using AR, especially on mobile terminals such as smartphones and tablets. Keeping in mind that you can get started right away, we plan to focus on methods that mainly use open source libraries.

First of all, after briefly reviewing what AR is, we will talk a little about application examples that our team is thinking about working on AR, and briefly introduce the contents of each future series. ..

What is AR (Augmented Reality) in the first place?
What exactly is "AR"? It may not be necessary to explain much anymore, but I will briefly review it.

AR concept

The word "augmented reality" is often used as the Japanese translation of AR Augmented Reality. Speaking of the original English word "augment", is it close to the feeling of "intentionally increasing something"?

Quoting the beginning of the explanation of AR on Wikipedia, it is "a technology that additionally presents information using a computer in the real environment, and a term that refers to the environment itself in which information is additionally presented."

By adding "something" that supplements the real world, I think that the technology that presents more information than the reality in front of us and the environment itself represented by that technology will be called AR. I will.

Although not specifically covered in this series, this definition not only adds visual information, but also presents auditory information by voice, or is olfactory with something like odor / scent. It may be considered that the presentation of such information is also included in the category of AR.

Affinity with mobile

I think that the spread of mobile devices such as smartphones and tablets has greatly contributed to the background that the word AR has become familiar. As I mentioned earlier, AR is a technology that adds information to the real world. In that respect, mobile devices that can be carried around and are always experienced by the person carrying them can be said to have a very high affinity as a tool for realizing AR.

From the perspective of creating a mechanism to realize AR, many mobile devices that have become generally available in the last few years are also equipped with cameras and various sensors. Moreover, the ability when viewed as a "computer" has also improved, so it can be said that it is becoming equipped with sufficient ability to enrich the expressions to be added to the real world.

In addition, with the inclusion of general-purpose platforms such as Android, it has become relatively easy to develop devices that access native devices.

From now on, when we say "AR", we will use it on mobile terminals from the theme of this series unless otherwise specified.

Three main technical AR implementation methods
By the way, this series deals with things that visually add information, but even if you say that you add information visually in a bite, there are technically some techniques. I think this part will be a review, but I will briefly explain the differences between these techniques. First of all, if you look at it roughly, it can be divided into two because of its technical characteristics.

One is called "location-based AR" that presents information using location information that can be obtained from GPS, etc., and the other is directly in front of you by applying technologies such as image recognition and space recognition. It is called "vision-based AR" that presents information by recognizing and analyzing the environment in.

Vision-based AR can be further divided into two types. One is a "marker type" that presents information by recognizing a fixed-shaped figure called a "marker", and the other is not a fixed-shaped figure but an object or space itself that actually exists in the real environment. It is a "markerless type" that recognizes and identifies, identifies the presentation position based on it, and makes information appear.

I will summarize a little more about each technical feature.

[1] Location-based AR

As mentioned earlier, location-based AR displays additional information by linking it to location information that can be obtained from GPS, etc. The place to present the information is decided based on not only the position information by GPS etc. but also the direction by the magnetic sensor (the direction in which the person who is trying to see the information is facing) and the inclination by the acceleration sensor (elevation angle and depression angle of the line of sight). I am.

The merit is that elemental technologies such as location information acquisition and orientation / tilt acquisition are relatively easy to handle with currently available devices and platforms, so if it is basic, it is special. It can be realized without using a separate library.

On the other hand, as a disadvantage, there is a large part that relies on GPS as the main location information source so far, and there is a possibility that the accuracy of the display position of additional information will shift. Recently, the error has been reduced due to the performance improvement of the GPS processor installed in the terminal and the addition of correction information, but it is difficult to completely prevent the deviation from occurring.

However, if the quasi-zenith satellite system and Wi-Fi-based location system, which are currently talked about, reach a level that can be used on a daily basis, this point should be resolved, so we will continue to improve these technologies and environments. I would like to expect it.

For location-based services, overseas AR platforms such as "Layer" are famous.

Layar Browser — Layar via kwout

[2] Marker type vision base AR

The marker type AR identifies the presentation position of the additional information by being recognized, and uses the marker that is the key for the appearance of the additional information. A marker is a figure with a specific pattern to be recognized.

Normally, when realizing marker-type AR, it is common to use a library or engine that has the function of recognizing markers and presenting specific additional information to the recognized markers. What the marker pattern should look like depends on the library you are using.

As a merit, since it is possible to determine the presentation position of additional information by placing a marker, it is possible to accurately present additional information to the place you want to present, and the OSS library that can be used immediately is open to the public, so it is relatively relatively It is easy to work on.

As a library, "ARToolKit" developed by Mr. Hirokazu Kato of Nara Institute of Science and Technology is famous, and there are many other publicly available libraries based on this. As I will briefly introduce later, "NyARToolKit" and "AndAR" used in this series are also based on ARToolKit.

ARToolKit Home Page via kwout

Disadvantages are that it is difficult to place the marker as a physical place or environment because the marker must be prepared separately in printed matter and the marker itself must be placed in the actual environment. In some cases (in terms of problems, landscape, etc.), it cannot be used.

[3] Markerless vision-based AR

Markerless AR identifies and presents the presentation position of additional information by spatially recognizing the object existing in the real environment and the environment itself without using a specific marker. ..

The merit is that there is no need to prepare a special marker separately, and additional information can be presented pinpointly to places where it is difficult to place markers due to physical space or landscape problems, or to the landscape itself. .. "PTAM" researched and developed at Oxford University and "SmartAR" developed by Sony are known in Japan. Regarding PTAM, it is not for mobile, but the source code is open to the public, so you can try it out.

Marker-free AR technology "SmartAR" developed by Sony --ITmedia News via kwout

Disadvantages are that the amount of calculation is inevitably large in spatial recognition and object recognition, so the demand for hardware capabilities is high, and while satisfying detailed requirements and maintaining accuracy, and paying attention to the amount of calculation. When it comes to this, it is necessary to have considerable specialized knowledge about spatial recognition and object recognition.

In this respect, it is difficult to reach out easily. It can be said that it is the most technically difficult method so far.

Table: Summary of three main AR implementation methods from a technical perspective

Location-based

AR Vision Base AR

Marker type Markerless type
Element technology Location information Marker recognition Object recognition
Realization method Superimpose additional information on the position information acquired from GPS etc. Superimpose information on special two-dimensional data called "marker" Recognize specific objects and figures and superimpose information
Advantages ○ Can be realized by combining relatively dead technologies such as GPS, magnetic sensor, and acceleration sensor
○ Relatively easy to implement ○ Pinpoint information superimposed on a specific object
○ Technically mature
○ Easy to try with OSS library ○ Information can be superimposed on a specific object in a pinpoint manner
○ No need to prepare special markers
○ Applicable to places where markers cannot be placed (do not want to be placed) such as landscapes and objects
Disadvantages ● Depends on GPS accuracy (accurate positioning requires a combination of multiple technologies) ● Physical / psychological barriers to preparing markers ● In all aspects of computational complexity, stability, and accuracy There are challenges and it is not easy to try
● Academic knowledge is often required