In the age of 3D avatars!

In the age of 3D avatars!

In recent years, the evolution of semiconductors has greatly evolved, and 3D data and peripheral technologies have dramatically evolved. Humans are about to enter an era where they can do a lot of work and chores by relying on 3D data avatars. In the age of avatars, you can do multiple things at the same time by duplicating as many times as you need, when you need them. This time, I would like to introduce an article about creating an avatar.

I investigated how to make a 3D model of a real avatar human by 3D scanning, so I summarized it.

Hello. I'm Tanaka from Next System.

It is sudden,

Do you want to be a real avatar?

What is a real avatar in the first place?
An avatar that is a 3D model of yourself in the real world.
* Although it is also called a live-action avatar.

When creating a real avatar
Many people go to a studio surrounded by a camera 360 degrees and have them scan.
I'm making a real avatar, but can't I do this easily?

As I introduced in the article I wrote earlier,
In order to become a real avatar in the explosive hit social VR "VRChat" from this year, we have previously searched for various ways to scan humans in 3D and make them avatars.

A story of trial and error to make a real avatar and play with VRChat

Recently, it is difficult to go to the studio to scan in order to become a real avatar with VRChat, so I tried to blog a story that I tried and errored to see if I could become a real avatar in some other way. ..

After all, in the above article, I chose a method of simply 3D modeling a human figure instead of 3D scanning and pasting the image of the whole body photograph taken before as the texture from above.

But! !!

I also want to make a real avatar using 3D scanning!

During this time, I couldn't find a good way, so I used an alternative.
Isn't it possible to find a good way to become a real avatar even by using 3D scanning?

so! !!

I investigated how to scan humans in 3D without going to the studio and summarized the results.

Not just for the purpose of becoming a real avatar in VRchat
With the focus on VR and MR fields in the future, there seems to be an opportunity to turn human figures into 3D avatars.

Use Structure Sensor
A compact 3D shape scanner that attaches to the iPad. You can easily use it by using the corresponding app!

Structure by Occipital --Give Your iPad 3D Vision

With the Structure Sensor, the future is in your hands. Rapid 3D scanning of objects and people, 3D maps of interior spaces, and mixed reality experiences where ...

There were several compatible apps, so I actually used them and compared them.

itSeez3D
I was able to scan cleanly.

For non-members, it costs $ 7 (free version) to export one body, or $ 250 a month for 100 bodies a month / 1200 bodies a year (more than that, $ 4 per body), but the quality From the point of view of removing the floor, the combination of iPad + Structure Sensor + itSeez3D seems to be good.

How to use

Prepare an iPad with StructureSensor installed and itSeez3D installed
Launch itSeez3D and follow the app's instructions to scan your entire body
・ It takes about 2 minutes to scan
・ At this time, it is better to have many feature points around (floor and wall).
・ At this time, it is better to be in a place that is not affected by sunlight.
・ When moving around, it is better to show your feet
Send scan data on mobile devices to the cloud for processing
・ It takes about 10 minutes
Download the model via email or itSeez3D web account
-Select the "OBJ" format, which is the file format compatible with the software used in step 2.
・ Model download is charged
・ Unlimited number of 3D models that can be saved in the cloud
・ Model data according to capacity can be saved on the iPad
* The state before processing on the cloud varies depending on the scan time, but 200 to 300 MB,
Data is saved in a size of about 10 to 20MB after processing on the cloud

The impressions of the employees surveyed are as follows.

Since it is a type that captures the face in the first shot and then scans it in 3D, the face can be taken beautifully.
Unlike other apps I investigated, I was a little worried because I didn't see any warnings such as "moving too fast", but I was able to take better pictures than I had imagined.
Scan results

It seems that the way to shoot the crotch part was a little sweet, but if you take care, it will be beautiful.

ItSeez3D + structure sensor (Full body) by nextsystem on Sketchfab

Detailed information

Price ・ $ 7 per non-member
・ 100 members ($ 250 / month) per month ($ 4 per member)
Export format PLY, OBJ, WRL
With automatic floor deletion function
URL https://itseez3d.com/
Recommendation level ◎
Remarks ・ Processing in the cloud is required to obtain accuracy (it takes about 10 minutes)
・ Two patterns of writing from the web and sending by email
・ The amount at the time of writing varies depending on the license

Skanect
The texture is out of focus overall.
There are many shapes that can be taken reasonably well, but it seems that it is quite difficult to shoot the texture clearly.

As for the usability, it was difficult to judge whether it was possible to scan accurately or take a picture because it is not displayed on the iPad how far it was able to scan during scanning.

There are two types of export methods, and in the case of Very High, it seems that appropriate PC specifications are required to improve the quality.

Scan result: When exporting in Medium version

It's totally vague.

Structure Sensor + SKANECT by nextsystem on Sketchfab

Time required
Calibration time: Approximately 2 minutes
Scan: about 3 minutes
Processing / Export: Approximately 4 minutes
Recommendation level △
Scan result: When exporting with Very High version

It is clearer than the Midium version. I think it's mostly taken, but the texture borders look a bit jerky like a watermelon.

Structure Sensor + Scanekt: Very High by nextsystem on Sketchfab

Time required
Calibration: Approximately 2 minutes
Scan: about 3 minutes
Processing / Export: Approximately 18 minutes
Recommendation level △

Detailed information

Price Pro license for $ 129
* 1 Pro license is included for each Structure Sensor.
Export format PLY, OBJ, STL, VRML
No automatic floor deletion function
Remarks
Works on Windows and mac
Streaming from ipad to PC via Wifi
Compared to itSeez3D, it's hard to scan because you're looking at the monitor
Can be exported free of charge ・ Only for the first time, it is necessary to calibrate with the Structure Sensor Calibrator app.
Up to 5000 pages can be exported in the free version
This restriction is removed with a Pro license (scan URL is the result of the Pro version)
The accuracy differs depending on the resolution setting, but the processing time also differs. ・ Manual floor deletion is possible.

Scanner
I tried it with a sample app published by a company called Occipiatal that made Structure Sensor.

As for the impression I used, the texture of the first part I took at the beginning of shooting was beautiful, but the other parts were blurry.

An email address is required for 3D model output, but since email was not set on the company's iPad, I'm shooting a video for the time being.

Scan results

I have a face! !! !! The texture of the back half is strange.

Detailed information

Price App = Free
Export format OBJ, STL, PLY
No automatic floor deletion function
URL https://structure.io/apps
Remarks ・ Automatic floor deletion
・ Color is possible, but the texture is considerably distorted
Reference: https://www.youtube.com/watch?v=nFyT2q8f_i4
・ E-mail settings are required for output

M3D Scan
I tried it a little during the free period, but the quality seems not to be so different from the Scanner.

After all it seems that it is a high hurdle to shoot the face firmly and clearly.

Scan results

Detailed information

Export format OBJ, STL, PLY
No automatic floor deletion function
URL http://www.mirage-3d.com/
Remarks ・ Partial or whole body scan is possible
・ Color scan settings are possible
・ Coarse texture that reads the floor
・ E-mail settings are required for output

Captevia
It looks like a 3D scan app originally made for orthopedics.
The thumbnail image of the 3D avatar list displayed on the screen seems to be colored, but when I preview the 3D model, the color disappears.

why···!

Perhaps it can be operated from the settings, but
I didn't have time this time, so it's left as it is.
Please let me know if anyone knows the cause.

Time required ・ About 2 minutes for scanning
・ About 1 minute for processing
Recommendation level ×
Detailed information

No automatic floor deletion function
URL http://rodin4d.com/en
Recommendation level ×
Remarks ・ E-mail setting is required for output (or iTunes)
・ Can only be output in a unique file format
・ The texture is rough, which reads the floor.
-It is a color scan at the time of scanning, but it cannot be displayed in color on the preview screen.

3DfitME, 3DsizeME
This also seems to be a scan app developed for medical purposes.
It seems that there are two types, fitMe and sizeMe, but when I tried using them, I couldn't tell the difference.

It seems convenient to be able to upload to Dropbox, but because it is a unique file format,
It may be difficult to divert.

Scan results

Am I the only one who thought that the legs were like grilled eggplants?

Time required ・ About 2 minutes for scanning
・ About 1 minute for processing
Recommendation level ×
Detailed information

Export format imed
No automatic floor deletion function
URL https://techmed3d.com/
Recommendation level ×
Remarks ・ No difference has been found (separated as soon as it is discovered)
・ Can be output to Dropbox
・ Can only be output in a unique file format
・ The texture is rough, which reads the floor.

RealSense D415 and D435 recently released by Intel.

It's so popular that it's still hard to get, but it's a very good camera.

I tried each with one RealSense D415 and one upward compatible ver. D435.

Scan results
seems strong. What is this metallic feeling?

RealSenseD415 + RecFusion by nextsystem on Sketchfab

Time required about 3 minutes
Recommendation level △
Detailed information

Price RecFusion = 99 €
RealSense D415 = 24,300 yen
RealSense D435 = 28,350 yen
Export format PLY, OBJ, VRML, STL
No automatic floor deletion function
URL https://www.recfusion.net/index.php/en
Remarks ・ Scan with real sense in your hand
・ RecFusion is scanning software for one sensor
RealSence D415 x 3 + RecFusionPro
We also got the RealSense D415 and tried to see if it could be scanned at a fixed point.
Three units are installed at different heights.

Scan results
It's an impression that it's taken pretty well.
However, it seems that the machine specs are quite important when there are 3 fixed points, and it was an impression that the accuracy could not be improved if it was less than the following specs.

■ PC specs
CPU: i7-8700K, GPU: GeForce GTX1080Ti
* When I took a picture with my mouth slightly inflated to hide my protruding teeth, my jaw disappeared.

Time required 60 to 80 seconds
Recommendation level ◯
Detailed information
Price RecFusion Pro = 499 €
RealSense D415 = 24,300 yen
RealSense D435 = 28,350 yen
Export format PLY, OBJ, VRML, STL
No automatic floor deletion function
URL https://www.recfusion.net/index.php/en
Remarks ・ RecFusion Pro is scanning software for multiple sensors.
・ Some quality scan is possible when scanning with a PC with high specifications (CPU: i7-8700K, GPU: GeForce GTX1080Ti).
・ Scan on the turntable
・ Three sensors are fixed https://twitter.com/next_kinesys/status/978557045438783489 ″

Summary
From the viewpoint of quality and floor removal, it costs about 840 yen per body for non-members,
The combination of iPad + Structure Sensor + itSeez3D seems to be good.

A low running cost plan is a combination of iPad + Structure Sensor + Skanect, but the quality is lower than itSeez3D and there is no automatic floor removal function (manual deletion is possible).

Other than the above, a combination of RealSence D415 x 3 + RecFusion Pro is a plan that does not require running costs, but the quality is lower than iPad + Structure Sensor + itSeez3D.
Also, there is no automatic floor deletion function (manual deletion is possible).

↓ In other words, this is what it means. ↓

・ Quality-oriented → iPad + Structure Sensor + itSeez3D
・ Emphasis on running cost ① → iPad + Structure Sensor + Skanect
・ Emphasis on running cost ② → RealSence D415 x 3 + RecFusion Pro