Analysis and Synthesis of Face Image

Just after Morishima-lab. was founded in 1987, topics about “face analysis and synthesis” formed a basic streams which was originated at “Model-based Image Coding” and the ancestor of this team produced “Future Cast System” and “Dive Into the Movie” as a pioneer of an innovative entertainment system. Recently, this team is challenging the wide variety of research topics about analysis and synthesis of facial expression, lip synchronization, face aging simulation, modeling of personalized face attribute and gaze simulation.

Motion Analysis and Motion Retargeting

The purpose of our motion analysis and retargeting research is to produce a new value in captured motion. Not only transferring one motion to another character with original impression, but modifying an original motion to add a new emotion, to emphasize a limited feeling of cell animation and to append a professional sense, skill and arrangement to original dance motion have been challenged in this group.

Creation and Viewing Assistance System for Video and Music Content

There are huge number of audio visual content on the Web site like YouTube or NicoNicoDoga and the number has been expanding drastically day-by-day. To find and get the one’s most favorite music or video content quickly and efficiently is one of the most important topic in data retrieval research area. Also as CGM (Consumer Generated Media) is focused recently, an amateur or novice user can have a chance to produce and upload his own content to Web site and these are becoming a new cultural phenomenon. So the needs to a content authoring tool for amateur users expected to be increasingly expanding. This project is strongly supporting this need by producing innovative technologies and authoring tools considering the drastic change of a style of content production and content viewing environment.

Sound and Speech Signal Processing

Sound and speech signal processing have been featured for a long time in our teams as well as CG, CV and content processing. Especially an automatic lip-sync animation generation from pre-scored voice has been a very essential topic for a long time. Recently, music retrieval from big data is challenged using query by music sentence played by any instrument.

Physics Based Simulation and Shape Deformation

Physics based simulations of cloth and liquid become one of the most important topics in computer graphics in accordance with the recent dramatic advances of fabrication technology. Especially, a calculation cost reduction in cloth simulation along with a garment transfer with keeping an original design and style are composing the hottest topic in this research area. Also a photorealistic facial animation along with facial soft tissue and expression muscle simulation has been a challenging topic for a long time without an epoch making solution.

3D Estimation and Reconstruction of Face Geometry

This is one of the core projects of Morishima-lab. in which Future Cast System is drastically updated and advanced to be a very affordable technology. A face geometry estimation method with light calculation cost and high accuracy only from a single picture using face prior from a collected face database is proposed at first. Then from video captured face image, a much more accurate 3D face geometry can be reconstructed by utilizing a Structure-from-Motion method to calculate 3D coordinates of sparse feature points as well as a former 2D feature points and face priors of frontal face image. And now, new trial is performed to achieve a higher depth accuracy by utilizing a normal information calculated by Shape-from-shading or by utilizing only a diagonal face silhouette curve information.

Non-photorealistic Animation and Rendering

Japanimation (Anime) is a traditional culture in Japan and one of the most popular content in the world. What is the reason anime grabs the heart of people in the world. Maybe, it’s an original world and image taste drawn by artist’s hand with Kansei. It often not only includes contradiction between 3D world and real Newton physics but original characteristics in each anime character’s appearance and motion. The goal of this project is to realize the skill and the touch of artist quantitatively, to retarget the original sense to another content and to assist a creator to produce a new content efficiently and effectively with keeping the content’s charming point and sensation. In fact, we produce an effective and directable authoring and creation tool to be used both by a professional or an amateur creator.

Measurement and Real-time Rendering of Translucent Object like Skin, Fur and Hair

CG合成における質感の再現は、重要な課題です。特に、半透明物体や細い糸状の物体などが、光を当てた時にどのような反射を起こすかを高忠実に再 現する研究は、極 めて困難な課題であり、重要な研究テーマを構成しています。これまでに、人間の皮膚や大理石のような半透明物体の高速レンダリング手法の提案からスタート し、その後、実写映像からウサギの毛皮の再現や、埃の再現、画像からの人物のアルベド推定などに取り組んでいます。

Sketching Interface for Freehand Drawing Animation 2014.6-2015.3

2D freehand animation enables viewers to intuitively experience artistry and feeling. Among these techniques, GIF animation (e.g., LINE’s stamp and Twitter icon) has attracted worldwide attention in social networks. To present a worldview using 2D freehand animation, anime-like techniques such as flip books and motion comics are employed. However, the creation of 2D freehand animations has always been a time-consuming and skill-demanding process. Software such as Adobe Photoshop provides some assistance by creating animations from a small number of key frames and generating in-between frames automatically. However, creating the numerous key frames of animation, such as those in a flip book, requires significant skill and time. Therefore, our project develops […]