There are huge number of audio visual content on the Web site like YouTube or NicoNicoDoga and the number has been expanding drastically day-by-day. To find and get the one’s most favorite music or video content quickly and efficiently is one of the most important topic in data retrieval research area. Also as CGM (Consumer Generated Media) is focused recently, an amateur or novice user can have a chance to produce and upload his own content to Web site and these are becoming a new cultural phenomenon. So the needs to a content authoring tool for amateur users expected to be increasingly expanding. This project is strongly supporting this need by producing innovative technologies and authoring tools considering the drastic change of a style of content production and content viewing environment.
The target of research is to produce several authoring tools for all kinds of content like music, video and music video. “LyricsRadar” is a visual music navigation system by a query of lyrics semantics to search the most favorite song very quickly by mouth operation on the lyrics radar chart. “RecoMusi” is a music recommendation system considering an atmosphere included in an input image. “Query by Phrase” is a music retrieval system based on a query of short melody played by any instrument without any musical score. An automatic video summarizing system of racket sports can produce a very short movie (10 minutes) from a long (a couple of hours) on-air sports content by an automatic priority criteria of human interest. Human retrieval system is proposed to search an impressive photo from huge database on the Web based on the impression of hair color, style and direction. Music video viewing system is proposed to pick-up video shots which include a favorite artist face or singing voice. As an authoring system, “VRMixer” is produced in which the boundary between real world and virtual world is eliminated to share the common space and play together at the same time with people located remotely. Automatic synchronization system between music climax and video climax is proposed by controlling the length of music adaptively. “DanceReProducer(DRP)” is a typical authoring system which can produce an appropriate movie automatically synchronizing with the input music by rearrangement of copy and paste of existing video segment on the Web.
This project is supported by JST OngaCREST project.
- 研究機関 産業技術総合研究所チーム、早稲田大学チーム、明治大学チーム、京都大学チーム
Each point on the space is corresponding to a music piece and then music with closer impression are located closer on the space. When mouth-over a point in the space, semantic and music information are displayed in a radar chart with real-time.