The amount of video material that has been – and is being – produced, is enormous. The BBC Archives, for instance, date back to 1890 and include more than 1 million hours of playable material. Moreover, video archives – both public and private ones – continue to expand at a dazzling pace.
That might seem a good thing at first – as more content becomes available on a growing number of topics – but it also comes with its challenges. After all, finding the exact content you are looking for becomes more difficult and takes considerably longer – not in the least since current search engines rely on human annotations (known as ‘metadata’) to retrieve relevant material. Turning these gigantic libraries into accessible collections is a real challenge – especially since existing tools require quite some human interaction to get to satisfactory results (both before and during the actual search).
AXES: a Game-changer in Mining Video Archives
Existing search engines require upfront (manual) tagging to make the video content searchable (you can only get out what you put in). And even as search results are shown, more manual effort is required when fast-forwarding the selected videos in order to find the specific fragment you are interested in. Finally, current tools sometimes require uploading content to public databases, which is often not an option due to copyrights and privacy issues.
“But now there is AXES, a real game-changer in interacting with video archives,” says Prof. Dr. Tinne Tuytelaars – a Belgian researcher from iMinds - KU Leuven and scientific coordinator of the four-year European AXES research project. “We built an intelligent tool that allows users to search video archives without having to tag everything in advance. Bringing together the knowledge from European experts in a wide range of technology domains, we created a system that automatically recognizes buildings, places, faces, objects and events within video content. And thanks to its integrated speech recognition technology, users can also search on spoken words within the video fragments.”
“Another unique feature of AXES is that users end up at the exact video fragment that matches their query; no more fast-forwarding is required. And we addressed the upload issue: the AXES technology can be downloaded and safely applied to users’ private video collections,” she adds.
Self-learning Wizard for Media Professionals, Journalists, Researchers and Consumers
The tool targets media professionals, journalists, researchers and consumers. It comes with three interfaces that have been built according to the specific requirements of those user groups – to help each of them get the most out of their video archives.
Instrumental to the latter is the system’s ability to learn. If users insert a query such as ‘horn rimmed glasses’, the tool will first consult the Internet to learn what ‘horn rimmed glasses’ are – and use that knowledge to search the video archive; a search that is language-independent, by the way. Important efficiency savings are the result.
“Organizations such as BBC, Deutsche Welle, or the Netherlands Institute for Sound and Vision invest big-time in digitizing and storing huge quantities of historic and contemporary broadcast material. These investments make sense only if those data are easily accessible and searchable. This is a real problem when audiovisual material is concerned; a problem that was successfully tackled by AXES,” says Roeland Ordelman, research manager at the Netherlands Institute for Sound and Vision.
The AXES consortium made a large part of the technology available as open source. Users who want to experiment with the AXES technology and apply it to their own archives, can download it via http://www.axes-project.eu/?p=2433. Developers can download, configure and link AXES components via https://github.com/kencoken/axes-lite. Note – the AXES technology is meant to run on a server or a powerful laptop.
The AXES project aimed at opening up audiovisual digital libraries, increasing their cultural value and their exposure to the European public and academia at large. The resulting tools provide various types of users with new ways to interact with audiovisual archives, helping them discover, browse, navigate, search and enrich the content. More info at: http://www.axes-project.eu/
- Funding Scheme: FP7-ICT integrated project
- Project number: 269980
- Project Coordination: Philippe Rohou, ERCIM
- Project Scientific Coordination: Tinne Tuytelaars, iMinds - KU Leuven
iMinds – the digital research center of Flanders, Belgium – combines the strength of its 900+ researchers at five Flemish universities to conduct strategic and applied research in areas such as ICT, Media and Health. Together with its research partners (companies, governments and non-profit organizations), iMinds translates digital know-how into concrete products and services. In addition, iMinds supports researchers, young entrepreneurs and start-ups in the successful market introduction of their ideas.