New multimedia "recognition" software in development

System 'spots multimedia content'

The system can recognise a picture of a crowded beach

A smart system that can recognise and label the content in photos and videos is being developed by researchers. They believe it will help people to organise, find and share the mass of multimedia data being generated by cameras, camcorders and phones.

The EU-partnered project, carried out by the Acemedia consortium, is to be showcased at the Information Society Technologies meeting in Helsinki.

The technology is currently in a prototype phase.

People don't want to have to spend time managing their content manually, they just want to be able to view it whenever and however they want

Paolo Hobson

The system, which runs on PCs, uses image technology and knowledge analysis tools to scan images looking for collections of pixels that make up recognisable shapes, such as people or faces.

It is able to identify and label the individual components in a scene, for example a holiday snap of a day at the seaside could be tagged with sky, sea, rock, people etc.

However, by collating these different components, it can also give the scene a broad theme - in the case of the holiday snap, it would be tagged "beach scene".

With moving images, it can assess and label them in a similar way, frame by frame, but can also use the information about the static and moving components, for example, to distinguish between a stationary rock and a moving car, to determine the reference tags.

Easier to find

The idea behind the system, said Paola Hobson, the Acemedia project coordinator who is based at Motorola Research Labs, was to give people the framework to organise their multimedia content.

Currently, tags for pictures and films need to be added manually, and as the multimedia data we generate increases apace, this can be time-consuming.

"People don't want to have to spend time managing their content manually, they just want to be able to view it whenever and however they want," said Dr Hobson.

"For that to happen, multimedia content needs to become intelligent."

The concept, she said, was to have an easier way of organising their still and moving images, and to enable people to search for their multimedia content using simple keywords, such as beach, to pull up all of the content that matches the search-term.

She said this would also make it simpler to groups and share images with friends and family.

Acemedia, which has 13 industrial and academic partners, is also working on accessing this information from a set-top-box, linked to a PC, or a mobile phone.

So far, the system it has created is limited to certain scenes, such as a beach or a tennis match, but the researchers say it is a proof-of-concept and are seeking to apply it to more domains.

And other projects are also underway attempting to do this: a research consortium, called MESH, is looking at ways to apply the technology to news images, making them easier to search and group.

Dr Hobson said the ultimate aim would be to lead to automatic analysis technologies that could recognise and label any scene and its contents, and this, she believed, in the future would be possible.