M-PIRO: Multilingal personalised information objects
Key words: Coherence, relevance and appropriateness in information presentation
Personalised information objects
In a world of burgeoning information resources we need evermore responsive and intelligent ways of locating and delivering these resources. The EU Framework 5 project, M-PIRO, has been conceived to bring together leading research work on natural language generation and speech synthesis with the multimedia database, information management and virtual reality technologies being developed for public access to museum and gallery resources. In particular the M-PIRO project is developing the concept of a personalised information object, that is something which can respond to a request for information which takes account of what the requester knows already and how related information is also available.
Our chosen context is the heritage environment and the project is working closely with museums, galleries, and other "memory institutions" to develop technology which will be responsive to their special needs. M-PIRO's technology will allow textual and spoken descriptions of exhibits to be generated automatically from an underlying language-neutral database and existing free-text descriptions. The resulting descriptions, generated in three different languages (English, Greek, and Italian), will be tailored according to the user's interests, background knowledge, and language skills. For example, simpler vocabularies may be used to respond to children than to mature subject experts. The descriptions will be part of collection information systems, and will be delivered to public access kiosks in museums, over the web and in increasingly sophisticated immersive virtual reality environments.
The M-PIRO project builds on previous experience of researchers within a European consortium, and will provide many original improvements with respect to the current state of the art in natural language generation, speech synthesis, user modelling, and their interaction with adaptive hypermedia and virtual reality systems. The project also builds on existing museum collection information database systems, and will utilise the information management and design expertise from the museum partners. We take as a starting point the ILEX project, carried out at the University of Edinburgh. ILEX developed text generation facilities, and its demonstrator described a virtual collection of jewellery. These facilities are being extended in M-PIRO to support multiple languages in both written and spoken form, and to address both web-based and virtual-reality settings. More advanced user modelling facilities are also being explored, and authoring tools are being developed to facilitate the use of the system with new collections. For speech synthesis, we are developing a state-of-the art multilingual Concept-to-Speech system. We are adopting the FESTIVAL Text-to-Speech platform, also developed at Edinburgh, as a common environment for incorporating resources for Greek and Italian provided by other partners. Furthermore, we are extending the FESTIVAL system to include novel features for improving the naturalness of the synthetic speech.
Connecting with curators
Museum missions are changing from the traditional, curatorial concern for scholarly understanding of objects and their conservation to the interpretation of the objects in their historical and cultural context and making these interpretations widely accessible. The M-PIRO project will provide tools and techniques to help this interpretative process by facilitating information delivery in a number of different languages.
Keeping the visitor in mind
By tracking what visitors see and what descriptions have been presented to them, we can refine previous models of their interests and knowledge, producing descriptions that match better their needs. We can also organise our descriptions, so that they draw connections between previously seen exhibits, convey general information about styles or periods, and offer pointers to other related exhibits, thus helping the visitors obtain a more cohesive view of the presented information.
From database to description
All museums are under considerable pressure to make their collections more accessible to the public. In general this is a response to the recognised need for better understanding of, and education about, history and cultures in a pluralist, multi-culture world. There are major developments underway throughout Europe to create large-scale digital resources of museum and gallery collections. These will be linked via the web and other education networks. The M-PIRO project is designed to facilitate access to and interaction with this content.
As a part of the project, we will be developing Concept-to-Speech synthesis for English, Greek and Italian using novel techniques to increase the naturalness of speech and by tightly integrating the natural language generation system with the speech synthesis system.
In human speech, there are subtle effects, for example according to whether an item referred to is known to the speaker. In order to further enrich some aspects of the synthetic speech our approach takes into account the constraints of the domain in which the synthesised speech has to be delivered, as well as the knowledge the system has about what the hearer is acquainted with. More naturalness of the produced speech will be achieved by using trained decision trees for prosody matching and by adopting unit selection on a variety of prerecorded speech segments.
The aim of the M-PIRO project is to create systems that will complement current developments in cultural heritage IT. During the course of the project there will be rapid increase in the deployment of IT throughout museums in Europe and the world. Specialist networks will emerge and public/private partnerships will emerge to exploit culture content. M-PIRO systems will become recognised components in such ventures.
Also the M-PIRO software architecture will have application beyond the heritage environment. It is clear that catalogue descriptions of objects are ubiquitous in e-commerce and M-PIRO will provide improved means of delivery catalogue content, especially in the Internet enabled, 3G-telephone world.
This project was completed in January 2003. You can read about the results in more detail in:
Isard, A., J. Oberlander, I. Androutsopoulos, C. Matheson, 2003. "Speaking the Users' Languages", IEEE Intelligent Systems Magazine 18(1), January-February 2003, pp 40-45. Special Issue "Advances in Natural Language Processing.".
M-PIRO Partners and Funding
University of Edinburgh, Language Technology Group, UK
Istituto Trentino di Cultura, IRST - Centro per la Ricerca Scientifica e Technologica, Cognitive and Communications Technologies Division, Italy
National Centre for Scientific Research ``Demokritos'', Institute of Informatics and Telecommunications, Software and Knowledge Engineering Laboratory, Greece
National and Kapodistrian University of Athens, Department of Informatics, Speech Group, Greece
Foundation of the Hellenic World, Greece
System Simulation Ltd., UK