An Infrastructure fo Supporting the Evaluation of Interactive Information Retrieval

Norbert Fuhr, University of Duisburg-Essen, Germany

A testbed for the evaluation of interactive information access consists of three components: (1) a collection of documents, (2) a set of tasks/usages, and (3) a system. Whereas in most evaluation initiatives only the first two components are provided by the organizers, evaluaiton of interactive retrieval requires some standardization on the system side, too. Starting with the INEX interactive track in 2004, our group has developed an infrastructure for this type of evaluations. With the Daffodil (now ezDL) framework, we provided an experimental framework for interactive retrieval, that allows for easy exchange or extension of the system components. Moreover, this framework also contains tools for organizing lab experiments: Besides extensive logging (including the possibility to exploit eyetracking data), the system also allows for presenting questionnaires at all stages of a search session (pre-/post- task/session), as well as the scheduling of search tasks and monitoring task time. While the search frontend can be employed decentrally at participating sites, all data is collected in a centralized database, from where it can either be exported in XML format or directly analyzed via appropriate evaluation routines.

Bio: Norbert Fuhr studied technical computer science and received a PhD (Dr.-Ing.) from the Technical University of Darmstadt, Germany in 1986. He became associate professor at the University of Dortmund in 1991. Since 2002, he is full professor at the University of Duisburg-Essen, Germany. He has been a co-founder of the INEX initiative for the evaluation of XML information retrieval in 2002 and been involved in the organizing of INEX's interactive track since 2004.
His current research interests are information retrieval models, user interfaces for retrieval systems, and their evaluation.

Ontology-based data management

Maurizio Lenzerini, Sapienza University of Rome, Italy

Ontology-based data management aims at accessing and using data by means of a conceptual representation of the domain of interest in the underlying information system. In this talk I will provide an introduction to ontology-based data management, by illustrating the main ideas and techniques for using an ontology to access the data layer of an information system. After an introduction to the notion of ontology in this context, I will describe an architecture for ontology-based data access, I will discuss the issue of choosing the appropriate language for expressing the various components of the architecture, and I will illustrate the main advantages one gains in managing the information system through the ontology. Finally, I will discuss the issue of developing methodologies and tools for the design and usage of ontology-based data management solutions

Bio: Maurizio Lenzerini is a professor in Computer Science and Engineering at the Università di Roma La Sapienza, where he is currently leading a research group on Artificial Intelligence and Databases. His main research interests are in Knowledge Representation and Reasoning, Ontology languages, Semantic Data Integration, and Service Modeling. His recent work is mainly oriented towards the use of Knowledge Representation and Automated Reasoning principles and techniques in Information System management, and in particular in information integration and service composition. He has authored over 250 papers, and has served on the editorial boards of several international journals, including Journal of Artificial Intelligence Research, Information Systems, IEEE Transaction on Data and Knowledge Engineering, and Logical Method in Computer Science. He is currently the Chair of the PODS (Principles of Database Systems) Executive Committee, and he is a Fellow of ECCAI, and a Fellow of the ACM.