Item Details

An Empirical Performance Evaluation of Relational Keyword Search Systems

Coffman, Joel; Weaver, Alfred
Format
Report
Author
Coffman, Joel
Weaver, Alfred
Abstract
In the past decade, extending the keyword search paradigm to relational data has been an active area of research within the database and information retrieval (IR) community. A large number of approaches have been proposed and im- plemented, but despite numerous publications, there remains a severe lack of standardization for system evaluations. This lack of standardization has resulted in contradictory results from different evaluations, and the numerous discrepancies muddle what advantages are proffered by different approaches. In this paper, we present a thorough empirical performance evaluation of relational keyword search systems. Our results indicate that many existing search techniques do not provide acceptable performance for realistic retrieval tasks. In particular, memory consumption precludes many search techniques from scaling beyond small datasets with tens of thousands of vertices. We also explore the relationship between execution time and factors varied in previous evaluations; our analysis indicates that these factors have relatively little impact on performance. In summary, our work confirms previous claims regarding the unacceptable performance of these systems and underscores the need for standardization�as exemplified by the IR community�when evaluating these retrieval systems.
Language
English
Date Received
20121029
Published
University of Virginia, Department of Computer Science, 2011
Published Date
2011
Collection
Libra Open Repository
Logo for In CopyrightIn Copyright

Availability

Access Online