Citation:
Blocks, D., Binding, C., Cunliffe, D., & Tudhope, D. (2002). Qualitative Evaluation of Thesaurus-Based Retrieval. In Research and Advanced Technology for Digital Libraries, Lecture Notes in Computer Science (Vol. 2458, pp. 75-107). Springer Berlin / Heidelberg.
Goals of Research:
- To analyze at a micro level the user’s interaction with interface elements and reasoning in order to illuminate tacit sources of problems and inform iterative interface design decisions.
Research Questions:
- What are the problems at different search stages if we want to integrate the thesaurus into the search process?
- What are the pros and cons of FACET information retrieval interface?
Methodology:
- Qualitative research method
- Pilot study with eight participants
- Data gathered included transcripts of think-aloud sessions, screen capture movie files, user action logs and participant observer notes
- User reasoning and interaction sequences has been studied to achieve a better insight of benefits and problems of FACET interface
- Data about incidents during search process has been analyzed to identify the possible reasons behind problems
Summary:
- FACET is an experimental system that investigate the possibilities of term expansion in faceted thesauri based on measures of semantic closeness. Getty Art and Architecture Thesaurus has been used for this purpose. FACET calculates a match value between a query term and an index term depending on traversal cost. This in turn feeds into a matching function that produces ranked results, including partial matches, from a multi-term query (p. 348). This study is a formative evaluation of a prototype thesaurus -based retrieval system. Qualitative method has been used to study user search behavior. Two prototypes has been used in one hour sessions and participants complete four search tasks.
Findings:
- Four major incidents have been observed, window switching, incidents in browsing behaviors, breaking a task down into concepts, and reformulation of query
- Result of analysis led to important issues such as, the allocation of search functionality to sub-windows, the appropriate role of thesaurus browsing in the search process, and the formation of faceted queries and query reformulation. Some lower level interface issues also have been noticed
Future Works:
- “An integrated Query Builder tool, which combines searching/browsing with query formulation and maintains the top level facets, [to] better reflect the search process and the thesaurus as a source of terms for the query.” (p. 360)
Important References:
- Tudhope D., C. Binding, D. Blocks and D. Cunliffe. Compound descriptors in context: a matching function for classifications and thesauri, Proceedings Joint Conference on Digital Libraries (JCDL’02), Portland, ACM Press, 2002.
