paper-conference

Meta-Evaluation of Dynamic Search: How Do Metrics Capture Topical Relevance, Diversity and User Effort?
Complex dynamic search tasks typically involve multi-aspect information needs and repeated interactions with an information retrieval …
PDF Cite Code DOI
Meta-Evaluation of Dynamic Search: How Do Metrics Capture Topical Relevance, Diversity and User Effort?
Using Audio Transformations to Improve Comprehension in Voice Question Answering
Many popular form factors of digital assistants—such as Amazon Echo or Google Home—enable users to converse with …
PDF Cite Dataset Poster
A Formal Account of Effectiveness Evaluation and Ranking Fusion
This paper proposes a theoretical framework which models the information provided by retrieval systems in terms of Information Theory. …
PDF Cite DOI
A Living Lab Study of Query Amendment in Job Search
Errors in formulation of queries made by users can lead to poor search results pages. We performed a living lab study using online A/B …
PDF Cite Poster DOI
An Axiomatic Analysis of Diversity Evaluation Metrics: Introducing the Rank-Biased Utility Metric
Many evaluation metrics have been defined to evaluate the effectiveness ad-hoc retrieval and search result diversification systems. …
PDF Cite Code DOI
Desirable Properties for Diversity and Truncated Effectiveness Metrics
A wide range of evaluation metrics have been proposed to measure the quality of search results, including in the presence of …
PDF Cite Poster DOI
How Many Truth Levels? Six? One Hundred? Even more? Validating Truthfulness of Statements via Crowdsourcing
We report on collecting truthfulness values (i) by means of crowd-sourcing and (ii) using fine-grained scales. In our experiment we …
PDF Cite
Informing the Design of Spoken Conversational Search: Perspective Paper
We conducted a laboratory-based observational study where pairs of people performed search tasks communicating verbally. Examination of …
PDF Cite Dataset DOI
Second International Workshop on Conversational Approaches to Information Retrieval (CAIR'18): Workshop at SIGIR 2018
The CAIR'18 workshop will bring together academic and industrial researchers to create a forum for research on conversational …
PDF Cite DOI
Towards a Basic Principle for Ranking Effectiveness Prediction without Human Assessments: A Preliminary Study
We present in this paper a preliminary study about the Observational Information Linearity (OIL) assumption as a basic principle that …
PDF Cite