Twenty-five years of end-user searching, Part 1: Research findings

Abstract: Markey limits her survey of the last twenty-five years of research on end-user searching to only intervention-free studies, namely transaction log analyses. Her summary of search behavior patterns is intriguing.

She notes that these searches have few queries and few terms, ie. end-user searching is brief. Next, Mackey examines twelve claims made about advanced search features and the studies that support and fail to support these claims. Among the most relevant findings are:

• End-users respond to the results of their queries in a speedy fashion: Mackey notes that users are less likely to view results past the first 1-2 pages or to scan more than 50 results.
• Users want "early contact with the data" so they can spend most of their time browsing and viewing the data, rather than the search results pages.
• Users report a high level of satisfaction with their own unsophisticated searches.

She concludes by arguing that IR (information retrieval) is not a simple event, as end-users:

• do not use Boolean operators;
• enter 2-4 word queries;
• are satisfied by their own immediate searching;
• occasionally use the more advanced quote and plus-minus operators, but not relevance feedback; and
• have multiple contexts influencing them.

Review

I enjoyed Markey’s focus on unmediated studies of user behavior.  Yes, the limitations of mere transaction logs are significant, but the searches are certainly less influenced by outside factors.

On the other hand, some of my most useful and implementable web site changes have occurred through observation of usability tests by end-users. In fact, I think that one should do follow-home usability studies so that the user is in their natural environment and will be slightly less-inhibited in their searching.

It was interesting, but not surprising, that the relevance feedback (“see more like this”) was underutilized. Relevance feedback functions are seldom implemented well – even in Google.

And the user preference for early data contact makes perfect sense – it is the classic usability device of providing deep links for expert users and/or providing the scent of information for general and novice users.

Markey, K. (2007). Twenty-five years of end-user searching, Part 1: Research findings. Journal of the American Society for Information Science & Technology, 58(8), 1071-1081. Retrieved September 28, 2008, from Library, Information Science & Technology Abstracts Full Text.