Pubmed limbo

In my experience, PubMed works beautifully the vast majority of the time. It does an excellent job parsing search terms and they’re always adding new features (e.g., you can search using full names now, not just first initials). PubMed works so well, that I’m actually surprised when it fails to find the paper I’m looking for. But it does happen.

LSTOTT has a great post about articles lost in PubMed limbo. It’s a real phenomenon. They also identify an article which is in the database, but does not get returned using standard searches that should match. Which happens more often, in my opinion.

Database jocks call this latter problem an indication that recall < 1. (The former problem is just a mistake in QA, that is, someone forgot to include the article). This is the proportion of relevant documents that are actually returned. LSTOTT thinks PubMed's recall may be declining (they call it "leaky"). What do you think? Google Scholar does an excellent job of finding well-cited articles, including the ones PubMed misses. This is because there is no one point of failure that can prevent indexing: if the article is cited, then Google will index it. But this strength is also its weakness: relevancy is often sacrificed in favor of citation counts. Furthermore, the output is ordered by citation counts, which is not typically a useful parameter when I'm searching for a paper. PubMed's reverse chronological ordering is better. But really, both systems should make it easier to re-sort the results.