Which journals are the most influential? The answer obviously depends on the individual who is answering, and is sensitive to variables like one’s field, subfield, position, and recent professional history — but that doesn’t stop people from compiling lists of the most important publications within broad disciplines like “biology”.

The methodology by which one makes the comparison is also important. If one proceeds by “impact factor” (a complex, proprietary, and increasingly challenged function of how frequently papers in a given journal are published), the answers tend to converge on a few very high-profile journals. If one takes a poll of librarians and experts chosen by librarians, the resulting list overlaps somewhat with the one derived from impact factor, but with far greater diversity (possibly as a result of honoring journals that were once great but whose readership has fallen off somewhat in the past few decades). Neither list mentions open-access journals at all.

But citation is just one means to measure the importance of a paper. Speaking personally: there are lots of papers that change the way I think, and contribute tremendously to the intellectual “backstory” of my projects, but never end up getting referenced in a primary paper. Beyond that, citations take a long time to accumulate; they generally don’t even start appearing until more than a year after a paper is published.

What if we were able to measure the actual use of a paper by scientists, irrespective of whether they eventually got cited? One could measure the rate at which papers were downloaded from journal websites, and indeed this is already being proposed as an alternative metric of journal impact.

There’s also a lot of free information floating around on social bookmarking sites like Connotea and Mendeley. One could ask which journals are publishing the articles most likely to be bookmarked and shared by users of those sites. Doing so reveals that open-access journals may be a good deal more influential — in the sense of actually being read by a large number of working scientists — than predicted by conventional metrics.

Of course, all of this discussion presupposes that there is some reason why we need to pick a “best” journal at all. Good search engines, in conjunction with rapid indexing of the primary literature, have greatly flattened out the landscape — however, at the same time, the proliferation of journals have caused that same landscape to greatly expand. We need some filter on the literature, but it’s increasingly unclear to me whether selecting papers to read based on the brand name on the journal’s cover (which I never see anyway) is a good solution to that problem.