The Care-ful Reviewer

Peer Review as if People Mattered

John W Maxwell 

John W. Maxwell is Associate Professor & Director of the Publishing Studies Program at Simon Fraser University in Vancouver. John's research has focused on scholarly publishing innovation, the history of media technology, and the cultural trajectories of personal and educational computing. John often teaches a course on text processing histories at the Digital Humanities Summer Institute. He has been at work in new media since the early 1990s, especially in web development, educational technology, SGML & XML, and content management.

Simon Fraser University

jmax@sfu.ca

November 09, 2022 

For many years now, the open social scholarship community in Canada has examined its practices and capacities for scholarly communication in the digital age, both in terms of making the scholarly discourse richer, more efficient, and more responsive, and with an eye to making scholarly discourse in the humanities more relevant and interesting to audiences outside our specific disciplines and indeed the academy itself. Attention to "new knowledge environments" has proved both fruitful and inspiring, but the scholarly community remains rooted in a set of very traditional scholarly communications forms/practices: conference presentations, journal articles, and books. These traditional forms are rooted in---even arguably constitutionally defined by---peer review practices. Whether these traditional forms have bright futures in the digital age is a topic for another discussion, but it seems fair to argue that peer review itself is and will continue to be a constitutional component of scholarly communications.

What to make of peer review, then? As an artifact largely of the twentieth century and the late age of print, we might expect its role to shift in new, digital formats and genres, and its form and function to be responsive to disciplinary and methodological innovations. And yet, there is a sense in which peer review remains a stubborn, poorly understood, and ritualized practice. We generally lack good conceptual models of the what, the how, and the why of peer review practices, even as we consistently uphold their centrality to scholarly work.

As such, this essay is an exploration of peer review in theory and practice, and an attempt to work out what it might mean in the context of the humanities specifically, and especially in terms of open social scholarship. In this essay, I take the scholarly journal as the fundamental case, and as such much of the discussion that follows is an appraisal and attempted re-imagining of some fairly conventional forms. My aim here is not to praise or condemn peer review itself, nor any particular flavour or format of it in practice. Rather, the goal is to understand what peer review might ideally be for in open, humanities scholarship: how we might think about it, how to identify the precepts upon which our practices might be founded, and indeed, where its heart lies.

Puzzling over Peer Review

Over the past two decades, there has emerged a terrific amount of scholarly literature about peer review---in response to perceived crises in the practice and in scholarly communications more generally. In reviewing a wide swath of this literature, I have been impressed mainly with what a mess it is.1 The multitude of perspectives and objectives, and researchers' appetite for either experimenting with peer review forms or with surveying peer review practices seems bottomless. Out of all this literature, however, it is hard to come to general conclusions, apart from the heightened interest in the topic and the general sense of anxiety that seems to pervade discussions of the trajectory of scholarly communications generally.

The primary figure that I have come to in my reading is of peer review as a black box. The black box of peer review allows us to treat it as a thing---and thus to use it, rely upon it, and take it for granted---without needing to dig too deeply inside to sort out how (or indeed if) it works. Indeed, my sense of the breadth of scholarly inquiry about peer review seems more concerned with its outward presentation: who is doing what with it, how do people feel about it, whether we think it is trustworthy, and so on. Rarely is there much real inquiry about what it means or exactly what it is for (beyond a gesture toward "quality")---or indeed what the consequences of it are for our scholarly communications environments. Arguably, this is unsurprising, as such questions are probably too large to pose of "peer review" writ large, their answers are likely to be rather different in different scholarly contexts, disciplines, venues, and so on. So while we see a great deal of study of how peer review is done in medicine, or in the natural sciences, or how an experiment in open review at journal X played out, the point and rationale for peer review remains taken for granted.

Beside questions of editorial and management practices at journals or presses, peer review is broadly understood as a hallmark of scholarly publishing---something without which it simply wouldn't be scholarly publishing anymore. In this sense peer review is a ritual process undertaken by all "serious" scholarly publications that serves as a kind of talisman against a various spectres: personal subjectivity, interpersonal echo chambers, amateurism, sloppiness, idiosyncrasy, poor quality, biases of various kinds, and of course "bad actors" of one sort or another. Partly and fully closed (formerly called "blind" and "double-blind") peer review,2 in service against specific forms of bias, prejudice, and dangerous power dynamics, also serve to make the process that much more opaque and difficult to interrogate in practice---and as a result, the familiar old "DBPR" (double-blind peer review) likely remains the tacit benchmark.

Does peer review---in its common forms---actually guard against these various spectres? Such a question is at least gestured at in the literature. But of course it is nearly impossible to devise a test that would determine it one way or the other, and so what we are left with are scattered anecdotes, war stories, and a general feeling that the practice is doing what it is supposed to, at least most of the time. Or else it is not. Jon Tennant and Tony Ross-Hellauer note:

...there is a real danger that advocates of reform in peer review do not always appreciate the often-limited scope in our general understanding of the ideology and practice of peer review. Ill-informed generalisations are abound, for example, the oft-heard ‘peer review is broken’ rhetoric, compared with those who herald it as a ‘golden standard’. (2020)

There is perhaps most broadly a sense that peer review has a role in ensuring "quality," and that peer review is at the heart of judgements of quality. Without descending into a philosophical discussion around "what is quality?" a recent study by Esteban Morales and colleagues at the Scholcomm Lab noted that

…faculty most commonly define high quality academic journals based on the review process, referring to the perceived rigor of the process of evaluating, gatekeeping and editing academic articles for the journal (e.g., “rigorous review and selection of articles to publish”)

For example, the highest percent of respondents in our survey perceived quality as being related to the review process of a journal. However, most journals do not make article reviews public, suggesting that the review process of a journal is not widely known or understood by people outside their own experiences. (2021)

The black box remains. There is, generally speaking, no mechanism for really knowing how review actually works at any given journal (or in any given situation), beyond the black box of "peer review" itself---especially given closed reviews. Moves to increase the transparency of the review process are at least a nod to this problem, but the goals of transparency seem at odds with the goals of closed review itself, and so this seems unlikely to go very far. Efforts go in the other direction altogether, to "open" peer review, which of course raises other questions, not to mention the eyebrows of the traditionalists who may or may not be in charge of your funding, tenure, hiring, and so on. A fundamental tension exists between peer review's role as a ritualized piece of scholarly process and the ideals of transparency and openness in scholarship and science. It can seem as though current forms of academic peer review might be as problematic as no peer review at all. My argument is not for abandoning it, nor throwing off the received model in favour of a radical new format. Indeed, the focus of this article will be on the relations and dynamics within traditional closed review forms.

What remains steady is the solidly held idea that peer review is a necessary feature of scholarly communications, especially where funding and institutional legitimacy is concerned. Peer review is the mark of distinction for scholarly publishers, as for granting agencies and such institutional functions where accountability is required. But the logic is circular.3 What can we make of that?

What is Peer Review for, exactly?

There is a lack of consensus about what peer review is, what it is for and what differentiates a ‘good’ review from a ‘bad’ review, or how to even begin to define review ‘quality.’ This sort of lack of clarity can lead to all sorts of confusion among discussions, policies and practices. (Tennant & Ross-Hillauer 2020)

Much of the discourse around peer review focuses on its role in preventing bad things---but another way of approaching it is to look at peer review's more positive and generative functions. What are we doing when we peer review? Are we gatekeeping? Or are we filtering? Are we culling? Are we curating? Are we nurturing? Are these even the right options? Tennant & Ross-Hellauer make the case that peer review often orients us to a "judicial" role rather than a critical examination; do reviewers act like judges, jurors, or independent assessors?

Typically, by focusing on a binary outcome for articles (i.e. reject or accept), editorial peer review has become more of a judicial role than a critical examination [45], as the focus becomes more about the decision rather than the process leading to that decision. (2020)

The way in which a review process is positioned in a publication seems likely to influence its character. If a journal prides itself---and indeed trades on---a high rejection rate, then peer review may look more like gatekeeping than in a situation where editorial or thematic development is valued, as in a special issue, for instance, or a book series. The range of forms, functions, and practices under the singular banner of "peer review" makes studying it difficult.

In the sciences, for instance, the famous aspiration to reproducibility in research results may present a much more specific set of requirements for peer review, although this model is hardly a universal ideal. But does peer review do the same kind of work in the social sciences? What about the humanities? It seems that, despite the common terminology, peer review must necessarily have a differing character in different contexts. It can't, practically speaking, be all these things at the same time. And yet, in much of the discourse, it seems this way, because our ideas about what peer review is for are myriad and all tangled up with one another. Or at least so my reading of the literature tells me.

What are the goals of Peer Review?

Given this murky beginning, I want to focus in on the role of peer review in the humanities, and especially in the context of open social scholarship. If we narrow the focus thus, perhaps we can come to a clearer idea of what we want peer review to do for us: to clarify the goals of peer review, or at least the values we want to drive our practices.

Nearly every journal I have dealt with in my career employed a commonplace triage system---accept/reject/revise-and-resubmit---negotiated between editors and reviewers. What do those actually mean, and what are the ramifications for them in the context of the publication, and in the context of the authorship of scholarly works? Rejection is perhaps the simplest of the three, but what does this imply? It may be that the work is inappropriate---a poor fit---for the particular venue to which it has been submitted. It may mean that the scholarship is not up to the standard expected by the reviewers and/or editors---especially if there is limited space available. It may mean that the work is bad in some identifiable way: not original, insufficiently researched, off-putting, or even outright offensive. In all cases, though, the implication of a 'reject' decision is that the work cannot be fixed up in order to be accepted; the relationship between work and publication will not be established.

Acceptance means... what? That the reviewers and/or editors approve of the work; that they are prepared to guarantee the quality of the scholarship; that they want the work in their journal; that it is a good fit; that it advances the scholarly discourse that the journal participates in.

Most complex is revise and resubmit. This implies that substantial flaws exist but... we still love you? Or perhaps that there are minor flaws but the reviewers take them seriously and must be satisfied that they are addressed? It may also mean that the reviewers and/or editors cannot deal with it now but maybe next month. Here the possibility of improvement, redemption, perfectibility, is held open. Note the role of context here, beyond the internal merits of an article.

In all three, there is a sense of editorial teleology, a vision of what the scholarly discourse in the journal should be, and whether a submitted work can or can't be imagined as part of it. In this image the emphasis is on the publication as a whole, be it an entire journal or a special issue, as opposed to simply the individual merits of an individual submission. The relationship between text and context drives the purposes to which peer review is put.

Accountability in a Closed Place

An important term comes up often in discussions of peer review: accountability. The primary sense of accountability in peer review is accountability to the scholarly community---that the published work is true and worthy of inclusion in the discourse. One of the most basic functions of peer review is to ensure this accountability, on behalf of the publication, to its audience and community. A reader's encounter with a published work relies on the sense of trust implied (and indeed evoked, in form, tone, style, and in paratextual cues) by this supposed relation. The work of the publisher (indeed, the editors and reviewers) is at heart designed to establish this very distinction---especially as compared with self-published works on a scholar's website or in a public repository.

There are of course other layers of accountability to which we should also pay attention, such as reviewers' and editors' accountability to the author, with respect at least to fairness and responsiveness, and especially with respect to bias, power differential, and freedom from reprisals. Closed reviews can be seen as both furthering and hindering these layers of accountability. The double-edged sword of closed review is invoked by both sides of seemingly endless debates about the relative merits of anonymity, closure, and privacy in the review process. Indeed, the practice of anonymizing authors seems to have been promoted in the 1970s as a way of guarding again gender bias (Pontille & Torny 2014). Yet anonymous reviews are also seen as a mechanism that hides bias and abuse, working against the goals of accountability. Of course counterarguments are ready to hand: many reviewers would report that author anonymity is a fiction anyway, since scholarly communities are often small enough that scholars and their projects are recognizable. Conversely, open reviews invite the criticism that optics and politics get in the way of fair and properly critical assessments. So we continue to work in a world in which the old "double-blind" standard is almost a given, not because it is best, but because alternatives are not sufficiently compelling.

It may be worth also considering the relationship between scale and anonymity; keeping authors and reviews anonymous effectively casts them as strangers---a convenient modernist figure of fairness and objectivity. In a large enough circle, even in the absence of anonymity per se, strangers would be the rule, and the rare exception would be someone you actually know. In such a world, closed review would help guard against unfair bias against categories of people, and especially in the case of exceptions to the rule of strangers. But if author anonymity is often in fact a fiction and reviewers can well guess who a paper's author is, then the scale isn't sufficient to make stranger-relations the norm. Are scholarly communities large enough to aspire to impersonal objectivity? As with everything else, it depends.

One strong argument for anonymity (of both author and reviewers) is that a rejection recommendation seems impossible without it; a rejection in an open review context is a visible, embarrassing, uncomfortable situation for all involved---and difficult to separate from personal relations, politics, or perceived unfairness. In practice, if both authors and reviewers are strangers to one another---or at least can act as if they were---the social and collegial relations that bind scholarly communities together are perhaps better able to withstand the repercussions of rejections in the review process. This assumes, of course, that the possibility of rejection is in fact desirable.

The relations between scholars, in their relative roles as authors and reviewers, are of course foundational to all of scholarly communications; we are not in fact perfect strangers, and the process of reviewing and publishing material is not (yet, thankfully) fully algorithmic. Kathleen Fitzpatrick (2021) makes careful note of the editors acting as middlemen, managing the anonymity and secrecy, and indeed interpreting the findings back to the authors. Journal editors, indeed, necessarily play a role in rising above the anonymous interplay of authors and reviewers to ensure that the whole mechanism actually functions to get articles published. There is, of course, a considerable amount of editorial control over peer review: who reviews what, how much back-and-forth is allowed, how reviewer recommendations (usually two or three reviewers) will be interpreted, balanced, and indeed presented to authors. Authors' interactions are of course with the editors, not the reviewers directly.4 And so, lest we tend to believe that closed review is in fact a closed process, the mere existence of editors should remind us that this is at heart a deeply relational process---a topic which we will turn to later in this article.

Who are Peers, anyway?

The other major aspect of peer review---which is again often left unquestioned---is that review is supposedly performed by "peers," whoever that might mean. A knee-jerk response to the question might be as simple as "fellow scholars," but it is rarely that simple. So who are peers?

In one version of the story of peer review, "peers" are identifiable with the "modest witness" of experimental science, which Donna Haraway (1997) famously drew from Shapin & Shaffer's account of 17th-century science in Leviathan and the Air Pump---a figure recognizable (ironically enough) in the stranger rendered by anonymous review, and where "objectivity" is supposedly achieved collectively. In this version, a peer is the disinterested everyman to whom experimental science (and by extension contemporary ideals of open and public science) ought to be intelligible. This is a low bar for peers, and also unrealistic---the people for whom your scholarship makes most sense are more likely the people that actually care about it. The idea that peers, most broadly framed, are people just like us, is an unsatisfying figure---one that papers over all sorts of power differentials like race, class, gender, age, and rank that actually shape the dynamic of our professional and scholarly lives. Indeed, in contrast to this blithely egalitarian and apolitical ideal, Fitzpatrick (2010) reminds us that in the early cases of royal societies' reviews, the peers who judged whether things were fit for print were peers in the sense of "peers of the realm." Not everyman at all.

Yet such appeals to history might lead us to think of peer review as an ancient scholarly practice. While there are some early modern precursors, the more interesting story is that peer review practices are surprisingly recent. As Aileen Fyfe et al. point out in a UK Arts & Humanities Research Council briefing,

The term ‘peer review’ came to public prominence in debates over grant funding in the USA in the 1970s (Baldwin, 2017), and has since been extended to cover a variety of processes by which academics formally evaluate each others' work. [...] For [learned societies], referring papers to suitably-qualified members of the society or university for close scrutiny before publication was part of an editorial system that was intended to emphasise collective rather than individual responsibility, as well as to decide on the appropriate use of institutional resources. (2017, 12)

The figure of peer review that emerges in this account---in the context of grant-funding, in the evolution of learned societies as centres of institutional and cultural capital, and in the concurrent rise of scholarly publishing's prestige economy---is entirely about scholarly and capitalist power structures. Jon Tennant et al. note how peer review serves to maintain the scarcity required to maintain a prestige economy in publications, even while the scarcity of print economics has given way to digital publications and digital economics:

The economic motivations for continuing to impose selectivity in a digital environment, and applying peer review as a mechanism for this, have received limited attention or questioning, and are often simply regarded as how things are done. Use of selectivity is now often attributed to quality control, but may be more about building the brand and the demand from specific publishers or venues. (2017)

Peers, here, are the figurative means by which scholarly legitimacy is conferred by institutional and corporate structures. It is worth noting that this is a markedly different figure than the collegial service or "academic citizenship" (Dean & Forray 2018) ideal that emerges in surveys of individual academics' attitudes toward peer review. Fyfe et al. note:

The co-option of peer review by profit-oriented publishers now sits in tension with the perception of individual academics, who (despite complaints about the rising burden) remain largely committed to the traditional vision of refereeing and editorial work as a voluntary service to the wider academic, or disciplinary, community. (2017, 12)

In a sense then, though we might imagine peer reviewers (especially when we are imagining ourselves) as peers in the sense of fellow scholars out there in the world, structurally at least, peers play the role of hierarchical gatekeepers; a doubled figure that mirrors the double image of the academy as a collegial research/educational community and at the same time an instrument of late-capitalist agendas.

Thus, we live in a world in which peer review---especially in journal publishing---has become standardized and its rationale flattened, where boundary-policing practices have evolved into a mechanism to filter the volume of work created by many researchers while upholding a prestige-based publication economy manifested in high rejection rates. As scholars, the academic credentialing process has often come to mean subjecting oneself to such hierarchical ideals, especially critically impacting early-career scholars. The old adage, "publish or perish" may oversimplify, but most scholars will recognize the pressure of a competitive publication agenda, whether they actively participate in it or merely feel the anxiety coming off it. It is worth asking if this operates as an incentive structure... or as a threat.

The pressure to publish in prestigious venues is widespread but also somewhat ethereal: the pressure comes from within---an example of what Nicky Agate et al. call an "Abilene paradox"---from our perceptions of our colleagues' and our institutions' values, where the criteria for 'excellence' are vaguely or circularly defined, and where there is always an impulse to compare ourselves with how things appear elsewhere. Moxham & Fyfe describe it thus:

The adoption of peer review by a wide variety of humanistic and social science disciplines reveals both the long-standing (if contested) envy of the epistemic rigour apparently associated with the natural sciences, and the professionalizing desire to adopt what has come to be seen as ‘proper’ academic practice. (2018)

This is just one version of the pressure of the neoliberal university. Such pressures lead to a culture of "competitive thinking," in the words of Kathleen Fitzpatrick:

...thinking that is compelled by what sociologist and economist Thorsten Veblen called "invidious comparison" or what Fluck refers to as the "race for professional distinction." It is the competitive that has undermined the capacity for community-building, both within our campuses and between our campuses and the broader public. (2019, 33)

Is this, then what peer review has become? The image that emerges is of a neoliberal field where scholars are all independent actors, seeking recognition; where publications compete in a prestige hierarchy based on managed scarcity; where reviewers are all too often cast in the role of judges, in a gatekeeping capacity that serves scarcity in the name of accountability; and where closed, anonymous practices keep peer review in a black box that is difficult to open.

Furthermore, there is an assumption of scale and relations between strangers, of unsolicited and context-less submissions, and a frictionless transactional process: indeed, a market paradigm where academic darwinism prevails, and we talk about "incentives." The modest witness is accompanied by the nameless, interchangeable researcher, an automated editor, and an aggregate expectation of fairness over a large enough interval. This image does not encourage collegiality, the ideal of a community of inquiry, nor even collaboration, except in rigidly standardized forms.

Does anyone Care about Peer Review?

This somewhat depressing picture of contemporary peer review led me back to the literature on care ethics. Care ethics offers a perspective that is solidly outside the neoliberal, individualistic model. Virginia Held writes:

Understanding care leads us to a relational view of the person, and away from the view of the person as liberal individual: free, equal, autonomous, self-sufficient, adult. The outlook of care understands us as dependent and interdependent. It emphasizes and values caring relations, not just the dispositions or actions of individuals. (2015, 23–24)

Indeed, it is precisely the human and relational element in peer review---this very human and relational process---that we need to (re-)consider. Bethany Nowviskie, writing of the digital humanities more generally, provides an inspirational prompt that works as well in our present discussion:

Can an ethic of care help us build the kind of capacities we need in the humanities right now? Can it help us integrate the macro- and micro-attention that humanities work requires in the digital age? (2015)

Care, in Fischer and Tronto's classic definition, is "everything we do to maintain, contain, and repair our world, so that we can live in it as well as possible." Joan Tronto's classic book about care ethics, Moral Boundaries: A Political Argument for an Ethic of Care (1993) frames this kind of relation in explicitly political terms, and especially in terms of unequal power relations. This is a worthy starting point, but do we not also have an ethic of care toward our equals? And to our heroes, elders, and leaders even? And importantly for our question here, do we not have an ethic of care to the work itself, and not merely to the subjects or objects thereof. Indeed, what of care for the nonhumans in our world?

In applying care ethics to peer review, let us put the focus on relations and relationality. Rebecca Kennison helpfully noted that the aim of peer review should be engagement rather than judgement (2016); engagement puts the emphasis back on relationality as opposed to a transactional view of scholarly communications. What then is relational in peer review? We can pretty easily identify a number of key relational facets that will be familiar to most of us:

  • the author's connection to the work;
  • the author's connection to the publication, and by extension, to the field;
  • the reviewer's (and probably the editor's) interest in the work and its development;
  • the reviewer's (and editor's) interest in the author and their development;
  • the editor's (and reviewer's) consideration of readers: researchers downstream, students, interested colleagues, etc.;
  • the reader's consideration of the article and by extension of the publication and its role; and
  • the reader's conception of the author, and of their own (potential) authorship and participation.

The list could be elaborated further without too much trouble, to include consideration of power differences among participants. My colleague Alyssa Arbuckle, for instance, has pointed out that in seeking a process-oriented approach to journal editing one can pay attention to the career stage and development of scholars---especially junior scholars---and that through careful management of communications and selection of reviewers (even within a fully closed model) mentoring, equity, and personal development can become part of the process.5 Such a model, as enacted in INKE partnership's proceedings in previous years, puts the emphasis on the relations, and not just on the outcome. If relationality in peer review is so easily evident in a few examples, then how might we work to properly foreground it in how we design and structure editing and reviewing for our publications?

Toward a model of Care-ful Peer Review

But all of this, as I hope you hear, is not about our publications, or about our publishing systems, but about us — about how we relate to one another, about how we engage with one another as we discuss our work. And thus all of it is within our power to improve — especially if we act as a community of practice, with an emphasis on community. (Fitzpatrick 2019)

Kathleen Fitzpatrick's writings are ever inspirational, but her invocation of "community of practice" here is worth dwelling on for a moment, as it brings to mind the insights made by Jean Lave and Etienne Wenger in their 1991 book, Situated Learning: Legitimate Peripheral Participation, which is about the dynamics of communities of practice. The book's subtitle is an awkward mouthful, but "legitimate peripheral participation" is a critical idea in Lave & Wenger's theory of communities of practice, and it is helpful in our discussion of care in scholarly publishing, editing, and reviewing. Lave & Wenger's insight is that communities of practice are not populated by static entities. Rather, practitioners in communities are on their way to something, they are becoming something. In their model, all participation is in some way peripheral, and no matter how peripheral, it is still legitimate. They write:

Legitimate peripheral participation refers both to the development of knowledgeably skilled identities in practice and to the reproduction and transformation of communities of practice. (1991, 55)

If we think of scholarship generally as a community of practice, there are many kinds and layers of participation, depending on things like career stage, academic role, proximity to particular scholarly discourses, and so on. This is perhaps most obvious in the roughly apprenticeship-like model of graduate school, where junior scholars work with senior scholars and in doing so develop and grow individually as well as helping to reproduce and transform the community itself. It is not hard to think of these moves in practice:

  • undergraduate students on their way to being grad students;
  • grad students on their way to being researchers and research leaders;
  • grad students on their way to being faculty members;
  • Jr faculty on their way to being authorities;
  • mid-level faculty on their way to becoming mentors; and
  • Sr faculty on their way to making room for others.

The image pertains just as well to scholarly communication: participants are variously readers, authors, reviewers, and editors---perhaps in different capacities at different career stages---but these formal modes of participation hopefully overlap and even blur together generatively in individuals along their career journeys. A care-based approach to review (and editorial, and publishing itself) would attend to the relationships, not just between scholars as (neoliberal, self-interested) individuals, but between participants on their way to becoming, and to the community that hosts those relationships and their discourse. This has important potential beyond just the world of professional scholars, since disciplinary outsiders and members of the world beyond the academy are also legitimately peripheral participants on their way to becoming something.

It is easy enough to think of disciplinary boundary policing as a core function of peer review. But how can peer review nurture, and curate---and blur and perforate---boundaries, if these are the dynamics that we acknowledge, if not actually care about in open social scholarship.

Instead of agents in field, how about a garden?

Scholars sitting in pairs in a garden talking
INKE Proceedings Review, circa 2018

Here is an idealized yet actual image of peer review practice, from back in the before-times: in a lovely garden, we would gather, working in pairs in conversation, and with a suitable nod at least to the ideals of anonymous review6 we would consider each article and where we thought it was going, and write voluminous commentary and critique to the authors. This was good; never before or again have I received better and more generous review comments; nor have I ever written more engaged review comments. Consider the contrast to what is undoubtedly the norm now, in which an automated system mediates between an editor and a pool of possible reviewers, emailing assigned reviewers about deadlines (perhaps multiple times!), capturing reviews as so many fields in a fill-able form, and automatically routing these comments back to editors and authors. This is a very different figure of scholarly practice.

In the university environment, we exist always in a flood of messages and communications. Some of those messages are interesting, some are not, but it always exceeds our ability to truly pay attention to everything. This normalized condition of being suspended in so many overlapping circles of discourse, and the necessary incompleteness of our participation as a result, calls for some mediating structures. Peer review is such a mediator, or at least it should be. Beyond credentialing and disciplinary policing, a more active figure of peer review has it playing a role both in the nurturing of our works (not to mention nurturing the authors) and---critically---in helping find and address an audience which may interpret them deeply and meaningfully.

To think of peer review this way moves away from the reviewer as gatekeeper and closer to the figure of reviewer as privileged reader and active collaborator in the trajectory of a scholarly work. A peer reviewer has a unique relationship to an emerging work of scholarship, and has unique agency in what will happen to a work: whether it is any good, whether it makes the right connections, and whether it should be seen, and by whom. And while the "reviewer as privileged reader" continues to centre the agency of individuals in the process, the situation of the reviewer in a network of discourse---as opposed to hidden in the black box of a transactional model of peer review---leads us back to Lave and Wenger's image of "legitimate peripheral participation" where members with more centrality and agency play a part in bringing newer and more marginal members into better connection, richer engagement.

Audiences and communities of practice

Traditionally---that is, in the tradition of print culture---the audience for a scholarly work is seen in a strictly disciplinary perspective, as the work is "published" specifically into that disciplinary milieu. Such a view of publication lends itself easily to the black box, an abstracted functionalist model of publishing that can be filled by various corporate third parties and their algorithms and data flows (such entities may have their own goals and agendas quite separate from those of the scholarly community). Publication in this sense establishes and preserves the scholarly record, and settles matters of "paternity" (Guédon 2001) within the discipline, but little more.

But the example of groups like the Radical Open Access Collective and the ScholarLed consortium7 demonstrates a refreshing refusal to abstract away the process of publishing, rather seizing the means and the purpose themselves and in doing so keeping publications local, relevant, and small-scale. To do so is to buck the trend of internet capitalism to greater and greater scale, and to acknowledge the infrastructural and processual facets of scholarly work while admitting our positions of compromise to those facets. The payoff, though, is greater agency in defining the reach and purpose of scholarly work. It is an acknowledgment of entanglement, in Donna Haraway's sense, and a commitment to getting on with living as well as we can in the face of it. This kind of entanglement seems to me entirely compatible with the senses of "care" that we have considered here, in which relationality and trajectories are taken as fundamental. The point is to consider our entanglements and, as Haraway famously put it, "to make a difference in the world, to cast our lot for some ways of life and not others. To do that, one must be in the action, be finite and dirty, not transcendent and clean." (1997, 36) Such a framing runs counter to the ideals of objectivity and neutrality that animate traditional views of peer review; this no doubt raises a set of fairly obvious critiques about bias and privilege. Yet if there is to be anything to Rebecca Kennison's call for "engagement" over "judgement" this is a risk that has to be worth facing.

If we can imagine open scholarship as having broader audiences than the strictly disciplinary ones, then we can imagine peer review as a developmental mediation, between author and reader, between work and discourse, between scholarship and its audience(s). Because a peer reviewer has a deeper, context- and content-level engagement with the material, the author, and, ideally, the reader downstream---who, we must remember, is always/already an author themselves, on their way in or out or across our community. These are the relationships that we have an opportunity to nurture and foreground.

Bibliography

Adema, Janneke, and Samuel A. Moore. 2017. “The Radical Open Access Collective: Building Alliances for a Progressive, Scholar-Led Commons.” Impact of Social Sciences (blog). October 27, 2017. https://blogs.lse.ac.uk/impactofsocialsciences/2017/10/27/the-radical-open-access-collective-building-alliances-for-a-progressive-scholar-led-commons/.

Agate, Nicky, Rebecca Kennison, Stacy Konkiel, Christopher P. Long, Jason Rhody, Simone Sacchi, and Penelope Weber. 2020. “The Transformative Power of Values-Enacted Scholarship.” Humanities and Social Sciences Communications 7 (1): 1–12. https://doi.org/10.1057/s41599-020-00647-z.

Butchard, Dorothy, Simon Rowberry, Claire Squires, and Gillian Tasker. 2017. “Peer Review in Practice.” Academic Book of the Future. London: UCLPress https://doi.org/10.14324/111.9781911307679.15.

Cox, Ailsa. 2021. “The Silence of Peer Review.” In Strategies of Silence. Routledge. https://doi.org/10.4324/9781003030911.

Dean, Kathy Lund, and Jeanie M. Forray. 2018. “The Long Goodbye: Can Academic Citizenship Sustain Academic Scholarship?” Journal of Management Inquiry 27 (2): 164–68. https://doi.org/10.1177/1056492617726480.

Deville, Joe, Samuel Moore, and Tahani Nadim. 2018. “The Commons and Care.” Mattering Press. https://hcommons.org/deposits/item/hc:19817/.

Fitzpatrick, Kathleen. 2010. “Peer‐to‐peer Review and the Future of Scholarly Authority.” Social Epistemology 24 (3): 161–79. https://doi.org/10.1080/02691728.2010.498929.

———. 2019. Generous Thinking: A Radical Approach to Saving the University. Baltimore: Johns Hopkins University Press.

———. 2021. “Opening Up Peer Review.” Kathleen Fitzpatrick. blog post. June 11, 2021. https://kfitz.info/opening-up-peer-review/.

Fyfe, Aileen, Kelly Coate, Stephen Curry, Stuart Lawson, Noah Moxham, and Camilla Mørk Røstvik. 2017. “Untangling Academic Publishing: A History of the Relationship between Commercial Interests, Academic Prestige and the Circulation of Research.” Zenodo. https://doi.org/10.5281/zenodo.546100.

Guédon, Jean-Claude. 2001. In Oldenburgś Long Shadow: Librarians, Research Scientists, Publishers, and the Control of Scientific Publishing. Washington, D.C: Association of Research Libraries. http://www.arl.org/arl/proceedings/138/guedon.html.

Guédon, Jean-Claude, and Ray Siemens. 2002. "Peer Review and Imprint" in The Credibility of Electronic Publishing: A Report to the Humanities and Social Sciences Federation of Canada. https://web.archive.org/web/20040803194600/http://web.mala.bc.ca/hssfc/Final/PeerReview.htm.

Haraway, Donna J. 1997. Modest_Witness@Second_Millennium.FemaleMan©_Meets_OncoMouse™. New York: Routledge.

Held, Virginia. 2015. “Care and Justice, Still.” In Care Ethics and Political Theory, edited by Daniel Engster and Maurice Hamington, 19–36. Oxford, United Kingdom: Oxford University Press.

Horbach, S.P.J.M., and W. Halffman. 2018. “The Changing Forms and Expectations of Peer Review.” Research Integrity and Peer Review 3 (1): 8. https://doi.org/10.1186/s41073-018-0051-5.

Jackson, Liz, Michael A. Peters, Leon Benade, Nesta Devine, Sonja Arndt, Daniella Forster, Andrew Gibbons, et al. 2018. “Is Peer Review in Academic Publishing Still Working?” Open Review of Educational Research 5 (1): 95–112. https://doi.org/10.1080/23265507.2018.1479139.

Joy, Eileen A. 2020. “Not Self-Indulgence, but Self-Preservation: Open Access and the Ethics of Care.” In Reassembling Scholarly Communications: Histories, Infrastructures, and Global Politics of Open Access, edited by Martin Paul Eve and Jonathan Gray, 317–29. Cambridge, MA: The MIT Press. https://doi.org/10.7551/mitpress/11885.001.0001.

Kennison, Rebecca. 2016. “Back to the Future: (Re)Turning from Peer Review to Peer Engagement.” Learned Publishing 29 (1): 69–71. https://doi.org/10.1002/leap.1001.

Kingsley, Danny A. 2016. “The Peer Review Paradox: An Australian Case Study.” Cambridge: Office of Scholarly Communication, Cambridge University Library. https://www.repository.cam.ac.uk/handle/1810/256773.

Kumar, Malhar N. 2010. “The ‘Peer Reviewer as Collaborator’ Model for Publishing.” Learned Publishing 23 (1): 17–22. https://doi.org/10.1087/20100105.

Lave, Jean, and Etienne Wenger. 1991. Situated Learning: Legitimate Peripheral Participation. Cambridge: Cambridge University Press.

Maryl, Maciej, Marta Błaszczyńska, Agnieszka Szulińska, and Paweł Rams. 2020. “The Case for an Inclusive Scholarly Communication Infrastructure for Social Sciences and Humanities.” F1000Research 9 (October 22, 2020): 1265. https://doi.org/10.12688/f1000research.26545.1.

Morales, Esteban, Erin C. McKiernan, Meredith T. Niles, Lesley Schimanski, and Juan Pablo Alperin. 2021. “How Faculty Define Quality, Prestige, and Impact of Academic Journals.” PLOS ONE 16 (10): e0257340. https://doi.org/10.1371/journal.pone.0257340.

Moxham, Noah, and Aileen Fyfe. 2018. “The Royal Society and the Pre-History of Peer Review, 1665–1965.” The Historical Journal 61 (4): 863–89. https://doi.org/10.1017/S0018246X17000334.

Niles, Meredith T., Lesley A. Schimanski, Erin C. McKiernan, and Juan Pablo Alperin. 2020. “Why We Publish Where We Do: Faculty Publishing Values and Their Relationship to Review, Promotion and Tenure Expectations.” PLOS ONE 15 (3): e0228914. https://doi.org/10.1371/journal.pone.0228914.

Nowviskie, Bethany. 2015. “On Capacity and Care.” Bethany Nowviskie. October 4, 2015. https://nowviskie.org/2015/on-capacity-and-care/.

Pontille, David, and Didier Torny. 2014. “The Blind Shall See! The Question of Anonymity in Journal Peer Review.” Ada New Media. https://adanewmedia.org/2014/04/issue4-pontilletorny/.

Royal Society. 2015. “The Future of Scholarly Scientific Communication: Conference.” London: The Royal Society. https://royalsociety.org/-/media/events/2015/04/FSSC1/FSSC-Report.pdf?la=en-GB.

Tennant, Jonathan P., Jonathan M. Dugan, Daniel Graziotin, Damien C. Jacques, François Waldner, Daniel Mietchen, Yehia Elkhatib, et al. 2017. “A Multi-Disciplinary Perspective on Emergent and Future Innovations in Peer Review.” F1000Research 6 (November): 1151. https://doi.org/10.12688/f1000research.12037.3.

Tennant, Jonathan P., and Tony Ross-Hellauer. 2020. “The Limitations to Our Understanding of Peer Review.” Research Integrity and Peer Review 5 (1): 6. https://doi.org/10.1186/s41073-020-00092-1.

“Transparency in Standards and Practices of Peer Review.” 2018. Peer Review Transparency. https://www.prtstandards.org/pub/zi2i5dt4/release/1.

Tronto, Joan C. 1993. Moral Boundaries: A Political Argument for an Ethic of Care. New York: Routledge. https://doi.org/10.4324/9781003070672.


  1. Indeed, the outpouring of scholarship on peer review is vast and continuing. In this article I have drawn on a number of high-level survey articles; see Butchard et al. 2017; Tennant et al. 2017; Jackson et al. 2018; Horback and Halffman 2018; Maryl et al. 2020, as well as a major 2015 conference on Scholarly Communication hosted by the Royal Society. 

  2. This nomenclature is drawn from https://www.prtstandards.org ---with "closed" replacing the older variations on the word "blind." 

  3. My point is not that circular logic is necessarily a bad thing---indeed, many institutional forms are shaped, generatively, by circular logic. Rather, the trouble is that circular logic makes things hard to interrogate, and hard to change. 

  4. In the case of automated review workflow systems, it is of course possible that reviewer comments pass back to authors without editors' oversight. This is hardly an ideal situation. 

  5. Alyssa Arbuckle, personal communication, July 4, 2022. 

  6. As the mode in this case was indeed "fully closed" review, we did not consider the authors' situation or trajectory; this would be a further step toward a more care-based approach. 

  7. On the Radical Open Access Collective, see https://radicaloa.disruptivemedia.org.uk/; also Adema & Moore 2017. See also ScholarLed Consortium: https://scholarled.org/; also Eileen Joy, 2020. “Not Self-Indulgence, but Self-Preservation"; Deville, Moore, and Nadim, 2018. "The Commons and Care."