What

The chosen article that we will review and discuss is…

• The conundrum of sharing research data (doi:10.1002/asi.22634)

Researchers are producing an unprecedented deluge of data by using new methods and instrumentation. Others may wish to mine these data for new discoveries and innovations. However, research data are not readily available as sharing is common in only a few fields such as astronomy and genomics. Data sharing practices in other fields vary widely. Moreover, research data take many forms, are handled in many ways, using many approaches, and often are difficult to interpret once removed from their initial context. Data sharing is thus a conundrum. Four rationales for sharing data are examined, drawing examples from the sciences, social sciences, and humanities: (1) to reproduce or to verify research, (2) to make results of publicly funded research available to the public, (3) to enable others to ask new questions of extant data, and (4) to advance the state of research and innovation. These rationales differ by the arguments for sharing, by beneficiaries, and by the motivations and incentives of the many stakeholders involved. The challenges are to understand which data might be shared, by whom, with whom, under what conditions, why, and to what effects. Answers will inform data policy and practice.

Dr Ester Ehiyazaryan-White, Senior Lecturer in Childhood Studies will take us through this paper highlighting some salient points and opening discussion.

Where

This event will be hybrid, if you wish to attend in person we will be meeting in Room 12.1.01 (Level 1) of the Charles Street Building. Add the event to your Google Calendar using this link If you aren’t able to attend in person but would like to join the Journal Club you can do so using Google Meet.

When

2025-10-21 @ 13:00-14:00

Future Events

To stay abreast of up-coming events you can either sign up to the mailing list or subscribe to the RSS feed. ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

What

The chosen article that we will review and discuss is…

• The replication crisis has led to positive structural, procedural, and community changes | Communications Psychology

The emergence of large-scale replication projects yielding successful rates substantially lower than expected caused the behavioural, cognitive, and social sciences to experience a so-called ‘replication crisis’. In this Perspective, we reframe this ‘crisis’ through the lens of a credibility revolution, focusing on positive structural, procedural and community-driven changes. Second, we outline a path to expand ongoing advances and improvements. The credibility revolution has been an impetus to several substantive changes which will have a positive, long-term impact on our research environment.

Dr Romain Thomas (Head of Research Software Engineering, University of Sheffield) will take us through this paper highlighting some salient points and opening discussion.

Where

This event will be hybrid, if you wish to attend in person we will be meeting in Room G07 of the Hicks Building, University of Sheffield.

Add the event to your Google Calendar using this link.

If you aren’t able to attend in person but would like to join the Journal Club you can do so using Google Meet.

When

2025-03-03 @ 12:30-13:20

Future Events

To stay abreast of up-coming events you can either sign up to the mailing list or subscribe to the RSS feed.

What

The chosen article that we will review and discuss is…

• OSF Preprints | Working Paper No 6: Exploring Research Transparency, Positionality and Reproducibility

This paper provides a summary of a project which exploredthe meanings attached to the terms ‘transparency’, ‘positionality’ and ‘reproducibility’ by researchersfrom different fields, through dialogue betweenpractice researchersin the artsand psychology.

The project consisted of three phases:

1. A pre-workshop discussion paper was developed to provide initial reflections on each of the terms.
2. An in-person workshop brought together researchers from psychology and the arts to uncovera plurality of perspectives on the terms through dialogue.
3. Post-workshop reflections from the organisers highlightedsignificant points of commonality and dimensions of difference arising from the workshop.

The reflections summarised in this report represent different views from within and across the fields of arts practice research and psychology.There was no attempt to reach a consensus in relation to these terms, but participants and organisers felt that there was inherentvalue in this cross-disciplinarydialogue. The discussions helped to surface assumptions about research that may be obscured by belonging to a specific research community.

Dr Eddy Verbaan (Head of Library Research Support, Sheffield Hallam University) will take us through this paper highlighting some salient points and opening discussion.

Where

This event will be hybrid, if you wish to attend in person we will be meeting in Room 12.1.01 Level 1 of Sheffield Institute of Education, Charles Street.

Add the event to your Google Calendar using this link.

If you aren’t able to attend in person but would like to join the Journal Club you can do so using Google Meet.

When

2025-01-29 @ 11:00

Future Events

To stay abreast of up-coming events you can either sign up to the mailing list or subscribe to the RSS feed.

What

The chosen article that we will review and discuss is…

• “Best Paper” awards lack transparency, inclusivity, and support for Open Science | PLOS Biology

Awards can propel academic careers. They also reflect the culture and values of the scientific community. But do awards incentivize greater transparency, inclusivity, and openness in science? Our cross-disciplinary survey of 222 awards for the “best” journal articles across all 27 SCImago subject areas revealed that journals and learned societies administering such awards generally publish little detail on their procedures and criteria. Award descriptions were brief, rarely including contact details or information on the nominations pool. Nominations of underrepresented groups were not explicitly encouraged, and concepts that align with Open Science were almost absent from the assessment criteria. At the same time, 10% of awards, especially the recently established ones, tended to use article-level impact metrics. USA-affiliated researchers dominated the winner’s pool (48%), while researchers from the Global South were uncommon (11%). Sixty-one percent of individual winners were men. Overall, Best Paper awards miss the global calls for greater transparency and equitable access to academic recognition. We provide concrete and implementable recommendations for scientific awards to improve the scientific recognition system and incentives for better scientific practice.

Zuzzana Zagrodzka (PhD Student) will take us through this paper highlighting some salient points and opening discussion.

Where

This event will be hybrid, if you wish to attend in person we will be meeting in G22 Blue Room of the Regents Court.

Add the event to your Google Calendar using this link.

If you aren’t able to attend in person but would like to join the Journal Club you can do so using Google Meet.

When

2024-11-28 @ 12:30

Future Events

To stay abreast of up-coming events you can either sign up to the mailing list or subscribe to the RSS feed.

What

The chosen article that we will review and discuss is from the Annual Review of Applied Linguistics.

• Promises and perils of positionality statements (2024). DOI:10.1017/S0267190524000035

Positionality statements have increasingly become the norm in many strands of social science research, including applied linguistics. With reference to current research, theory, and the author’s own work, this paper reviews some of the promises and perils of such statements, including their performativity and lack of reflexivity. The author concludes by arguing that positionality statements need to offer both more and less, to be better targeted, and be more effectively and widely utilized within the field of applied linguistics.

Dr Jenni Adams (Open Research Manager) will take us through this paper highlighting some salient points and opening discussion.

Where

This event will be hybrid, if you wish to attend in person we will be meeting in Meeting Room 315 of The Diamond.

Add the event to your Google Calendar using this link.

If you aren’t able to attend in person but would like to join the Journal Club you can do so using Google Meet.

When

2024-10-24 @ 12:00

Future Events

To stay abreast of up-coming events you can either sign up to the mailing list or subscribe to the RSS feed.

What

The chosen article that we will review and discuss is from ICWSM Workshop (2015) Technical Report WS-15-18 (Standards and Practices in Large-Scale Social Media Research) /

• Making social media research reproducible (2018). doi:10.1609/icwsm.v9i4.14685

The huge numbers of people using social media makes online social networks an attractive source of data for researchers. But in order for the resultant huge numbers of research publications that involve social media to be credible and trusted, their methodologies, considerations of data handling and sensitivity, analysis, and so forth must be appropriately documented. We believe that one way to improve standards and practices in social media research is to encourage such research to be made reproducible, that is, to have sufficient documentation and sharing of research to allow others to either replicate or build on research results. Enabling this fundamental part of the scientific method will benefit the entire social media ecosystem, from the researchers who use data, to the people that benefit from the outcomes of research.

Ric Campbell (Research Data Steward) will take us through this paper highlighting some salient points and opening discussion.

Where

This event will be hybrid, if you wish to attend in person we will be meeting in Meeting Room 410 of the Arts Tower.

Add the event to your Google Calendar using this link.

If you aren’t able to attend in person but would like to join the Journal Club you can do so using Google Meet.

When

2024-07-12 @ 12:30

Future Events

To stay abreast of up-coming events you can either sign up to the mailing list or subscribe to the RSS feed.

What

The chosen article that we will review and discuss is from F1000 Research

• Ten considerations for open peer review (2018). doi:10.12688/f1000research.15334.1

Open peer review (OPR), as with other elements of open science and open research, is on the rise. It aims to bring greater transparency and participation to formal and informal peer review processes. But what is meant by `open peer review’, and what advantages and disadvantages does it have over standard forms of review? How do authors or reviewers approach OPR? And what pitfalls and opportunities should you look out for? Here, we propose ten considerations for OPR, drawing on discussions with authors, reviewers, editors, publishers and librarians, and provide a pragmatic, hands-on introduction to these issues. We cover basic principles and summarise best practices, indicating how to use OPR to achieve best value and mutual benefits for all stakeholders and the wider research community.

Jim Uttley (Lectuer in Architectural Science) will take us through this paper highlighting some salient points and opening discussion.

Where

This event will be hybrid, if you wish to attend in person we will be meeting in Meeting Room G07 of the Hicks Building.

Add the event to your Google Calendar using this link.

If you aren’t able to attend in person but would like to join the Journal Club you can do so using Google Meet.

When

2024-05-16 @ 12:00

Future Events

To stay abreast of up-coming events you can either sign up to the mailing list or subscribe to the RSS feed.

What

The chosen articles that we will review and discuss is from Ecology and Evolution

• Accelerating the open research agenda to solve global challenges (2024). doi :10.1002/ece3.10887

Harnessing science-based policy is key to addressing global challenges like the biodiversity and climate crises. Open research principles underpin effective science-based policy, but the uptake of these principles is likely constrained by the politicisation, commoditisation and conflicting motives of stakeholders in the research landscape. Here, using the mission and vision statements from 129 stakeholders from across the research landscape, we explore alignment in open research principles between stakeholders. We find poor alignment between stakeholders, largely focussed around journals, societies and funders, all of which have low open research language-use. We argue that this poor alignment stifles knowledge flow within the research landscape, ultimately limiting the mobilisation of impactful science-based policy. We offer recommendations on how the research landscape could embrace open research principles to accelerate societies’ ability to solve global challenges.

Zuzanna Zagrodzka (PhD Student, Ecology and Evolutionary Biology) will take us through her recently published paper looking at the alignment of open research policy in stakeholders, its impact and recommendations on improving the situation.

Where

This event will be hybrid, if you wish to attend in person we will be meeting in Meeting Room 105 of the The Arts Tower.

Add the event to your Google Calendar using this link.

If you aren’t able to attend in person but would like to join the Journal Club you can do so using Google Meet.

When

2024-04-18 @ 14:00

Future Events

To stay abreast of up-coming events you can either sign up to the mailing list or subscribe to the RSS feed.

What

The chosen articles that we will review and discuss are from Educational Research and Evaluation

• Replication is relevant to qualitative research (2022). doi : 10.1080/13803611.2021.2022310

Replication has received increasing attention over the last decade. This comes on the heels of prominent instances of data fabrication (National Academies of Sciences, Engineering, and Medicine, 2017) and estimates that few studies attempt to replicate previous findings (Makel & Plucker, 2014). Replication has been called the Supreme Court of science (Collins, 1985), as well as a basic building block of scholarship. One persistent question in informal conversations that we have not seen addressed in formal writing, is replication’s relevance to qualitative research. Qualitative research is “a situated activity that locates the observer in the world” and “consists of a set of interpretive, material practices that make the world visible” (Denzin & Lincoln, 2011, p. 3). Some have argued that replication “missed the point” of qualitative research (Pratt et al., 2020, p. 3). However, in a survey of nearly 1,500 recently published education researchers, less than 10% of qualitative researchers reported that replication should never be used (Makel et al., 2021). Given the prevalence of qualitative research in education, it is important to examine replication’s relevance. In this commentary, we argue that replication is relevant to the qualitative lens in at least three ways. First, replication supports the established values in qualitative research of transparency and intentionality. Second, replication can be used to assess the well-established tradition of transferability. Third, replication can evaluate connections between reflexivity, as evidenced by positionality statements, and qualitative research findings.

…and an article in response…

• Is replication possible in qualitative research? A response to Makel et al. doi: 10.1080/13803611.2024.2314526

There has been much debate in recent years about how open research practices, which have been promoted in efforts to improve research robustness, may (not) be appropriate for qualitative methodologies, particularly in educational research. Among these is a concern for replication efforts. Makel et al. (Citation2022) make the case that “replication is relevant to qualitative research”. The authors argue that concerns surrounding the transferability, intentionality, and transparency of qualitative research may be eased, or responded to, by replication studies. Here, I argue that there are three fundamental questions that need unpacking before declarative claims can be made about the relevance of replication to qualitative research. This includes critical questioning of what we mean by: replication, qualitative research, and rigour. I address each of these issues and encourage a more nuanced appraisal of how replication may, or may not, be epistemologically, ontologically, or methodologically compatible with the goals of qualitative research.

Jenni Adams (Open Research Manager, Scholarly Communications Team) will give a brief summary of the article and will open discussion by highlighting some of the points they thought were good/bad about the paper.

Where

This event will be hybrid, if you wish to attend in person we will be meeting in View Room 4 on the 4th Floor of the The Students Union.

Add the event to your Google Calendar using this link.

If you aren’t able to attend in person but would like to join the Journal Club you can do so using Google Meet.

When

2024-03-26 @ 12:00

Future Events

To stay abreast of up-coming events you can either sign up to the mailing list or subscribe to the RSS feed.

What

The chosen article to that we will review and discuss is an essay from The Journal of Electronic Publishing

• Pooley J. Surveillance Publishing The Journal of Electronic Publishing (2022). doi : 10.3998/jep.1874

This essay develops the idea of surveillance publishing, with special attention to the example of Elsevier. A scholarly publisher can be defined as a surveillance publisher if it derives a substantial proportion of its revenue from prediction products, fueled by data extracted from researcher behavior. The essay begins by tracing the Google search engine’s roots in bibliometrics, alongside a history of the citation analysis company that became, in 2016, Clarivate. The essay develops the idea of surveillance publishing by engaging with the work of Shoshana Zuboff, Jathan Sadowski, Mariano-Florentino Cuéllar, and Aziz Huq. The recent history of Elsevier is traced to describe the company’s research-lifecycle data-harvesting strategy, with the aim to develop and sell prediction products to unviersity and other customers. The essay concludes by considering some of the potential costs of surveillance publishing, as other big commercial publishers increasingly enter the predictive-analytics business. It is likely, I argue, that windfall subscription-and-APC profits in Elsevier’s “legacy” publishing business have financed its decade-long acquisition binge in analytics. The products’ purpose, moreover, is to streamline the top-down assessment and evaluation practices that have taken hold in recent decades. A final concern is that scholars will internalize an analytics mindset, one already encouraged by citation counts and impact factors.

Adam Partridge (Open Research Training Lead / ELIXIR-UK DaSH Fellow at University of Sheffield) will give a brief summary of the article and will open discussion by highlighting some of the points he thought were good/bad about the paper.

Where

This event will be hybrid, if you wish to attend in person we will be meeting in Meeting Room 1 of The Diamond, 32 Leavygreave Road, Sheffield, Broomhall, S3 7RD.

Add the event to your Google Calendar using this link.

If you aren’t able to attend in person but would like to join the Journal Club you can do so using Google Meet.

When

2024-01-18 @ 12:00

Future Events

To stay abreast of up-coming events you can either sign up to the mailing list or subscribe to the RSS feed.

What

The chosen article to that we review and discuss is a recent publication Nature Human Behaviour

• Protzko, J., Krosnick, J., Nelson, L. et al. High replicability of newly discovered social-behavioural findings is achievable. Nat Hum Behav (2023). doi : 10.1038/s41562-023-01749-9

Failures to replicate evidence of new discoveries have forced scientists to ask whether this unreliability is due to suboptimal implementation of methods or whether presumptively optimal methods are not, in fact, optimal. This paper reports an investigation by four coordinated laboratories of the prospective replicability of 16 novel experimental findings using rigour-enhancing practices: confirmatory tests, large sample sizes, preregistration and methodological transparency. In contrast to past systematic replication efforts that reported replication rates averaging 50%, replication attempts here produced the expected effects with significance testing (P < 0.05) in 86% of attempts, slightly exceeding the maximum expected replicability based on observed effect sizes and sample sizes. When one lab attempted to replicate an effect discovered by another lab, the effect size in the replications was 97% that in the original study. This high replication rate justifies confidence in rigour-enhancing methods to increase the replicability of new discoveries.

Neil Shephard (Research Software Engineer at University of Sheffield) will give a brief summary of the article and will open discussion by highlighting some of the points he thought were good/bad about the paper.

Where

This event will be hybrid, if you wish to attend in person we will be meeting in Workroom 2 of The Diamond, 32 Leavygreave Road, Sheffield, Broomhall, S3 7RD.

Add the event to your Google Calendar using this link.

If you aren’t able to attend in person but would like to join the Journal Club you can do so using Google Meet.

When

2023-12-06 @ 13:15 Sheffield Reproducibilitea Journal Club is back in time to see 2023 out.

Future Events

To stay abreast of up-coming events you can either sign up to the mailing list or subscribe to the RSS feed.

Follow-up

This paper was, as is now noted, Retractaed as it was pointed out that it was itself not adhering to the standards it was suggesting should be. There is some further discussion on the Statistical Modeling, Causal Inference and Social Science blog What’s the story behind that paper by the Center for Open Science team that just got retracted?.