Thursday 7 February 2019

Separate research reports from just so stories

Let's assume you are a researcher, and you did some researcher, alone or in a team.
You want others to know what you did and found.
Now there are a few options, not mutally exclusive from each other.

A) you go to conferences, tell your colleagues about it - maybe there is even a poster about your research but that's it
B) you use platforms such as the Open Science Framework to upload your material, data and analysis scripts (please with readme files)
C) you write it up as a research protocol and submit it (uh, hang on, what does that mean?)
D) you contact a journalist and shout out a fancy "just so story" - that is the focus is on how you interpret your finding (data) - how you got there is reduced to a bare minimum (or even moved to a supplementary section which can be something like B)

You think the conference thingy is common? partly right, but few researchers stop there
Some have adopted B) but this is not providing your research a lot of publicity, even fellow researchers will have a hard time finding it and understanding it - there is the "WHY should I care" missing - or said differently: it is hard to understand what all that material / data is about. 

Then there is C). Aehmm, no, not any longer but maybe soon again. Few write honest research protocols, the current publish-perish "industry" encourages more the last option - you sell your research as a nice and tidy "just so story". The more media training you got (from home, in school or at a university) the more you can write like a journalist, i.e. engaging, fancy articles ...

We got into D) instead of doing C) due to wrong incentives in the publishing sector of academia. Research is not any longer disseminated in a journal and then evaluated by peers (post-publication review porcess). Research protocols have to pass through a door of editors, reviewers and worse: false understanding of what research publishing is about. This false understanding created an awkward "hierarchy" of journals - seen in a weird metric called impact factor - and a subsequent training of new researchers into "having to publish" in certain journals.
Worse: in Scandinavia some journals give "points", others no points.
Though this system has at least one advantage: the journals are screened and predatory journals are not listed in the list of journals giving points.

So called high impact journals are not better when it comes to the research being reproducible, proportion of fraud etc. These journals just have a status. Let's use marketing language: they became a brand. The problem is: they tend to make research protocols into just so stories - yes the researchers do it because of the incentives.
Worst part - easy read at first, but lots of details missing when one wants to replicate.

What research needs, is not a brand, open science needs the ingredients = research protocols.
No coke just coffeine

Any discussion about open access journals is missing the point. Open Access journals do not convert just so stories back to honest and transparent research protocols. Full open science requires a change in incentives: value preprints and encourage "commenting" on preprints - allow a public discussion. Not a few (rarely unbiased) reviewers to comment blindly on your work.

Take the pressure away from publishing in high impact journals, evaluate a researcher by her openness, collaborative spirit, transparency, creativity - but not solely on how novel the result is and novelistic she can write.