Saturday, January 10, 2015

Improving the Scientific Journal?



I've been trying to wrap-up and submit my dissertation research and have been dealing with a number of different journals. I recently looked up how expensive it would be to start your own journal and found that the physical costs would be under a few hundred dollars a year, while the costs of people (e.g., Editors, Copywriters, etc.) would cost a few hundred dollars per paper which does't seem all that expensive when profit margins are at least 20% for publishers. But in starting your own journal, what things would you do to improve the process? Below I offer a few possibilities, but am interested in other ways (and am not as radical as others have suggested 1, 2).

1. Pre-registration
Pre-registration, where the design of a study and its subsequent analysis protocol is submitted to a journal before competing the study, is an emerging, but controversial topic, with some in support, some against, and some unsure.  Pre-registration appears to be a fundamental starting point for many of the other suggestions below, with preregistration encouraging open access, data sharing and encouraging attempts at replication. One of the main concerns about pre-registration is that it will constrain scientists from the fiddling and post-hoc data-mining that currently runs science. However, I don't see why pre-registration with your original design and analysis plan can't also report new analyses that come after the fact. The difference with pre-registration is that your original and possible non-significant results are reported as well as the finagled and later thought of results. Especially since we are over-reporting positive results, it will be nice to see why experiments fail as well as see non-replications of high-profile studies that are currently suspect but not challenged in the literature.

2. Open Commenting/Changing Peer-Review
Some journal have adapted a two-step review process in which papers that pass an initial review are posted for open commenting as a discussion paper and then later reviewed in the traditional ad hoc review system. As the review process often takes a minimum of a few months and could potentially gone on for years, the open commenting period helps alleviate the strain of finding reviewers. This also opens up scientific dialog spurring and collaboration.

3. Open Access
Without open access most of scientific articles are only available to those who are willing to pay for the articles or work in a place who buys publisher access. Open access asks researchers to pay for their research to be made available to anyone once accepted. The Right to Research Coalition provides a list of benefits across various agents. Though Nature and other publishers caution that Open Access are not a panacea that some think.

4. Data Sharing
Data sharing provides a number of benefits including: scientific integrity, increasing the impact of research, preserving data for future use and teaching purposes. Some fields seem to be more interested in open data, but there are a some issues, like confidentiality and vulnerable subjects, that need to be kept in mind.

5. Removing Impact Factor
The impact factor (IF) - the number of citations a journal receives in a year divided by the number of articles published in the journal, is meant to be a measure of the impact and quality of scientific output. Over time, IF has become a shorthand prestige and has lead to both journals and scientists chasing every higher numbers. However, this chase has perverted whether any particular article is making an impact and how science is crafted and published.

Edit 1/15/2015 - Michael Eisen who co-founded PLoS did an excellent AMA on Reddit yesterday about academic publishing addressing many of these issues.

No comments:

Post a Comment