Peer Review Data
A Reviewer’s Quick Guide to Assessing Open Datasets
There is a growing appreciation of the importance of data storage, maintenance and sharing among the scientific research community. Increasingly, government bodies, funding agencies, and journals are making data retention and availability part of their requirements for grants and publications–and reviewers are often asked to evaluate author-supplied datasets as part of peer review.
Peer reviewing data can sound like a big commitment, but doesn’t have to be time consuming or difficult. When invited to review a manuscript with an associated dataset, perform these five quick checks to surface any potential issues.
Is the data accessible?
Is the data underlying the manuscript you are reviewing available at the url listed, or other location described? In cases where there are restrictions on sharing the data, have the authors clearly explained the reasons why?
Can you tell what you’re looking at?
Have the authors used a logical naming convention? Are the files accompanied by a clear description of the content and format?
Does the data you see match the data referenced in the manuscript?
Does the data relate to the study currently under review? Is each data set mentioned in the manuscript included?
Does the presentation of the data make sense?
Is the format, file type, and arrangement of the data clear, and appropriate for the type of study and the research that was conducted?
Does the data itself make sense?
Are the values reported physically possible and plausible? Do results fall within the appropriate range for the phenomenon described? Are the data points internally consistent?
That’s it! If a data set fails any of these five tests, be sure to mention it in your review.
Download a PDF
Download the Reviewer’s Quick Guide to Assessing Open Datasets, developed by PLOS in collaboration with the Cambridge Data Champions.
Download