Imagine asking a friend for a recipe to their homemade chocolate cake and getting one of the following two responses: First: Collect the material, mix with a KitchenAid Stand Mixer (Model KSM150PSOB) for 65 seconds with a whisk attachment, and bake using a GE wall oven (Model PK916SRSS) using the default settings. That is not going to be an easy recipe to follow. What are the ingredients? Can I get the same results with a different mixer or oven? Default settings, these two words are the bane of my existence, they are meaningless and should be taboo from scientific writing. The second response: It’s my grandmothers recipe (see Grandmas Cookbook, 1954).
What do these analogies have to do with science? The more I read my peers’ papers the more I see trending towards incomplete Material and Methods sections. Typically, I find that there is a fairly even mix of the above mentioned strategies to explaining what was done in a paper. Personally, I don’t find either explanation useful.
Below I will highlight some guidelines for writing Methods sections, some examples of useful and less useful descriptions, and some potential solutions for people writing their own papers.
The Goal: Before we start we need to know what we are aiming to achieve. This may be the reason this problem has been perpetuating. I will use Day’s How to Write and Publish a Scientific Paper as our benchmark.
…careful writing of the [Materials & Methods] is critically important because the cornerstone of the scientific method requires that your results, to be of scientific merit, must be reproducible; and, for the results to be adjudged reproducible, you must provide the basis for repetition of the experiments by others.
That is a perfect reflection of what someone should expect when reading about methods. Let’s look at two examples from published research and see how our peers have done.
These examples were chosen to highlight the growing problem with our ability to report in the digital age. As a group we are, generally, very good at putting our benchwork into text with enough detail that others could follow our experiments (Ex. 1). As technology has taken a front seat with current research I feel that we haven’t developed a successful strategy for documenting our computer based work.
Added to the lack of effective communication we have journal page limitations, thus, further forcing us into brevity. However, I don’t think that we can continue to use that as an excuse. With more and more journals going to online formats and the ability to create supplemental documents we must still adhere to the Scientific Method and provide detailed explanations. According to Day, but not to the reviewers, Example 2 should not have been “adjudged reproducible.” I cannot take their gene set and return the same results using the information they supplied. I have used SNAP in my own research and here are the steps I did to create one file:
I don’t think that ‘I used SNAP’ qualifies as “basis for repetition.” That is 11 different commands send to SNAP. The example above shows only one program and only one file manipulation. Most modern genome papers use and average of 6 different pieces of software; and of those 6 most require other programs to function, compounding the problem further. How do we report on these computational processes in a concise AND useful manner?
Solution. My first suggestion: error on the side of too much information. If you’ve used a program you need to describe your settings. For the commands found in my SNAP example above, I did exactly what was described in the MAKER readme file. In that case there would be no issue with stating “Custom HMM’s were created using SNAP as described in the MAKER readme.” However, if I’ve deviated in anyway from that procedure I must discuss those changes.
My second suggestion: if page limitations are an issue submit supplemental documents, or indicate where someone can go to get the detailed information required. Dequard-Chablat et al. (2011) provide an excellent example of this. Most journals don’t even print hard copies any more and all have SI online, do your part to make useful information available without worrying about destroying the rainforest.
My third suggestion: if you are going to cite previous work(s) make ensure the reference has done a proper job of detailing procedures. Perpetuating a terrible description is lazy and a cop-out; do the right thing and provide a decent protocol (your paper could become the new “see” paper and up your citations). Do not daisy chain methods citations! If you say ‘procedures followed as in Calvin & Hobbes (2001)’ and I go to that paper and it links me to another reference it becomes very annoying and time consuming.
Conclusions: Ninety percent of the people that read your publication will not even look at the methods, so I know that putting in the proper amount of information seem futile. However, for the 10% that need to know what you did give half-hearted details hinders the progression of their work, and ultimately slows down our progress as scientists. We’ve all built off of previous efforts and know how difficult it can be to recreate our own experiments lets make things easier for our peers and future graduate students by doing what we are supposed to, and be as transparent as possible.
Day, RA. How to Write & Publish a Scientific Paper. 4th Edition. Phoenix: The Oryx Press, 1994.
Volz, P., & Beneke, E. (1966). An aquatic fungal bio-assay method for detection of carcinostatic agents Mycopathologia et Mycologia Applicata, 30 (2), 97-114 DOI: 10.1007/BF02130356
Denoeud F, Roussel M, Noel B, Wawrzyniak I, Da Silva C, Diogon M, Viscogliosi E, Brochier-Armanet C, Couloux A, Poulain J, Segurens B, Anthouard V, Texier C, Blot N, Poirier P, Ng GC, Tan KS, Artiguenave F, Jaillon O, Aury JM, Delbac F, Wincker P, Vivarès CP, & El Alaoui H (2011). Genome sequence of the stramenopile Blastocystis, a human anaerobic parasite. Genome biology, 12 (3) PMID: 21439036
Déquard-Chablat M, Sellem CH, Golik P, Bidard F, Martos A, Bietenhader M, di Rago JP, Sainsard-Chanet A, Hermann-Le Denmat S, & Contamine V (2011). Two nuclear life-cycle-regulated genes encode interchangeable subunits c of mitochondrial ATP synthase in Podospora anserina. Molecular biology and evolution PMID: 21273631