What to do when you're wrong...

Sorry for the delay in posts. I've been busy being wrong.

As we know, one of the first tenets of "good" science is reproducibility. If I follow someone else's protocol, using the same starting material, I should get the same results. But if you do that, and you get a different result, how do you know whether the alternate result is because you didn't quite replicate the protocol, or because the original conclusions were wrong? Or what if you interpreted the same results differently?

I can imagine this is an especially large problem when you are new to a protocol (or at least that is my excuse), as you lack a sense of the range of possible outcomes. For example, a labmate was trying to interpret a catalase test earlier, and would have said that all our results were negative compared to the plate of mixed soil bacteria I had, where adding a drop of hydrogen peroxide caused a baking soda and vinegar type volcano effect. Two of the bacteria were supposed to be catalase positive, and if we added tons of hydrogen peroxide directly to a plate with a high density of bacteria and looked really really close for a few minutes and imagined a positive result, we got one or two small bubbles. This is exactly the same as if I drop a drop of the hydrogen peroxide on uninoculated media. So did the authors who stated these organisms were catalase positive actually mean it, or did some nervous inexperienced student  who was told to look for any sign of bubbling squirt a bubble onto the slide and say there was a positive result? Who decides where the boundaries of a positive or a negative result are - would it not depend on "how" positive or negative your control organisms are? Do people publish the organisms they used as controls alongside their determination of whether something shows a positive or a negative result? No evidence for these yet.

Maybe we should just mandate that all standards in genomics tables be replaced by full-colour pictures. Voluptuously bubbling hydrogen peroxide. Stunning Gram stains. Replace someone else's interpretation of ambiguous data with informative eye candy.

In another instance, I have been working (a bit too long) on trying to get well-published qpcr primers to work, and finding that the conditions as close to the original ones I can reproduce in my lab just don't work. I was afraid I had contaminated the freezer stock of my organism, or put too much or too little template, or used the wrong temperature...something that was my fault. I tried to come up with an answer and a solution for my PI for when I told her that things weren't really working. But apparently, however, what I thought was close enough to the original conditions probably is not; the primers were tested against a sequence placed in a purified plasmid, and I am using genomic DNA.

Which brings us to another class of issues with reproducing an experiment - what if you purposefully are not exactly reproducing an experiment, because you don't believe the methods used are valid and/or reliable and/or result in biologically meaningful conclusions? Why waste time and money trying to reproduce an experiment that was invalid when it was made, and still invalid now, just to try and compare your results to ones you cannot trust? At my stage, I think it is so you can fit in; the ability to reproduce something invalid is a valuable skill for perpetuating some of the falsehoods of science, which you have to be able to do in order to prove your worth in breaking the (methodological) status quo.

So why use a plasmid template for qPCR, even if it does a poor job representing the kinds of templates you will be comparing with from the environment? Because it is one of the standards for this kind of data collection; we can hide behind completely non-reproducible data, uncertain whether it is due to the methods, or because we are dealing with "complex" environmental samples that nobody else is likely to exactly mimic.

But no matter what I may do to try and convince myself I wasn't wrong, I was. I tried to reproduce results under the conditions I thought they should be done, not those in which they were intended.

So if you are wrong, either make sure you do it reproducibly so you can challenge what is deemed right, or do so using conditions nobody expects ever to exactly reproduce. But know when to accept that it is you, not science, not the protocol, that is wrong.