Checking Data

Following my brief piece on the sun and some of its potentially adverse effects, someone was kind enough to leave this comment:

"Don't forget that skin cancer is cured by melatonin and vitamin A.
Reference 1
Reference 2

I am extremely grateful that he wrote, because it gives me the opportunity to illustrate the point about checking the data. The person who wrote was obviously motivated by a desire to help, but he had been given some information that was not correct.

I pulled out both of these papers and discovered that they said something rather different.

The first paper in the Indian Journal of Medical Sciences is quite a good review article about melatonin. There are several reasons for us to be interested in melatonin, and I actually plan to write a brief review about it as soon as I have finished analyzing the literature. There is indeed some experimental work to suggest that melatonin may modulate and in some cases arrest the growth of some malignant cell lines (1. 2. 3.) But we are still a long way from being able to say that melatonin is a viable treatment option for melanoma.

The second paper does not talk about melanoma treatment at all: it is a paper about the possible use of Vitamin A as protection against cancer in sun-damaged skin.

As I mentioned in my piece about the new open access journals, our next task is to help people learn how to interpret the data in all of these papers. I am on the review and editorial boards for several scholarly research journals and it is amazing how often the article summary does not tally with the data in the paper.

And also how often people cite articles that they cannot have read. Within the last 24 hours, the editor of a well-known journal asked me to review a research paper that had been submitted for publication. There was very interesting data in the paper, but the author had made at least one error in his citations. He accidentally claimed that a piece of research had discovered one thing, when the research had actually said something quite different. I happened to know that because I wrote the original paper!

This problem of misinterpreting data keeps coming up, and I’m sure that we may all have missed something at some time. It’s sometimes hard to analyze all the ins-and-outs of a piece of research.

You may have heard about a bit of a scandal over a book on gender differences that was just published. I liked reading the book, but I was worried about some of the references about hormonal effects on behavior. I know this literature very well, and some of the statements and inferences were a bit questionable. Now several experts have been checking some of the citations about gender differences in language that were used by the author. They discovered that some were just plain wrong.

Have you ever heard that one about, “Women talk more than men?” It always sounded pretty silly. And of course, if something sounds silly it probably is. Does it matter? If someone is going to write a popular book, good luck to them: I actually think that scientists have a duty to make their work accessible to the public.

But if they quote a lot of papers to give themselves an air of authority, they really have to make sure that they are not misleading anyone. If some references are wrong, then it’s difficult to be sure which ones we can trust.

We cannot have one set of rules for a book written for professionals and another for the general public. Each must be created with the utmost care, clarity and attention to detail.

My last book and CD series – Healing, Meaning and Purpose – cites over 800 books and websites, all carefully checked by yours truly. I did this just so that people can follow up on what I say. That way my books, articles, papers – even this blog – become portals to a world of ideas and information that you can use to improve your life.




Comments

[Reg Adkins]

Richard you make an excellent point. The availability of research has become tremendous. However, the skills necessary to adequately assess and understand this research is rarely taught. A friend (and former professor of mine) Dr. Robert Bickel has written a book that addresses that skill. At least in the field of applied research the title, Multilevel Analysis for Applied Research: It's Just Regression! is only available on advance order at this point but might be worth adding to your library. I plan to add it to mine.
I enjoy the forthright nature of your work.
Reg Adkins

[Richard Petty]

Dear Reg,

Thank you so much for your kind note.

I never have any wish to expose other folks' errors unless they may cause harm. After all, I'm quite sure that I've made more than one mistake in my life!

I totally agree with you about the importance of inculcating the skills necessary to assess and evaluate research. Just this morning I was squirming when I heard a medical commentator on the TV completely misquote some new research.

Thank you also for the tip about Dr. Bickel's book: on the strength of your recommendation I have placed a pre-order at Amazon!

Kind regards,

RP

[Annalisa]

I just stumbled on this blog, and I'm looking forward to subscribing.

I agree that scientists have a duty to make their research accessible to the public, and my own blog is an effort to do so.

[Richard Petty]

Dear Annalisa,

Thank you so much for commenting.

I have just had a look at your work and I am going to link to you as well!

You may have seen that I also commented on Dean Radin's excellent new book here:
http://www.amazon.com/gp/product/1416516778?ie=UTF8&tag=publicparapsy-20&linkCode=as2&camp=1789&creative=9325&creativeASIN=1416516778

Kindest regards,

RP

[Stuart]

I loved the book the "Female Brain" and I think as a son, husband & father of a daughter I only wish I had read it much earlier. If you look at the negative comments they are only on the 2 points outlined in your blog. While I agree there should be as few errors as possible it does appear to be an over reaction for other reasons, none having anything to do with science. The idea that a women's life is so dramatically effected by hormones and not the evils of an oppressive society has stirred the pot. I see her departure from conventional feminist thinking to be the real issue.

[Richard Petty]

Dear Stuart,

I am sorry that we are not going to agree on this one, and that I must respectfully disagree with you.

I did not want to focus on the book The Female Brain. But if I must, let me make a couple of quick points. It is an enjoyable read. But the frank mistakes and misinterpretations are of huge concern. This is a great example of the Barnum Effect.

What so worries me about this kind of book is that it offers a kind of faux understanding. It is rather like the excitement that many of us feel when we first read Freud. Now, finally, we feel as if we have found explanations for thoughts, motivations and behaviors. But while psychoanalysis is an untestable meta-theory and many of its models and theories have been found wanting or even dead wrong, the assertions in the book are and have been tested. And many of them have been fund to be wrong.

I don’t want to go through a line-by-line analysis of the book. But let’s take a very simple example: her discussion about oxytocin. She simply supports the idea that oxytocin may be the affiliation (a.k.a. the cuddle) hormone. There is some supporting data in humans, but not much. In fact humans seem to act quite differently.

A book like this offers a shortcut to understanding that is built on sand.

I am very grateful for your comments, and I apologize for the delay in responding. They were so important that I am going to write a whole piece about Folk Psychology and Folk Knowledge that will, I hope, contribute to this important discussion.

Kind regards,

RP