Myths About Using Surveys and Questionnaires in Psychology

Source: Image by eslfuntaiwan from Pixabay

One of the most commonly-used measurement tools in psychology research and practice is what’s called the patient-reported, or self-reported, test. This method involves giving a patient, client, or research study participant a survey or questionnaire to gauge their experience. The survey or questionnaire often asks about how someone feels or what they think. Sometimes, the questions can be about physical symptoms such as pain, fatigue, and poor sleep.

A trend I have noticed is that surveys and questionnaires are often denigrated with pejorative terms like “subjective” or “unreliable." The implications of these criticisms are that asking people to report their feelings, symptoms and thoughts is not worthwhile—or at least is less worthwhile than more “objective” tools like blood tests and brain scans. The definition of subjective is that it is something based on a person’s feelings while something objective is unbiased or based on facts.

This second-class status of self-reported information is highly detrimental to patients and the public in general for several reasons.

A primary reason why the denigration of surveys and questionnaires is harmful is that it is based on a myth. Despite claiming to be objective, the opinion that survey and questionnaire data is not useful is, in fact, not based on facts and is, instead, subjective.

Let’s consider some of the other types of tests that psychologists use in research or that physicians may use in clinical practice. The first is blood tests measuring chemicals in a person’s body such as stress hormones (cortisol), signs of inflammation (c-reactive protein), and metabolism (glucose). Other tests include body scans of the body ranging from X-rays to magnetic resonance imaging. Another common test is to ask the clinician how a patient is doing; for example, asking them to rate a patient’s function in a certain area on a seven-point scale. A common myth is that all these different tools (blood tests, scans, clinician report, surveys, and questionnaires) are measuring the same thing. They are not. What is in someone’s blood or shown on a scan is not the same thing as the emotions they are experiencing, the symptoms they have, or the thoughts running through their mind. Someone’s emotions, thoughts, and symptoms can differ greatly from what their physician or psychologist believes they are experiencing. The myth that all these tests measure the same thing forms the basis for disregarding self-reported data.

Another myth about surveys and questionnaires versus other types of tests is that other types are more reliable. This is not necessarily the case. There is a laundry list of blood tests that are unreliable but still used. Surveys and questionnaires, just like other types of data, have to go through a process of development to ensure that they are reliable—and they sometimes may be more reliable than other forms of data. What is measured by surveys and questionnaires—such as emotions, thoughts, symptoms)—can vary from hour to hour (or even minute to minute) and this natural variability is often mistaken for unreliability. This myth about surveys and questionnaires feeds into the previous myth that self-reported data is not worthwhile.

Despite the logical arguments for self-reported data, there is also a moral argument. Self-reported information from surveys and questionnaires is a way to measure what a person perceives about themselves. To disregard these perceptions in favor of other tests that might not measure the same thing is, to be blunt, paternalistic. It simply says that “I know better than you what you are experiencing based on this blood test/scan/my own opinion.” I’m not saying that these other tests should not be used; only that self-reported data should not be considered as less valuable than other tests.

In case you are wondering why I just spent an entire post waxing poetic about survey and questionnaire myths, there are several reasons. First, disregarding what people say on surveys and questionnaires is disregarding their experience and that is highly invalidating and harmful. Another problem is that time and money is often wasted chasing an “objective” measure of some psychological experience when we could have just asked people. Surveys and questionnaires are often much cheaper both to develop and use than blood tests, scans, and other biologically-based tests. While each type of test has its own place and value, self-reported surveys and questionnaires should definitely be considered as valuable as other measures.