The Importance of Comparison

` I welcome all who come here to my critical thinking and science blog, and encourage my readers to comment on any of my posts.
` The only downside to this is that some visitors probably won't be familiar with how critical thinking and science really work. I know for a fact that most people don't quite understand these things and even worse, many are intimidated by them!

` If you will indulge me, I'd like to try writing about why we have scientific methodology:

` Human beings are able to come up with any idea they like to explain any phenomenon they choose, so goes our abstract nature. The problem is, there can be many sensible-sounding explanations for the same thing; plus, these different explanations can contradict one another!

` Even more, it can be easy to 'prove' many of them right, because quite often we can find what we're looking for as evidence to support our ideas: As long as we ignore anything that goes against our idea, it is easy to convince ourselves that we are correct even when we may not be - and even when other ideas actually make more sense than our own!

` I'm sure most of you know exactly the type of thing I'm talking about.

` Just for kicks, let's say you have this idea that cats are vile, nasty creatures. Therefore, every time you see a cat hiss and spit at someone, you might think to yourself; "Good for nothing cats!"
` But what if you believe that cats are quite nice by nature? Seeing the same cat hissing and spitting would have you thinking; "Aw, the poor kitty feels threatened!"
` Needless to say, that's why two people can see the same thing and have two different opinions about what is going on!

` The fact that more than one interpretation is possible for one event is arguably the main reason for the scientific method: You can make all the observations you like, but that is only the first step in figuring out what is actually going on.

` If you have two potential explanations for something - hypotheses - and you find just as much evidence for either one, how can you decide on one over the other? For a common experience we'll say your car won't start, and you figure it's because the battery is dead. So, you install a new battery and it starts up just fine.
` But, you ask, what caused the battery to go dead to begin with? One hypothesis is that there was a problem with the battery. Another hypothesis is that your voltage regulator went haywire at some point and drained your battery.
` How can you tell which one is right just by thinking about it? Face it, you're stuck! The only way to resolve the issue is to try to falsify each hypothesis - in other words, do your best to prove them wrong! In this case, examine the dead battery and the voltage regulator, and run the engine to see if the battery goes dead again.
` Of course, you can't just try proving one wrong, because proving one wrong does not automatically make the other right; the real explanation could be something you didn't even think of! (Perhaps one of your map lights was on all night? What about the alternator?)
` Clearly, you must try to prove both of them wrong! And if you find nothing wrong with either the old battery or the voltage regulator, well, let's just say you may find that cars are very complicated things.

` In a way, science is all about creating a large number of hypotheses with our big imaginations (and our ability to make logical inferences) and then 'weeding out' the wrong ones: Most of the ideas scientists think up in fact do turn out to be wrong.
` Though one cannot actually prove anything right, at least the hypotheses that are left are most likely to be true!
` So, how does one go about this 'weeding' business?

` Just the other day, I had been reading a bit of Keith Stanovich's How to Think Straight about Psychology. It gave me a few ideas about how to explain this:

` Stanovich says that one thing you need to understand is the importance of comparing one thing to another. If we are looking for patterns in the world, we cannot rely on one isolated event. In other words, if you're walking down the street and see a cat flying through the air, does it make sense to assume that you'll be likely to see this again, or that any cat could fly?
` Of course not, you say!
` That's why we must be careful of jumping to conclusions with our interpretations of events. A bit further investigation might show that the 'flying' cat was actually flung from the window of a nearby house by a person who is of the opinion that cats are nasty and vile!

` So, the way scientists find patterns is - drumroll, please! - observe a lot of instances of the same type of thing. That way, they can compare all their data and thus have a better idea of whether or not something is particularly unusual. It's a commonsense thing - really, it's the best way to figure out what to expect from the world.
` Patterns, in other words.
` The way scientists generally do this is to create (or find) very similar situations to observe, so that the events they are comparing really are comparable; apples to apples, rather than apples to oranges.
` Within those confines, the difference that one change makes is more noticeable. Also, by isolating events from various types of influences, you narrow the possibilities of what can happen.
` This is referred to as control, which Stanovich notes is the second main thing one needs to understand about scientific thinking.
` A typical example is lab experiments with hapless rodents. Let's say we have sixty lab rats of a particular genetic strain, all of which have a problem; someone has severed a nerve in the left hind leg (which the rats are probably thrilled about). So, they're all very similar in that way. They are also similar in that they all live in the same kind of cage and eat the same amount of the same type of food.

` What we have here is a controlled situation in which the only real hypothesis that would explain any healing of the rats' nerves would be that it just healed by itself. Not much chance of any interference, is there?

` That is, unless those people in white lab coats did their own interfering: In this experiment, twenty of the lab rats are left alone, twenty are injected with Drug X, and twenty more are injected only with saline solution (which basically does nothing).
` Why would scientists pump a third of the rats full of IV fluid? Because, the mere act of injecting the rats with something has an impact in itself! By injecting one group with the drug and another with an inert substance, they should be able to tell if the act of injecting the drug does anything different from only the act of injecting.

` This is especially important in medical studies using human beings; taking a fake medication or even a fake dietary supplement can have drastic effects on one's well-being. It can not only make pain go away, but it can even make one's condition improve! And yet, the person has actually swallowed nothing but sugar or saline!
` That's called the placebo effect, if you've never heard of it. Now, the main reason I refer to Stanovich in particular is because he brought up a very good example of what happens when you're not good enough at creating comparable groups:
` It is the case in which the portacaval shunt - a device which lowers blood pressure in the liver - was recommended for treating cirrhosis. Many doctors (and patients) swore by it, and it was quite popular until the mid-1960's.

` What happened?

` In 1966, a pattern was found among all the various studies which demonstrated the shunt's effectiveness: The conditions were not very well narrowed down.
` Many of the studies had no control group, so there wasn't anyone who thought they had the shunt implanted when they didn't. Among those studies, 96.9% were judged to show that the shunt was at least moderately effective.
` Some other studies did have a control group, though the patients were not assigned randomly to each group. Since people aren't as alike as lab rats, it's important to randomly assign people for treatment to make sure that similar patients tend to wind up in different groups, preventing a selection bias.
` In other words, if the people selected for the 'real' treatment have a lot of help and support from their families or are chosen specifically because they are 'good candidates', that's not very random is it? You're just rounding up the ones that have a better chance!
` In fact, it seems this really did happen because in 86.7% of these studies the shunt was deemed at least moderately effective: And yet, in the studies that employed random selection and a control group, doctors found that only 25% of them showed at least moderate effectiveness.
` That's not nearly enough evidence to show that there's any more to the shunt than a placebo effect or being implanted in people who had an advantage, so it stopped being used for treating cirrhosis.

` So you see, if the scientists involved hadn't realized that the only studies that had shown good results were ones without proper controls - and thus having poorer comparisons - medical doctors would continue installing the portacaval shunt in a situation where it wouldn't have done anything!
` More than that, if other, similar mistakes had not been caught in medical science... well, I personally don't care to think about that! But as I've said, the scientific method is to help us figure out which hypotheses don't make sense.
` In this case, the hypothesis that the portacaval shunt helps treat cirrhosis is one that doesn't make sense (though it has been shown effective in treating other disorders). The only really convincing explanation for why so many doctors held this hypothesis is that it was based on flawed studies.

` Scientists (especially medical scientists) have a lot of pressure on them just to make comparisons, don't they? It's easy to be wrong in this world. And, if anyone is going to get good at being objective, this is one thing they have to do.
` It is also one thing that separates science (or even simply critical thinking) from something that isn't. I find this to be particularly evident for things that tend to shock and confuse people: Say you're walking through a field one day and come across a dead cow whose eyes, mouth, tongue, udders and sex organs seem to have been removed with surgical precision.
` Disturbing reactions aside, the question you ask is "Why?"
` One can come up with any number of hypotheses for what has happened to the cow. I know this because it's been done.
` First of all, you must make comparisons: Is this somewhat unsettling sight very common? As it turns out, it is: This is merely what happens when animals such as crows and maggots eat out the softest tissue of an animal carcass. (Cow hide is particularly tough to poke a hole through, so they just eat what they can get.)
` Then, typically the carcass swells up to the point where the edges of the holes are stretched until they appear even more clean-cut than they had.
` In other words, this is a fairly normal condition of the remains of a cow (or other large animal) that has died of natural causes - something that just happens occasionally.

` Many other people don't take that view, however, commonly preferring to believe that extraterrestrial beings have been picking up cows all around the world, cutting certain parts of them out, and for some odd reason, dropping the carcasses back in the field where the cow once lived.

` Sure, I suppose the evidence seems to fit both explanations equally well - especially to people who think cows with missing soft parts is unusual - but seeing as this has spurred some people to actually watch these same changes happen to animal carcasses left out in a field, is there anything else to explain?
` In this instance, though, no hypotheses were technically proved wrong - instead, this is a case of the fact that simple and straightforward explanations tend to make more sense than ones in which all kinds of speculative and unnecessary factors are introduced. I'll have to write about that some other time.
` The case of the Swiss cheese cow, of course, is a bit more complicated than that: Nevertheless, I think it's a good - if morbid - example of something which spurred two very different hypotheses to leave you with.


` ...One more thing; if anyone is curious, a similar study to the one I described involving rats with severed nerves was actually carried out. The drug used was a chemical that stops scar tissue from forming, and the rats injected with it had a near-complete recovery while the other two groups only had a fairly good recovery.
` So, the scientists have pinpointed, scar tissue (at least in rats and probably in humans) seems to prevent nerves from healing.