A Question of Viable Scientific Questions

[Johnathan Clayborn]

As part of my PhD coursework several of my classes have me referring back to a book called "A Primer in Theory Construction" by Paul D. Reynolds. Overall the book is pretty solid and discusses the ins and outs of developing a scientifically sound theory in great detail. I was rereading the introduction to the book the other night and one particular passage jumped out at me:

"Such questions as 'why is there a moon?' or 'why are there societies?' or 'why is there life?' are beyond the capacity of science. These 'why does it exist?' questions are more of a religious or philosophical nature and cannot be answered by science." 

I completely disagree with Dr. Reynolds on this particular point. Dr. Reynolds has certainly had a long and prestigious career and I mean him no personal disrespect, but I still maintain that "why"?" is certainly a scientific question, and that is a point that I will vociferously advocate.

"Why?" is a question for the ages. It is a question of intense curiosity and one of the first questions that we, as human beings, learn to ask. How many of us as children annoyed our parents with countless iterations of this query? How many of our own children ask us this same inquiry? The pursuit of science is simply a journey of discovery to uncover the truth, whatever it may be. So-called "scientists" who falsify data in order to support politically or commercially biased results do an injustice to science and knowledge everywhere. The first scientists who ever experimented with anything did so to answer a curiosity of some kind that they observed. "Why does this happen?" was a common question that plagues the minds of the curious.

In many respects, philosophically speaking, the question of why is intrinsically linked to the question of how. You could even go so far as to argue that these two questions are really one and the same. For example, if someone were to ask "why is there a moon?" the answer would be "because back during the formation of the solar system a smaller planet collided with the Earth and the remnants of that planet coalesced into the moon that we know today, that's why we have a moon."  Unsurprisingly, the answer to this question is the as the answer to the question "how was the moon formed?". Questions such as "what purpose does the moon serve?" are philosophically pointless; the moon doesn't serve any purpose, it simply exists.

Entire fields of study have been created to answer the question of "Why?". I would argue that, as a scientist, the question of Why is what drives us to seek out that knowledge and understanding. "Why did the dinosaurs become extinct?" was a question that drove paleontologists the world over to keep searching for an answer. This even gave rise to multidisciplinary studies between paleontology and geology. Even questions like "why do societies exist?" can be answered through the study of sociology. By understanding what a society does, it can be argued why the human species and other branches of the animal kingdom have established them.

I believe that Dr. Reynolds' statement should be more accurately phrased as "Science cannot test the question of why, only the question of how.", but to say that the question of "why" is not within the realm of science is a preposterous folly. As it stands now there are several questions that should be within the realm of science to answer, but to which science is unquestionably befuddled; what caused the big bang to occur? We know When it happened. We know where it happened. But we don't know what caused it or how it formed. We're not even sure of the exact processes that lead up to it.

Scientific study is about investigation and understanding; how do things happen? what relationships exist between things? What things influence other things? And, perhaps as the most basic question of all, why do things happen? 

Ethics in Practice

[Johnathan Clayborn]

This isn't the first article that I've written on the topic of Ethics. But this article will take a different approach. First, I would like to start by making the assertion that Ethics and Morals are not interchangeable concepts, not completely. Ethics are concepts that are defined as right and wrong by the societal normalization of acceptable behavior. These are defined in our laws, our professional standards and guidelines, and the infamous Court of Public Opinion. This is wholly separate from the concept of Morals, which are defined as a much more personal belief of right and wrong. This belief typically arises from a number of factors, but the most prominent seem to be familial upbringing and religious influence.

Having established this distinction it should be somewhat obvious that cultures, countries, and people will not always align in these areas. sometimes two groups will have the same basic morals, but will be ethically unbalanced. For example, it may be morally unjust in both countries to kill someone. However, one country may believe that lying your way to the top is perfectly ethical behavior whereas the other country frowns upon it. Or, the two groups could be ethically aligned, but morally diametrically opposed. For example, in both countries dishonesty is severly frowned upon and highly unethical, however, country A is perfectly fine with the practice of infanticide so long as they are honest about it, while country B would never dream of it.

The class that I just started is PSYC 8703 - Ethical Standards in Psychology. I was reading some things on the internet today that got be wondering about the ethics of research, not just in the field of psychology, but research in general. Two years ago I conducted an informal experiment on Wikipedia. I wanted to test the accuracy of the information there (on a general level) and present the null hypothesis that "anyone can change the data", as is so often the case. To test this I set about creating a fake profile. I kept a spreadsheet of each page I visited, what the date and the time was, and exactly what change I made to that page. Some changes were blatantly fake, but others were much more subtle. In 100% of the cases, Wikipedia found my error and corrected it, within 15 minutes. Pretty damned impressive, really. After that they also made sure to revoke my edit access and then they banned my IP address. One of the administrators sent me a scathing email accusing me of vandalizing the wikipedia site. I explained that I was conducting an experiment and had a full record to put everything back the correct way and that my intent wasn't to valdalize anything, but instead to test Wikipedia's integrity. He pointed out that this study had been done already. I pointed out that most scientific studies need to be repeated with similar results before they are wholly accepted as fact, at which point he permanantly banned my fake account after accusing me of being unethical. Maybe he's right, I don't know. For me, it wasn't an unethical action as my intent was pure and in pursuit of truth. How can I independently verify the accuracy and efficacy of the site if they know that I'm breaking it? That test would have to be done blind. Is that unethical? I don't know.

But it's certainly not the first time something like this has been done. I learned today that several MIT grad students had done essentially this very same behavior. They created a software program called SCIGen that will randomly write "scientific research journals". Anyone who has any understanding of computer science can tell that these are obviously fake, but to the untrained eye they may appear legitimate. The three students submitted these fake papers to some of the largest journal publishers in the world, and more than 205 of them were accepted for publication (Slate, 2014). These researchers intentionally submitted ficticious data to a reference source to verify the accuracy of what was being published. They obviously kept records of what they submitted, and they told the publishers about their findings, but not until after the fact. That begs the question, if their test of the journal's Peer Review Process was the same as the general procedure that I employed to test Wikipedia, then would that make it unethical also? By comparison it seems that, at least according to the Wikipedia admin who banned me, yes it would. If not, what's different about their situation as compared to mine?

This tactic is fairly old for testing the accuracy of things. In 1973 a famous psychologist, Dr. David L. Rosenhan began what later became known as the Rosenhan experiment. Dr. Rosenhan and 7 friends and colleagues went to a dozen different psychiatric emergency rooms claiming that they heard voices. They gave fake names and jobs, but otherwise were honest and did not mislead the staff in any other way (Wikipedia). In all cases the testers were admitted for anywhere between 7 and 52 days. Once admitted they began behaving normally again and were all eventually discharged with diagnoses of "schizophrenia, in remission" (Slate, 2006). When he published his findings the article another local teaching hospital reputed his findings saying that his staff would not make those same errors. Rosenhan informed the hospital that they would be testing them and would be sending random imposters to their hospital over the next 3 months. At the end of the period the hospital director jubiantly reported that out of 193 patients they had identified 41 as definite imposters and another 42 as possible imposters. His victory was short-lived when Dr. Rosenhan wrote back saying that, in truth, he sent no imposters at all (Wikipedia). It's been quite some time since this test was conducted, but the realm of psychiatry has advanced since then. There's a well-established, peer-reviewed set of guidelines called the Diagnostic and Statistical Manual (DSM) that outlines the conditions for mental disorders. It was recently revised again and the 5th edition, DSM-5, was just released. The practices of medical insurance billing muddy the efficiacy of this tool somewhat, but it certainly helps in preventing erroneous diagnosis. Some recent experiments have attempted to replicate Rosenhan's experiement with varying degrees of success, but at least one journalist argued that the practice of sending fake patients to a hospital for the purpose of testing their diagnostic efficacy was "ethically dubious" (Slate, 2006).

That really is the crux of the question for today's article; is it ethically dubious to test if a system is really working by knowingly submitting false data to that system to see if it errors out? This is a standard practice in computer programming, but computers aren't people. As a psychology doctoral student, I can say with certainty that people behave different when they are being observed and tested than when they are not. Since the vast majority of times these human-driven systems are operating without direct supervision or observation, it becomes necessary to test them in a manner that replicates their day-to-day operation. This happens all of the time in our daily lives. Think about the old Fire Drills. Did they ever tell you that they were doing them? No. It defeats the purpose of the test if you're ready for it. What about so-called Secret Shoppers than companies employ to test to see if their stores are following proper procedures? They certainly don't tell the stores that they are coming. There are countless examples of this in daily life. If these examples of submitting false data to a system and observing the result are not ethically dubious, then what separates them from the examples above? Where does one draw the line? Is it when there's a monetary value associated to the test? Or when the consent from the test comes from the stakeholders themselves? What about situations where the test subject is being offered as a public service or a public domain? Since all of these are intrinsically the exact same operation, what could lead to some of them being ethically acceptable and others not?  

References:

http://www.slate.com/blogs/future_tense/2014/02/27/how_nonsense_papers_ended_up_in_respected_scientific_journals.html

SCIgen Website: http://pdos.csail.mit.edu/scigen/

http://en.wikipedia.org/wiki/Rosenhan_experiment

http://www.slate.com/articles/health_and_science/medical_examiner/2006/06/take_the_shrink_challenge.html