Caveat: I make mistakes and if the article actually follows the scientific method properly and completely I will gladly amend this post in response to comments that are accepted as appropriate and support that view, etc.
Using the scientific method to provide “evidence” of something is not as simple as one might think. In a recent article on kata the author used what he stated was an “independent and peer reviewed academic research” to put forth his proof that kata are invaluable to learning “real self-defense.”
Before anyone can actually study such a thing they have to find a true, accurate and valid definition of what real self-defense is and that does not appear to have been done. I am getting out of my thought path so lets get back to “What is the scientific method?” Along with that I pose to address, at least superficially, the results of the method that often come as “statistics.”
I quote, “The scientific method is the process by which scientists, collectively and over time, endeavor to construct an accurate (that is, reliable, consistent and non-arbitrary) representation of the world.”
Then I also present that often those who conduct such scientific methods of study or research are hampered by their own biases, i.e. their own personal and cultural belief systems that directly and indirectly affect their perceptions of phenomena but the method is supposed to address this by the use of a standard set of procedures and criteria so as to minimize the adverse effects those influences would have when developing a theory.
I quote, “As a famous scientist once said, "Smart people (like smart lawyers) can come up with very good explanations for mistaken points of view." In summary, the scientific method attempts to minimize the influence of bias or prejudice in the experimenter when testing an hypothesis or a theory.”
Four Steps of Scientific Method:
1. Observation and description of a phenomenon or group of phenomena.
2. Formulation of an hypothesis to explain the phenomena. In physics, the hypothesis often takes the form of a causal mechanism or a mathematical relation.
3. Use of the hypothesis to predict the existence of other phenomena, or to predict quantitatively the results of new observations.
4. Performance of experimental tests of the predictions by several independent experimenters and properly performed experiments.
Now, to remain within the confines of posting brevity I would redirect the reader to the following explanation of the scientific method.
After reading the article, “Introduction to the Scientific Method,” from Professor Frank L. H. Wolfs from the University of Rochester, New York, you may then address the following article that prompted this posting.
When you review the “proof” as addressed by the authors means, i.e. scientific method, I find some glitches that may not truly support the results regardless of whether they are actually valid as a SD and Teaching tool, i.e. kata.
An example: To qualify for a Self-defense phenomena and to find quantitatively results of observations, etc. that would prove this as true you would need to conduct the observation and research under a model that is validated as a real form of self-defense. Self-defense is a convoluted and complex phenomena in the real world and testing this under observation takes first, someone who is experienced, knowledgable and capable of observing a real life SD situation and encounter and that would take observing actual real life violent attacks against humans. This is not possible in large enough quantities to provide enough phenomena to come up with statistics that will quantitatively provides results through those observations.
Second, this also addresses “Several independent experimenters” as well as “Properly;y performed experiments.” Again, to stay true to reality in self-defense the experiment and observations would require at a minimum of kata training and practice under a reality based adrenal stress induced training and practice environment that is not a normal model used in teaching and training for self-defense especially in martial arts where kata dominate as a tool for training and teaching.
Although, in my view, kata are critical teaching and practice tools of martial arts regardless of whether for self-defense, fighting or competitions are truly invaluable. This is not a result of a scientific method of study but rather an opinion based on personal study over thirty-eight years of training and practice. Kata simply do not address all the aspects, fundamentals and applications within the world of self-defense.
Kata focuses solely on the physical applications once a person is in the “Fight.” Kata is only one very small aspect and is often used as a doorway toward more relevant knowledge and experiences that supplement what is necessary to apply self-defense.
I would further ask of the author of the study the following questions:
1. Can they actually address the groups biases and then show how they addressed them to minimize effects on the study?
2. Did they actually perform experimental testing so as to remove the most common error in mistaking their hypothesis for an explanation of that phenomenon?
3. Did they address results to mitigate effects from personal “common sense” and “logical” influences so that they conclude that “no testings was necessary?”
4. Did they accept any and all data or did they exclude data that would or might not support their hypothesis?
5. Did they feel any strong beliefs toward what was tested or exempted from the observational test criteria or did they address any feelings or external pressures or biases that would have provided a specific result?
6. Did they analyze data for the possibility they were seeking to find “something wrong” that would or could be removed to allow a more acceptable result?
7. The data used or collected, did they analyze it in determining its support of expectations, i.e. ones that agree and others that disagree with their expectations, etc. ?
There are far more questions that should be asked in the process of scientific method to test one’s hypotheses but did they cover all the bases? Is there report about their resulting “hypotheses,” or is it about a resulting “model,” or is it about their “theory?” See the article on the scientific method to see the definitions so that the presentation would have more effect and be more acceptable as a study from a scientific method.
Then we must take a look at the “statistics” that would support such a study. As I see the articles results I am not being made aware that they are supported by any developed statistics from the study or research. Statistics are a representation of that body of methods that characterize the kata as a SD tool as inferred thought the observations of a scientific study, those don’t appear to exist so is this a true scientific study method that provided the conclusions kata are invaluable for SD and Teaching?
If they had provided statistics to support the research what standard statistical model and methods did they use? Did they actually have a “representational sample” that qualifies the study for both the scientific model and statistical results according to the appropriate statistical model? Statistical studies are meant to provide a efficient and productive process in scientific research, right?
In the end, it as always brings such studies of the martial arts world into questions simply because the persons conducting the studies have a vested interest in results that support rather than disprove their perceptions, perspectives and beliefs. This seems dysfunctional and counterproductive that are often used by opposition to support their perceptions, perspectives and beliefs that martial arts kata are bullshit. I am not the opposition in this post but rather a “Questioner” of such efforts because I feel strongly that to half-way take a study to just support and vilify my personal beliefs over facts and truth and relevancy that spans all disciplines be they martial arts or sports like football are far more beneficial in the end.
Such posts as this one are meant to question so that others are not inadvertently guided down a wrong path when such things as self-defense can result in loss of life, liberties and personal wealth as well as freedom and so on dictate that if this study were true and complete and comprehensive then society is well served in lieu of being led down the bad path.
Are you thinking more or accepting under the guise of scientific study along with titles meant to support the acceptance of the studies and those who perform them? It is a bit like assuming because one is holding a title, often not validated, as expert/master/sensei, etc.
Caveat: I make mistakes and if the article actually follows the scientific method properly and completely I will gladly amend this post in response to comments that are accepted as appropriate, etc.
Also, I am not an expert on the scientific method or statistics but it don't take much comparison to determine if the study is questionable. Then again, it might at least prompt other research in an effort to validate the study.
No comments:
Post a Comment