I didn’t think I’d have time to write a reflection on this week’s readings, as we have a big assignment due in 8011 tonight. As it happens, though, I’ve managed to get the assignment mostly completed and I’ve managed to get through this week’s readings to boot. So, here are my reflections:
MacNealy, Chapters 4 & 5
MacNealy, M. S. (1998). Strategies for Empirical Research in Writing. Longman.
I feel like my feet are touching ground again, after a couple of weeks focused on cultural studies and critical methods. MacNealy offers a very practical overview of basic issues in quantitative research (Chapter 4) and the basics of experimental design (Chapter 5). I hope to refer back to these chapters as I’m constructing my own research proposals–using them as a checklist to prevent some of the mistakes I’m likely to make.
Key points from Chapter 4:
- The chapter focuses mostly on threats to validity, with nods to randomization and hypothesis (and null hypothesis) creation.
- Validity is “the degree to which a specific procedure actually measure what it is intended to measure” (55).
- MacNealy distinguishes internal validity (sources of rival explanations within the sample) from external validity (representativeness of the population).
- MacNealy offers diagnoses and remedies for many common threats to validity, including “history,” “maturation,” “mortality” (a term I don’t like for folks who drop out of a study), testing effects, interaction effects, random sample problems, etc.
Chapter 5 focuses on experimental design, principally on the means of reducing threats to internal validity.
- She steps through the process from selection constructs and variables to study, to designing experiments that address validity concerns, usually with a good example for each.
- She addresses Type I and Type II errors, which I can never keep straight. (Type I = rejecting the null hypothesis when the results are not significant enough to do so. Type II = not rejecting when the results are significant enough to do so.)
- She offers a rudimentary discussion of measures of central tendency (like means and medians) and of measures of significance in categorical variables (chi-square) and continuous variables (t-tests).
The two chapters in Gurak & Lay, Research in Technical Communication, for this week fit into this reading. The Charney chapter discusses experimental and quasi-experimental design. And Murphy’s chapter discusses survey research. Again, I view them mostly as “checklist” material for constructing a study design.
Charney’s “Empiricism is not a Four-Letter Word”
I enjoyed this article as a defense of empirical (and even “objective”) methods against some of the claims against them. One of Charney’s central claims is that “critics of science often conflate methods and ideologies in simplistic ways” (283). Essentially, she argues that the methods esposed by those who criticize scientific approaches will not be able to find solutions to the problems that the critics raise.
She spends quite a lot of time debunking misconceptions about science among its critics, giving voice to some rebutting perspectives. (I don’t expect she is being “objective” here in the sense that I think she believes in the scientific method…) She challenges the argument that “objective methods” are sexist. She spends quite a lot of time discussion the equation by critics of science of scientific indeterminacy and power politics. First, she notes that critics are not satisfied with the messy ways that science does its work. Then she addresses the claim that the scientific method is bound to entrenched power structures.
She continues with comprehensive treatment of motivations for objectivity, how scientific methods are socially constructed, relations between researchers and study participants, and objectivity as a means of creating collective authority, rather than relying on personal authority.
Eaton et al. “Editing in the Workplace”
This study used an online survey. It is interesting more for its methods than its results. Key questions arose in terms of sample selection (where the authors acknowledged that they would not be able to generalize) and in the distribution of responses (which could not be studied using means relying on a normal distribution, because the responses were not normally distributed).
In class, Aaron asked about ethical and validity concerns in surveying students. LKB counsels using the IRB form as a starting point, emphasizing student choices.
LKB also recommends Campbell and Stanley 1963 Experimental and Quasi-Experimental Designs for Research. She calls it the gold-standard primary source for exp and quasi-exp research.
As regards the second bullet from the first key points list: there are many (many!) types of validity in empirical science, and I think that the quote is true only of some of them (construct validity, if I’m not mistaking). Am I wrong?
Somewhat unrelated, how do you add small caps into posts? 🙂
@Argumentics:
As for your first comment, I don’t know. I’ll think about it though in the context of some upcoming readings I have do to. Do you have an authority in mind that would support what you’ve said? (I’d love to just look it up, but I’m absolutely swamped with readings… Fate of the grad student, I guess.)
On the small caps issue, it’s just part of the WordPress template I use, called Oulipo.
-Brian
You understand the sources well. I liked your comment that the Eaton article was more interesting for its method than the findings! I agree that the Eaton article is valuable in making survey methods visible. Regarding the validity question about different types of validity–yes, that’s true–but generally there are two categories of validity: internal and external. Grasping those two categories is helpful for understanding the rest.