Archive | Ethics RSS feed for this section

Gender as a variable in writing studies–presentations and paper accepted

14 Feb

As I noted back in November, I’m presenting “Gender as a variable in writing studies: Ethics and methodology” at the Writing Research Across Borders IV conference in Bogotá, Columbia, on Thursday, February 16. While preparing for WRAB, I wrote an article that has been accepted to appear in the peer-reviewed conference proceedings of the First Workshop on Ethics in Natural Language Processing in conjunction with the 2017 conference of the European Chapter of the Association for Computational Linguistics (EACL 2017), in Valencia, Spain. I’m traveling there to give that paper in early April. The article, titled “Gender as a variable in natural-language processing: Ethical considerations,” draws on similar principles as the WRAB presentation, but focuses on research in natural language processing.

Below are my slides and bibliography for the WRAB talk. Underneath that is the abstract and pre-print version for the EACL paper. (I’ll post the link to the official ACL anthology for the final paper when the proceedings are published.)

Slides for WRAB

Bibliography for WRAB

Abstract and pre-print for EACL paper

Researchers in natural-language processing (NLP) and related fields should at- tend to ethical principles in study design, ascription of categories/variables to study participants, and reporting of findings or results. This article offers an ethical framework for using gender as a variable in NLP studies and proposes four guidelines for researchers and practitioners. The principles outlined here should guide practitioners, researchers, and peer reviewers, and they may be applicable to other social categories, such as race, applied to human beings connected to NLP research.

Advertisements

Readings for 8011 for November 29

29 Nov

This week we’ll have a visit from Drs. Tom Reynolds and Patrick Bruch to talk about research in pedagogy. We had several readings to prepare for this discussion. Here they are:

Herndl, C. G. (2004). Teaching discourse and reproducing culture: a critique of research and pedagogy in professional and non-academic writing. In J. Johnson-Eilola & S. A. Selber (Eds.), Central Works in Technical Communication (illustrated edition., pp. 220-231). Oxford University Press, USA.

Young, I. (1990). Introduction. Justice and the politics of difference. Princeton  N.J.: Princeton University Press.

Connors, R. J., & Lunsford, A. A. (1993). Teachers’ Rhetorical Comments on Student Papers. College Composition and Communication, 44(2), 200-223.

Reynolds, T. J., & Bruch, P. L. (2002). Curriculum and affect: a participatory developmental writing approach. Journal of Developmental Education, 26(2), 12-20.

The Herndl and Young pieces are conceptual ones. Young’s is an intro to her book on justice. Herndl’s, written about the same time (early ‘90s, the cite above is to an anthology), makes an argument for research into the cultural and ideological issues in technical communication.

Iris Young’s intro chapter makes the argument that discussions of justice need to move away from questions of “distributive justice” (who gets what) to discussing questions of “social domination and oppression,” which impact folks who exhibit differences particularly. She claims that she is not offering a theory of justice, that to do so she would have to decontextualize justice (3-5). She presents her view of critical theory (5-7), but also critiques views of some critical theorists (7-8). She outlines what she’ll do in the rest of her book. Some claims she plans to make and support:

  • Chapter 1: She argues for the move from distributive justice (the “primacy of having”) to questions of oppression and domination (the “primacy of doing”).
  • Chapter 2: She defines oppression and puts it in the context of “politics.”
  • Chapter 3: She takes up the (de)politicization of social issues. She offers Pitkin’s and Unger’s definitions of politics.
  • Chapter 4: She argues that “impartiality…denies difference.”
  • Chapter 5: She discusses “some implications of modern society’s denigration of the body.”
  • Chapter 6: She argues for “principles and practices that… identify liberation with social equality that affirms group differences….”

Questions/comments: Why can a theory of justice not account for contextual matters? I’d like to read Chapter 5, as I think the academic mindset also denigrates the body. (The notion that all knowledge is mediated by language denies embodied knowledge and emotional knowledge, which are rarely articulated.)

Herndl’s article also embraces critical theory. He repeats that mantra that “knowledge is socially constructed or legitimized (an important distinction) through language and rhetorical activity” (p. 221), without discussing the important distinction he pointed out. His article discusses the tension in technical communication between reproducing dominant discourse. He argues that TC research should address power and ideology issues in the contexts of TC work; he singles out McCarthy’s study of DSM-III (not the later one) for talking about how DSM III shapes psychiatric discourse without addressing the power and ideology that inform that discourse. Herndl adds his own (purely speculative) thoughts: that the DSM III is motivated by power of insurance companies and legal concerns.

Herndl acknowledges that teachers of technical writing cannot set up their resistance against students’ instrumental intentions: Students believe they want to enter the professional discourses that their courses purport to prepare them for; they do not want the teacher telling them to resist those discourses.

Question/comment: This seems to me to be the central issue. Why can’t we show them all the doors, and let them decide which to walk through?

The Reynolds/Bruch article recounts survey research they did with students in a “developmental writing program” at the University of Minnesota’s (now defunct) General College. They discuss their approach of making “literacy work” of the class, where the students write “full-length essays” but also “study the place of writing in creating academic and other kinds of knowledge” (p. 13). “[A]n integral assumption of the curriculum is that student writing is real writing, that is, it is a part of the larger society and its various social forces and opportunities” (p. 13).

They administered a survey to students, asking them what they thought the focus of their class should be and what they thought the focus of it was. The three categories were “critical thinking,” “academic writing skills,” and “writing process and problem solving.”

This study was interesting in that it married quasi-quantitative methods (there are issues with any effort to generalize these results to students outside the U of M) with critical theory; I presume the journal in question favors educational research (which is often pretty quant or qual/quant oriented).

Question: The questions on the survey instrument appear inconsistent. Why?

The Connors-Lunsford article presents the results of a review of teacher’s “rhetorical comments” on some 3,000 student papers. This descriptive study suffers from a few methodological flaws (representativeness of sample, no accounting for intercoder reliability, etc.), but it provides an interesting picture. In a way, it’s a bit like the Reynolds-Bruch study: Connors and Lunsford examine the quantitative results, describing the “central tendency” of the sample they reviewed. But they also have the coders give an impressionistic review of the papers they looked at; that results in some understanding of the “tails” of the bell curve, including some extreme examples of nice and nasty instructor comments.

Question: Has this study been reproduced? Why not do it now? It would let us see whether their results hold true now (more than 20 years later).

Note to Lee-Ann: I forgot to post this before class today. Only doing it at 5:44p.m. I would not object to being docked for points on that basis. -BNL

Reflections on this week’s readings for 8011

19 Sep

This is my reflective memo for this week’s readings in 8011 Research Methods in S&TC.

Barton & Eggly

Barton, E., & Eggly, S. (2009). Ethical or Unethical Persuasion?: The Rhetoric of Offers to Participate in Clinical Trials. Written Communication, 26(3), 295-319.

One conclusion of this study: That researchers seeking informed consent from cancer patients to take part in clinical trials make “regular and rhetorical use of unethical persuasion” from the perspective of bioethics.

This is a broad conclusion in light of the researchers’ methods: They coded written transcripts of 22 interviews between patients and their oncologists that were recorded with the participants’ permission. Interestingly, nowhere does the study say how many doctors were involved, though we know it’s at least two.

They coded statements by physicians for valence with regard to three issues: Purpose of study, benefits of study, and risks of study. Valence was positive, neutral, or negative for each statement regarding these issues. Based only on the article, I have reservations about the approach to coding. I’m not saying it was bad, I just don’t feel the article provides enough detail to form a conclusion.

But the big problem is with the inference the researchers make: Based on these 22 observations, they conclude, “Physicians regularly present purpose, benefits, and risks with a positive valence.” “The data and findings from this study… clearly indicate that… normative neutrality is not characteristic of the discourse of trial offers.” Stunning, really.

Questions I’d like to ask the authors:

  • Did you seek confirmation of your coding method (its reliability) by providing a code guide to a third party and asking them to code a couple of the observations? The method described seems very unsystematic.
  • Why did you not address the limitations of the study at the points where you were making claims about the inferences drawn from it?
  • The statements reprinted in the article appeared largely devoid of the types of details often include in oral discourse analysis and recording. Were these removed for the article, or were they not present in the transcripts? Wouldn’t an analysis of the pragmatics of the situations have yielded a richer result?
  • How can examination of isolated statements by the doctor, without considering the manner in which they responded to patient comments or questions, determine whether the doctor was positively valencing participation in the trial?

Breuch et al.

Breuch, L. K., Olson, A. M., & Frantz, A. (2002). Considering Ethical Issues in Technical Communication Research. In L. J. Gurak & M. M. Lay (Eds.), Research in Technical Communication: Ablex Publishing.

This useful article provides a sort of checklist of ethical issues when planning a research project, but also provides a rationale for “actively reflect[ing] on ethics while conducting research,” advice that next week’s reading in Marshall and Rossman will echo. The high-level checklist here:

  • Understand the importance of research ethics
  • Review relevant ethics literature
  • Identify ethical issues in advance
  • Meet the requirements of IRBs
  • Don’t stop thinking about ethics once you actually start conducting the research.

Katz

Katz, S. B. (1992). The Ethic of Expediency: Classical Rhetoric, Technology, and the Holocaust. College English, 54(3), 255-275.

I generally like this article, and I agree with Katz that we need to imbue the practice of tech communication with an ethic that considers a variety of values. But I have some reservations about the soundness of Katz’s arguments.

Katz uses an example of technical writing by a Nazi bureaucrat to illustrate a problem he calls the “ethic of expediency” and then argues that this same ethic of expediency undergirds Western “technological capitalism.”

His contentions:

  • Aristotle embraced the ethic of expediency (EoE). Katz does not discuss this very fully and mentions but dismisses contrary views without much elaboration. I’m not confident that he has supported this claim well.
  • The Nazi regime combined the EoE, enshrining the notion that it was ethical and good to seek efficiency, even if solely for the purpose of greater efficiency.
  • His argument constitutes a “critique of the EoE that underlies technical communication and deliberative rhetoric, and by extension writing pedagogy and practice based on it.”

Some questions I have:

  • Katz contends that it “is well known that to perform well in a professional organization, writers must adopt the ethos of that organization.” Am I left wanting to see support for that claim merely because I am new to the field?
  • Katz says in “our own culture,” “science and technology become the basis of a powerful ethical argument for carrying out any program.” He continues: “Technological expediency actually subsumes political expediency and becomes an end in itself. Progress becomes a virtue at any cost.” Why does Katz not offer more concrete examples of this in “our own culture”? The examples of airline safety are rhetorically charged, but he does not actually consider the ethical calculus that underlies them – what is the cost of a single life to the rest of society? How should that cost be ‘allocated’ to those taking the daily risks associated with living?
  • Katz frequently begins paragraphs and sentences with phrases like “obviously” and “it is clear that….” Am I unreasonable for being skeptical about the claims that follow those expressions?
  • It strikes me that there are many examples of modern resistance to “technology at all costs” and “expediency at all costs” – for example, reactions to human cloning, “death panels” (though they would certainly be efficient),

Marshall & Rossman Chapter 2

Chapter 2 of Marshall, C., & Rossman, G. B. (2010). Designing Qualitative Research (Fifth Edition.). Sage Publications, Inc.

In this chapter, Marshall and Rossman identify the genres of qualitative research. They divide them into two broad groupings: “Major genres” and “critical genres.” Though they treat the major genres – ethnography, phenomenology, and sociolinguistics, in an engaging way, they spend a lot more time and energy describing the critical and emerging genres.

On the one hand, I could chalk M&S’s strategy up to the need to explain the critical and emerging genres more because folks are less likely to be familiar with newer genres. On the other hand, I’m concerned that this suggests that the “major genres” are growing less relevant. The latter possibility is troubling for me, as I see myself likely more situated among the more “traditional” methods. Of course, I’m interested in ideas of action and justice, but I’d like to start with more than a hunch before choosing a course of action or an object of my efforts to promote justice.

Miller

Miller, C. R. (1979). A Humanistic Rationale for Technical Writing. College English, 40(6), 610-617.

Miller’s article is a pleasure to read; I can see why it is so frequently cited. She argues that the positivist epistemology of modern science and technology lead to four features of pedagogy in TC that are undesirable.

  1. The subject of TC is not, or is poorly, defined.
  2. Pedagogy in the field emphasizes “form and style at the expense of invention.” She does not like that we teach “recipes.” This illustrates that tension between teaching students what they need to succeed, i.e., an understanding of how to adapt to genres and conventions, and teaching them to see the possibilities of shattering genres and conventions.
  3. Pedagogy emphasizes plain, objective language. I wonder, though, whether this is not the same type of problem as the last.
  4. The “tendency to analyze audiences in terms of ‘levels.’” This is the notion that we must tailor the reading to the reader’s capacity to understand the objective reality. I don’t sense that in the pedagogical experiences I’ve had in the field so far, though. Was this a bigger problem in 1979?

I do have some questions:

  • I’m still struggling with the notion that positivist epistemology precludes a nuanced view of language and technical communication; that it eschews an understanding of audience and rhetorical situation. Based on this article’s claims and those of Berlin (1987, cited in an earlier post) about the subject, I’m inclined to brush up my understanding of the modern take on epistemology. (I can’t think of a thing I’ve read since Locke that would illuminate me on the subject…)
  • I’m uncomfortable with the notion that the “new epistemology” of “cultural anthropology, cognitive psychology, and sociology” “makes human knowledge thoroughly relative and science fundamentally rhetorical.” I recognize the impact of what those fields have contributed in terms of undermining many claims to authority and Truth, but I also feel that there is a middle territory, one where sense perceptions and subjective conceptions can share the epistemological field. (Thus, another reason to review my understanding of epistemology.)
  • C.M offers this: “We can define scientific writing as written communication based within a certain community and undertaken for certain communal reasons. Technical writing occurs within a somewhat different community for somewhat different reasons.” Does this provide a workable disciplinary definition?

Great quote: “If rhetoric is irrelevant to science, technical and scientific writing become just a series of maneuvers for staying out of the way.”