2.21 Analysis of a survey

As an exercise to get some experience writing/designing surveys, you analyzed the Writing Option pre-post survey for Writing Majors.  This survey is (in theory) completed by all students when they BEGIN the writing option major (at the beginning of 2020=> the first course they should tak), and when they complete the Writing Option Major (at the end of 4817).  Its purpose is to give the Department an understanding of:

• what students know, feel, and can do with respect to our student learning outcomes when they enter the program
• what students know, feel, and can do with respect to our student learning outcomes when they complete the program

We started our discussion of the Writing Option Major survey by organizing which SLOs were covered by which questions (see below).

Student Learning Outcome #1: students will produce essays through a series of drafts that include exploratory writing or talk, as well as revisions that include addition, deletion, substitution and rearrangement.

1,5,7,11,12,14,15,

Student Learning Outcome #2: students will identify central ideas/themes of a text through class discussion and writing.

1,3,4,7,8,9,11,13,15,

Student Learning Outcome #3: students will use two or more methodologies from English Studies to develop original research or creative products.

2,4,6,7,

Student Learning Outcome #4: students will demonstrate ability to give a compelling oral presentation.

8, 10,13,

Student Learning Outcome #5: students will connect ideas from classroom assignments to contemporary issues in class discussion and presentations.

3,6,7,8,9,10,13

This distribution seems to indicate that each of the SLOs were covered by questions in the survey - so far so good => and then it got really interesting.  We started by having each group report separately - but soon everyone had so much to say that I just fielded comments.  Your feedback provides the following information + suggestions.

provide a comment box
simplify the scale for responding (yes, no, I don't know0
remove or revise questions that are not relevant, that are too vague or general, or that include language students students won't be sure how to interpret (things like discourse analysis, tetual analysis etc)
instructors will need to provide a clear explanation of what the survey is supposed to do if they expect students to "take it seriously" (provide "real" data)
revise/take out specialized language
do something so that transfer students (or students who take courses out of order) take the survey
ditch the last 4 questions or make them another survey
there should be some questions about personal writing
need to explain that this is "not a text" (this connects to explaining the survey's purpose)

So after I took all this in, (and in talking with Karl) it occurred to me that to do a good job revising this survey, we will need a "panel" of students.  Anyone interested?  We can talk about this in class.

For next class:

Read:  Rhetoric of Teacher Comments on Student Writing  Robin Martin

Blog 10: Reflect on what our discussion of the Writing Option major survey suggests about: 1) the way language works (think about Gee); 2) what kinds of information you can collect with surveys; and 3) what challenges researchers face when desigining quantitative instruments (things that are reduced to numbers => like surveys, "objective" tests and so on).