10.29 Re-cap on short analysis projects, interview protocols, and intro to oral history

Note on blogs and short analysis drafts: Yes, it is the middle of the semester and I am a little overwhelmed just like everyone else.  I had some other work that came due the same time as advisement, and I just wasn't able to get finished.  I will be giving you feedback on Blogs 7 & 9, together, since they both pertain to you interview protocol.  It may be a couple of days before I get my head above water, but know that I have good intentions and I am working on getting back to you.  I hope to have everything back in order by the weekend.   Thanks for your patience - and if you need some immediate help (or to get the hold off your account so you can register) stop by my office.

If you have sent me some drafty writing for the Short Analysis project, I will get back with some feedback, hopefully by the end of the weekend.

 WHAT WE DID IN CLASS
I started by giving a general response to what I was seeing on your blogs.

Short analysis project
Setting up the short analysis project:  In general, you are doing a great job of attending to the language features (and that is the hard part!), and you are getting the idea about using the language moves as evidence to answer a larger question.  Also, you are doing a good job of asking questions that the data can answer. Very good!   All interview data can ever really indicate is how the speakers act/feel/think.  The reason that is important to writing studies is because those indications suggest larger patterns about cultural discourses, Conversations, forms and so on.  Writing studies are interested in these larger formations because they shape our writing, talk and behavior (whether we are aware of them or not) => and to REALLY understand communication, it is important to think about the role played by the cultural Discourses, Conversations and identities that are out there = ready-made, for us to step into.

So what you needed to work on was identifying how the features you were noticing were shaped by larger cultural identities, Conversations, and ways of thinking and being associated with particular Discourse communities. We talked about what kinds of formations were associated with each of the two transcripts you might choose for your short analysis project (listed below, sort of approximating what we said).
Chat transcript

• Conversations about "danger" "sex" "good parents" and the Internet
• Identities for "children" as vulnerable, impressionable, and in need of protection (this is a very American conception of childhood - other cultures think of children as resilient, capable of processing experience more or less on their own, and able to take the good and the bad)
• Discourses for "rebellious adolescent" and "good sister/daughter" and "vulnerable child" (for A) and as research interviewer and friend for S
• Reflective patterns for talk typical of female talk (sharing stories and reflecting on what they mean)

Adult learner

• Discourses associated with digital technologies and computers (newcomers v experts)
• age/generational Discourses
• Female reflective storytelling-talk
• nontraditional student identity for M, research interviewer identity for S, middle aged women Discourse for both (collaborative, reflective, not coming to closure too quickly, lots of connections to other people and groups)

We also reviewed the assignment sheet.  As you draft your project - check the assignment sheet to make sure you are meeting the expectations.

Interview protocols
I also talked briefly about what I saw on the posts for the interview protocols. You are doing a great job of starting with warm-up questions and moving to open-ended questions.  For the most part, you are also asking lots of "tell me about. . ." "Can you talk a little more about. . . " and "Tell me a story where. . . "  This is the best.
A few suggestions.
Set up a brief statement of your purpose at the beginning of your protocol (as on the sample).  In most cases you will send your protocol to your participants BEFORE you do the interview, so they can think over what they have to say.  The statement of purpose will help them frame their answers.
Use the categories of analysis from your research essay and your research question to decide what kinds of questions to ask in your interview. Categories of analysis are the general groups of ideas/information that you will analyze.  In the commenting essay, the categories for analyzing the comments were length, type, Positive/negative, etc.  Your categories of analysis need to connect to the research article because you are using your research article to help you frame your study.  For example, if your research article states that children learn x, y, and z about literacy from their "play" - you will look at your participants' learning to see whether x,y, and z are there (or not) - and whether they learn other things as well (or not).
Be sure to ask participants to DESCRIBE their experiences before you ask them to ANALYZE/EVALUATE them.  Ask a "tell me about" question before asking a "what was it like" or
"which is better" or "how do you use. . ." question.  Both kinds of answers are useful, but the "tell me about" questions may give you information about how your participant feels/thinks that s/he is not able to fully articulate.

So good work on this.  Before you do your interview: get the signed copy of your consent form and  send me a copy of your interview protocol as an attachment to an email with the subject line: YOUR LAST NAME, Interview Protocol.

Developing a Research Plan.  Somewhere in there, we also talked about planning your research project, and you did some writing to nail down (one more time) the purpose, research question, and process for your research project.  The more writing you do here, the more clear you will be on what you are doing. 

Oral history.
We spent the rest of class talking about Oral History, and doing some thinking about best practices for oral history interviews.
We talked about the kinds of background we would need to think about and the kinds of questions we would ask to collect an oral history on 9/11.
You then had a too short time to have a short conversation (recorded on your phones) about experiences surrounding 9/11. 

For next class:
Blog 10.  Write to the points on the Developing a Research Plan Handout (posted to the right) as discussed in class.  As discussed in class, this needs to be an in-depth representation of what you plan to do and how you plan to do it.

Bring a copy (electronic or otherwise) of the essay you are using for your research project

During the first part of class, you will each give a short presentation on your research plan. You should come prepared to talk about each of the points in your blog post, the purpose of your study, a detailed statement of your research questions, how you will use your research article to frame your study, how you plan to collect and analyze your data, what research methods you plan to use, and what you need to do to get started. 

During the second part of class, we will review the moves you need to make to write a research essay, with particular attention to the literature review, and you will work on drafting/writing which ever section of your essay you are ready to write.

Thanks for the good class tonight, and see you next week.

Due dates for Short Analysis Project

The due-dates for the Short analysis project listed on the course calendar are conflicting, so I am posting the due dates we will use here, so there is no question.

Due Oct 29 (optional): writing for short analysis project if you want feedback.  You may turn in writing for the project as an attachment to the course email, and I will work with you/provide comments.

Week of  Nov 4: optional conferences on short project/research project

Due Nov 12: Final Short Analysis project .

10.22 More discourse analysis and doing interviews

We spent the first part of class reading through the list of language features and what they (might) mean (see previous post), thinking of examples, and deciding what we thought each feature might mean when speakers use it.

Just before this discussion, each of you shared a possible research question for the Gaming transcript, and just after it you spent some time coming up with a question for your short analysis project. For a question for this assignment:

 1. it needs to be a question that is important to writing studies

2.  it needs to be "answerable" in terms of information in the transcript

With respect to the first feature, we talked over some of the kinds of questions that writing studies researchers might be interested in.  This list might include questions about:

• how interviews work
• features of different Discourse communities and how they influence the way we talk about particular topics
• how identities (Discourses) shape the communication dynamics in conversations (and interviews)
• how Discourse affects literacy learning
• attitudes about literacy, writing, school and just about anything else that will influence literacy learning
• how Discourses/identities/past literacy experiences affect student attitudes toward writing/literacy
• connections between literacy learning and other attitudes, activities, and identities

As you can see, this is a pretty broad list. 

 

Some observations we made about how to phrase a "good" research question for this project were that the questions would  need to be asked in terms of what the transcript can show. For example, the gaming transcript cannot support an answer to questions about whether or not gaming experience helps students learn software for school=> it can only show what the participants in this interview say about their perspective on that question. 

The rest of class was spent talking about how to design interviews.  We looked at the handout on doing interviews, and we looked at a sample interview protocol. 

We noted the following.

• The beginning of the interview is a place for participants to warm up.  Ask questions that are easy to answer.
•  An open-ended, general question about your focus at the end of the introductory material can give your participant a chance to talk about your topic in his/her own language, before you begin directing the conversation to specific features of your question.
• It is a good idea to organize your interview - either chronologically, by topic, or both.
• You will probably want to ask the same questions - related to different aspects of your topic - over and over again to give your participant multiple opportunities to call up information assosciated with your focus.
• Ask lots of descriptive, tell me about, do you have a story questions, rather than questions that demand direct information.
• As you come to the end of your interview make sure to give your participant an opportunity to raise anything they want to say that you might not have covered.

For next class:

Read: Best Practices for Oral History

Blog 9: Draft an interview protocol for your research project.  If you are not doing interviews, draft an interview protocol that you could use if you decided to do your project a different way.

Next class we will talk about doing oral histories, which is a particular kind of interview. We will also talk about your assignment sheet for the final project and map out the rest of the term's work.

Thanks for the good class tonight, and see you next week!

Handout on language features and what they (might) mean

language features and what they (might) mean

10.15 Part 2: Discourse, discourse, Primary Discourse, Secondary Discourses, transcripts = and discourse analysis

Note: See the previous post for a summary of presenting the informed consent forms to participants.  This post lists the prompt for Blog 7 - the in-class writing you did on how you would present your project to participants.

We spent the middle part of class talking over James Gee's introduction to Discourse.  These were some of the observations we made during the discussion.

Literacy is connected to Discourse

Little d discourse is language in use

Big D Discourse includes beliefs, ways of thinking = a pattern for performing an identity

Dominant Discourse is the identity/ways of being/language  that is "enforced" by culture

Primary Discourse = values beliefs ways of relating to the world we learn as children

Discourse is not just what you say but how you say it

We learn secondary Discourses from school, media, work peer groups etc

You need to understand Discourse before you can research it

Our interpretations derive from our Primary Discourse

Your own discourse will influence the way you “hear” and “say” things = consciously and unconsciously

There is translation (difference) between languages – and between Discourses

School teaches Dominant Discourse

Transcripts – we hear what we expect

Style and structure are part of Discourse

Analysis of the gaming transcript.
Next we looked at a sample transcript, the Gaming transcript,  and made some observations about what was going on.  We noticed particular patterns in language use; repetitions, presence of particular language "moves" (such as the different ways B minimized his expertise, and Ch's persistence in focusing on connections between software and games= would counting the different ways these two speakers persisted in these moves help justify posing this opposition as a focus for the conversation?); the organization or sequence of the conversation (how statements set up the statements that follow); who spoke the most; who controlled the conversation; who asked questions, and so on.  Here is the marked up transcript we created in class.

We also went over the assignment sheet for the short analysis project.  This was to give you a heads up on the project - where we will be going.  The work we did in class on the Gaming transcript was practice for this project.  You will work on analyzing transcripts for your short analysis project next class.  We will also work on set up/doing interviews (handout distributed in class). 

We ended class by listing some communities/groups which have Discourses. Here is a list.

School

Most sports/games have Discourse communities (football, baseball, soccer, scrabble. . .)

Churches

Nationalities

Ethnicities

Professions (Doctors, lawyers, accountants, teachers, steel workers, farmers. . .)

Politicians – by party & positions within parties

Students

Gamers (online/game systems) by game genre or gaming group 

Music communities

Fan communities, including different fan fiction groups

Interviewers (for the news, for celebrities, for research, for jobs = all different discourses)

So, with all that in mind, this is what to do for next class.

Read: look through transcripts for Data Set 5 (posted to the right).  Review what we did to analyze 5-1 ( the gaming transcript).  Think about which of the other two you would like to use for your Short Analysis project. 

Blog 8: Pose a research question about what is going on in the Gaming transcript.  Do your best to ask a question that will allow you to discuss criteria requested on the Short Analysis Assignment Sheet.   List/point out the evidence from the transcript that you will use to develop your answer.  (One important criterion for a good research questionis that it is a question your data set can answer. That is, there needs to be evidence in the data to support an answer.) 

10.15 Part 1: Informed Consent

We spent the first part of class planning and practicing how to get informed consent from your project participants. 

I handed out:

• a signed (by me) informed consent form
• a signed (by me) A-V consent form
• a debriefing form.

You started a paragraph to cover the specifics of your project so you could talk to your participants about your study,  your processes for subject selection, the purpose of your study, and you data collection methods. 

After watching Tania talk through the process for getting informed consent, you practiced with a partner. 

In addition to covering all the points pertaining to your study, made sure to:

• point out the contact information
• inform participants that they can withdraw with no penalty/consequence at any time
• make clear the steps you will take to preserve confidentiality
• ask if your participant has any questions
• get a copy of the signed forms for you to turn in to me, and give one to the participant

You also looked through the A-V form.  As you present the A-V form, make clear that participants can request that the recording equipment be turned off at any point during the interview, and that the A-V form only pertains to data recording.  Again, draw participants' attention to the contact numbers, and create two sets of forms, on for you to turn in to me, and one for the participant to keep .

At the end of your data collection you will provide participants with a debriefing form.  Thank your participant, explain how their participation was important, point out the contact numbers, and thank them again.

The second part of class focused on Discourse, Dominant Discourse, Primary Discourse, Secondary Discourse, and discourse analysis (our talk about the transcript).

Because that is another topic - I am setting it up as a separate post.

For next class:

Blog 7: Post the material you will use to describe your project when you present your informed consent materials.  Even if you will not actually be using human subjects, and will not need to present consent forms, write up a your project as if you were using participants so you will have training in presenting consent forms.

10.8 Part 2: Developing research questions + what to do for next class (Data Set 5)

NOTE:  Class on 10/15 will meet in CAS 307.

During the second part of class we discussed Data Set 4: 4 papers with teacher comments.  We spent some time discussing the comments: their purpose (why teachers might create them), the form different comments take, and how students might receive different comments.

For the different purposes comments might serve (I did not get my notes for this, so this is probably not exactly what we had on the board), our discussion put forward the following.

Purpose for teachers comments:
to get students to revise
to support students in taking authority over their own work
to provide the teachers perspective on "how student is doing"
to give readerly feedback (say what was understood)
to motivate students to write

You then worked in groups to think about the categories for analysis from Ferris/Martin, the different purposes for giving comments, or some other set of features of comments on student writing => and to come up with research questions that might help teachers/students understand more about how teacher commenting works (what it does, what makes it effective, how it works, and so on).

Groups: (Let me know if I got these right)
Tanaera, Sam
Silvia, Carolina, Jaleel,
Sarina, Julissa, Rachel
Bri, Danielle, Danielle

As discussed when we created questions for the ethnographic notes, a good research question needs to be relevant to the data (that is, there needs to be enough information in the data to answer it); it needs to be "important" (it needs to explore something useful, new, or contested for writing studies), and it needs to pose a research process that is "doable".

The groups came up with the following:

1. How does poor proofreading in comments affect the effectiveness of commenting?

2. How do questions work in comments
3. How do teachers use comments to give the student the option to express his/her own ideas versus demanding that students comply with teacher directions?  What factors would make these different approaches appropriate?
4. Analysis of headnote/endnote how they work versus how sidenotes work
5. How does the amount of text selected for side comments affect student response?
6. How does specificity of comments work in commenting? What does it do &  how does it contribute to (or not) comments effectiveness in achieving their purpose?
7. In head/ending comments, how does the use of examples contribute to effectiveness?
8  How do imperative comments work in getting students to revise
9. What are the effects of placing the head note at the beginning versus the end?
10.  What does it mean that there are not grammar comments?

Each group chose a question, but I didn't get the timing right for this class, and we did not have a clear discussion of what to do for the Blog before some of you left.  I thought it over and I don't really think we had a clear enough discussion for you to follow on the post I set up in the last chaotic minutes of class - so I am assigning completely different post (see below).  

Next week we will talk some more about how to set up and follow through with the analysis of a research question.  You did a great job last week on the ethnographic notes, and I don't want to confuse you. .

For next week:
Blog 6: Set up the analysis to answer the question chosen by your group by doing the following 
1. Identify your research question & discuss what features of the comments you will need to characterize in order to answer it.
2. State the categories for your analysis (for example, what features will you use to determine "effectiveness"? or to characterize headnotes versus sidenotes?  or to characterize the different "selections" made by side comments?
3. If you chose a question about "effectiveness" = describe the criteria/process for determining the comment's effectiveness which purpose(s) (from class discussion) the comments are effective for.

Read: Gee, Introduction to Discourse Analysis
In this reading, pay attention to the way he talks about Discourse communities, identity, and his discussion of how to analyze talk (transcripts).

Then, for next class, look through Data Set 5 (posted below ).

Chat room transcript

 

Adult learner transcript

 

Gaming literacy transcript

 

Come to class with some ideas about questions you might ask to use this data set to study Discourse communities, how individuals represent themselves as belonging to those communities (or not), and what discourse analysis can show about speaker's relationships to the material they are talking about.

 

 

 

10.8 Part 1: Form and content for research papers,

We spent the first part of class discussing Martin's essay on the rhetoric of teachers comments on student writing.  The notes from the board are pasted below.

rhetoric: the arts writers & speakers use to get their audience to "hear" them

Overall organization of this essay

Introduction
Literature review
Current study

                Methods

                Results
Discussion
Conclusions

In our discussion of the overall analysis, we noted that some authors merge some of these sections (for instance, the lit review + the introduction, or the results and discussion, or the discussion & the conclusions), but that this list represents the general order for the "moves" research writers make when presenting their work.
 Categories of analysis for Martin’s paper
Length
Type of comment
Use of hedges
Text specificity

As we discussed these categories for Martin's analysis, we noted that the paper might have been stronger if these categories were defined more directly, and earlier in the essay.  We figured out what she meant by these categories by looking at the appendix, and the discussion of teacher tendencies in the literature review.

 

What should a literature review do
1. tell what has been found relevant to your question
2. establish your authority/credibility
3. evidence of the importance for your research

What to include in the methods section

Identify your research subjects
Describe the context for the study
Describe how you collected your data
Present your research instrument (survey, interview protocol, etc)

Findings for this research
Students did not revise in response to positive comments
Did correct grammar comments
Type was the strongest predictor (more so than features) for student revision

(4 most likely to respond to = list on last page)

This essay also noted a long list of similarities and differences between the present study's findings and Ferris' work.  We observed that the presentation of findings was hard to follow, and made the following observations about different ways to present them.

• don't try to present everything, just points relevant to the focus
• use visuals
• make better use of categories/other organizing ideas to present data (more, larger groupings)

 Suggestions for strengthening this essay

Organization= present definitions/focus before discussing
Create a focused summary of the literature review=> only summarize/discuss points relevant to your focus
Work on the data presentation: don’t present ALL the results, only results relevant to the research question; use visuals; organize data by points about what the data mean

Throughout this discussion, you did some talking/thinking about how to organize/present your research essay.

 

 

10.1 Developing a research plan - and surveys!

We started class with some writing to develop your research projects. You wrote some drafty writing to answer the 5 prompts in this post. (See previous post: Developing a Research Plan).

We then moved on to a discussion of surveys.   We started with a general discussion about what surveys can and can't do, some of the problems associated with surveys, and different kinds of surveys and survey questions.

what surveys can and can't do
can do:
collect a large body of data fairly cheaply
provide statistical answers to clearly framed questions
because surveys rely on self-reports, it is difficult to get accurate information about what the respondents don't know, don't understand, or have misinformation about = all you can find out is what respondents think

can't do
unpack the reasoning/particular interpretations that underlie the ways respondent complete surveys


some of the problems associated with surveys
poor return rates
people do not always answer them honestly
often, even when respondents have good intentions, there can be miscommunications
different cultures, as well as individuals with different experiences rate/respond to survey questions differently

survey formats
fill in the blank/write a response
multiple choice
rank on a scale

We then spent some time looking through the data collected for the MA in English studies program.  As pointed out on the earlier post on surveys, surveys need a clear purpose and objectives. This survey's purpose was to assess the program's effectiveness,
The survey's specific objectives were to assesss whether the program provided students with:

• courses and experiences that would support their career choices/personal aspirations
• a sense of increased confidence and proficiency with respect to writing practices they will need for their future
• strong background in theory and practice that defines writing studies.We used the survey designed to assess the MA in Writing Studies Program as a sample. 

Among other things, after reading through the survey you pointed out that it was confusing to use the same language on the pre- and post- surveys, since the students had different relationships to the program, and the questions did not make sense for both sets of students. 

Evaluating model research studies
During the second part of class we evaluated the study on texting.  We noticed that the beginning of the essay defines the problem.  The author's next move was to present the "pro" and "con" research on the effects of texting on writing.  The following section set up her methods: her overall approach to collecting and analyzing data.  The final section presented findings.

Our evaluation of her essay's form: In our discussion we pointed out that the paper would have been stronger with a more detailed presentation of her data (results) followed by a conclusion which discusses her answer to her research question in light of her data.

The study used three approaches for data collection: interviews with teachers, interviews with students, and analysis of written texts by students. 

Content of surveys for students and teachers

Students
How long have you been texting?

How often do you text?

 Do you notice yourself using textspeak in your texting/formal writing?

(suggestion = forward some texts)

 What types of abbreviations/how often

What do they think should be done to prevent textspeak in writing?

 

Teachers
Are you a texter?

Do you believe texting is impairing student writing?

Do teachers notice the effects of texting in student writing?

Is texting positive or negative?

What do you think should be done to keep students from using textspeak in writing?

 

Analysis of student writing
essays were analyzed for common acronyms used in texting

**essays were not analyzed for any of the positive effects of texting noted by researchers

 

Class assessment of this study:

We raised questions about the subject selection (is choosing friends the best idea?); the quality of the surveys (some questions were leading, needed more questions); the surveys should have been included in the study as appendices; the student writing might have been analyzed for more than just the presence of acronyms. 

Good work on this!

For next week:

Read: Rhetoric of Teacher Comments on Student Writing  Robin Martin

Blog 5: Develop your research plan.  Write into each of the 5 prompts.  At the end of your plan: list what you need to do to get started on your project;  list of what you need from me, let me know if you want another conference.

Good class & see you next week.

 

 

Developing a research plan

1. Identify your focus.  This is a general statement of what you are interested in

2. Identify your research question:  What in particular do you want to find out?  State your question in as specific terms as you can: the age/identity of your subjects, the location of your study, the particular activities/features you will focus on. 

Your research question is really a group of related questions, stated in specific terms, where you narrow in on what in particular you want to learn about in your study.

3. Who has studied this question and what do they say.  For this prompt - mention any article that you have read where researchers have explored answers to your question.  If you can't find any articles - tell me something about what you searched for and what you found (even if it wasn't right).

4.What do I need to find out to answer this question?  This prompt is to help you clarify and deepen your research question.

5.  What do I need to do to gather information that will answer my question?
 This prompt is to help you think about how to design your study.  Who will you work with?  Where?  what will you do together?

Surveys

Process for thinking about scope and format
1. Write a statement of purpose
2. Write specific objectives that relate to that purpose
3.Decide if you would conduct your survey by mail, phone, internet, or personal interviews
4.Justify your choice of collection methods

Process/guidelines for designing the survey instrument
Outline the various topics you want to include in the survey
Consider the purpose of each question you intend to ask
Decide on the structure (closed=multiple choice, checkbox  v open ended formats=respondents use their own language)
Avoid psychologically sensitive questions (questions that may trigger defenses or promote conflict in respondent)  For the survey of new students/graduates from the MA program, a sensitive question might be "Do you feel you are successful in your chosen profession?"
As you compose questions, review them for clarity
Short questions are preferable to long ones
Positive wording is preferable to negative wording
Ask only one idea with each question
Avoid leading question
Emphaize main concepts with italics or underlines.

Testing your survey
use a group similar to the group you will study
ask the pilot group for their help. Encourage them to note what is confusing, what is left out, and what doesn't need to be there
administer the survey in the same way you will use it in your study
collect your participants comments either through post survey conversations, or comments written on the survey itself
revise your survey in light of the testing