Innovation in techniques for teacher commentary on ESL writers ’ drafts

Recent technological advances make computer and Internet tools an attractive alternative to traditional written teacher commentary on students’ academic writing assignments. This presentation will discuss how one such tool was used for oral teacher commentary on the first draft paragraphs of intermediate level English learners’ (B1 in the Common European Framework of Reference for Languages) texts. Analyses of texts from treatment and control groups will show the commentary students received on their first draft, the changes they made to their first draft as reflected in their second draft, and the students’ attitudes towards the tool on each of three writing assignments collected at the beginning, in the middle and at the end of the term. The presenters will conclude by drawing comparisons between the video-based teacher commentary and recent work on written teacher commentary to discuss potential strengths and weaknesses of the technique illustrated in the study.


Introduction
Much of the published discussion of teacher commentary on student writing has been around the value and type of teacher commentary.Despite a proliferation of studies, however, researchers have yet to identify the best ways of commenting on English as a Subsequent Language (ESL, including English as a Second or a Foreign Language contexts) writers' work to achieve the intended revisions and, ultimately, improved writing quality. 1 Although the literature reflects some discussion on whether teacher commentary is even desirable, increasingly, research shows that teacher commentary is beneficial to student writing (Morra and Asís, 2009).More recent studies also show that teachers need relevant and repeated training when learning how to prioritize and formulate their comments to achieve the desired effects (e.g.McGarrell, 2010).Little has been written about the medium used for commentary.Interaction with writing teachers suggests that many of them still comment using handwriting but, increasingly, teachers are turning to electronic commentary such as 'track changes' in MS Word.While impressionistic comments from students suggest that ESL students, especially those from countries where English is a foreign language, find it difficult to understand the intentions of teacher commentary, deciphering teachers' handwriting presents an additional obstacle for many of them.In addition, L2 acquisition research favours multiple sources of input as a means to address different learning styles and preferences (Dixon, Zhao, Shin, Wu, Su, Burgess-Brigham, Unal Gezer, and Snow, 2012;Reid, 1987), as well as a means of increasing the amount of input learners receive, especially in a foreign language environment, where opportunities for input may be limited.The exploration of alternatives or supplements to writing teacher commentary thus seems worthwhile.One such potentially valuable tool for writing teachers may be in the form of video recordings if both teachers and students consider the medium acceptable and beneficial.

Background
Teacher commentary followed by revision of an evolving text is expected by both teachers and their students, especially in writing courses that adopt a process approach to writing, and forms an important part of ESL writing classes.Revisions based on teacher commentary have been shown to improve learners' texts (e.g.Conrad and Goldstein, 1999;Patthey-Chavez and Ferris, 1997) and control of basic grammar (e.g.Bitchener, 2008).However, their value has also been questioned and despite numerous studies there is no agreement on what kind of commentary is most appropriate.The delivery of teacher commentary in face-to-face classrooms has typically been in handwriting on hard copies or inserted into electronic texts, sometimes through short teacher-student conferences.Interactions with classroom teachers confirm that teachers typically deal with stacks of papers tucked under their arm or into their briefcase, although increasingly they turn to inserting their comments into electronic versions of their students' text.
The widespread use of and interest in technology among student populations suggests that aspects of technology that motivate additional language learning and learners' engagement with the language to be learned should be incorporated where possible.Many learners view the use of technology positively and are motivated by the inclusion of Information Technology tools as part of classroom learning.A recent review of technological tools facilitating learners' autonomous learning (Warschauer and Liaw, 2011) confirms both the diverse range of technological tools learners access and the appeal these tools have for the learners.Some of these tools are intended specifically for the development of L2 writing skills (Warschauer, 2010), but no studies seem to have explored screencasts for teacher commentary on ESL learners' texts. 2creencasts are video recordings of what occurs on the computer screen paired with audio.They have been used widely for demonstration purposes in commercial and educational contexts, especially in science education (see Thompson and Lee, 2012 for examples), and are considered easy to use and share.The tool allows users to capture an image or text on their screen.With the microphone on, they can then provide up to five minutes of oral commentary about the text on the screen.Once completed, the screencast generates a link that the teacher communicates to the student who can then access the link through a web browser anywhere he/she has access to a computer.The software used for the purposes of this study is JING (TechSmith, 1995(TechSmith, -2012)), available to anyone who has access to the Internet.For the purposes of this study, the free version was used.It allows for video recordings, i.e. a screencast, of up to five minutes each, saving the ensuing link to one's computer, then sharing or posting the link.A more powerful version that allows longer recordings and offers additional features is available for a fee.
The use of screencasts to enhance written teacher commentary through video recorded audio commentary that provides conversational elaborations of the codes or similar marks on the written page would seem particularly appropriate in light of writing students' difficulties in deciphering teacher commentary.Students' difficulties in deciphering teacher commentary encouraged Morra and Asís (2009) to introduce audio recordings of teacher commentary to facilitate comprehension of the comments.A recent quote in Thompson and Lee (2012, p. 1) from a student in a freshman composition class -"I can't tell you how many times I've gotten a paper back with further underlines and marks that I can't figure out the meaning of" -is likely echoed by many students in different writing contexts.The addition of screencasts to written teacher commentary would thus serve to provide auditory input in addition to the written, visual input.In addition, screencasts give learners an additional opportunity to listen to the teacher as often as they like, a feature that was both valued and helpful for language learners in Turel's (2011) study of repetitious exposure to multimedia listening software.Reading and listening to teacher commentary may support comprehension of teacher intentions in the commentary provided.The inclusion of screencasts as part of teacher commentary offers learners both written and oral input.The use of two modalities thus increases opportunities for input, which has been shown as essential for language development (see MacKey and Polio, 2009, for different perspectives on the role of input) and for noticing the gap (Schmidt, 1990) that is hypothesized to precede re-analysis and subsequent integration of forms available in the input.Input opportunities are further enhanced as screencasts are readily available on any device that can access Internet, allowing students to listen to their teacher's commentary anywhere and as often as they wish to do so.Given the appeal of electronic connectedness, learners may be motivated to take advantage of such additional listening/learning opportunities.
However, given the lack of information available on the use of screencasts for teacher commentary on student texts, an initial question is whether teachers and learners would be willing to accept the tool and find it worthwhile.One question that needs to be considered is whether a new technological tool would be easy to use and access for both teachers and learners.Another question is whether the tool would require additional time for busy teachers and learners.Thus, an exploratory study was carried out to trial an Internet-based tool for screencasts as a means for writing teachers to give their learners written commentary and to engage learners with this commentary to motivate revisions of their draft compositions.Students' draft compositions, teacher commentary and students' revised drafts as well as pre-and post-questionnaires were examined to determine the logistical and perceptual impact of screencasts on student writers and their teachers.

Setting
The study took place in the ESL program of a large Colombian university in an urban area where English was rarely used outside of English classes.Undergraduate students from different disciplines are required to take an English language course to develop their academic English language skills.The program has multiple sections of students at different levels of ability and provides a curriculum that teachers adhere to for each ability level.For the purposes of the study, the regular curriculum was followed; all writing assignments, topics and materials were part of this curriculum.Teachers were invited to participate; their willingness to do so and to allow for their comments on three student drafts to be included in the study determined which classes participated in the larger study.The Treatment Group (TG) and Control Group (CG) with the largest number of participants who completed all the texts and questionnaires are included in the following descriptions.

Participants
The participants in this study were a fairly homogenous group of learners in terms of native language (L1), language learning and educational background and academic goals.According to tests administered by the administration, all the participants had tested into level B1 in the Common European Framework of Reference for Languages (Council of Europe, 2001).They were native speakers of Colombian Spanish and none had lived in an English language country.The majority of participants had completed between 9-11 years of ESL instruction in the school system (min.3 years, max.14 years, av.9.5 years for Treatment Group; min.3 years, max.14 years, av. 10 years in Control Group).They were between 17-21 years old (TG av.21, CG av.20) and registered in a variety of disciplines.Nineteen participants (11 female, 8 male) in the TG and 17 participants (12 female, 5 male) in the CG completed all the drafts, revisions and questionnaires that were included in the following.

Procedures
One of the two classes was designated the Control Group (CG).This group received in-text or marginal teacher commentary with "track changes" in MS Word enabled in their electronic first draft texts.A second class designated the Treatment Group (TG) received written electronic teacher commentary in the same manner.In addition, TG students also received the teacher commentary as audio feedback in the form of a screencast on each draft text.Teacher commentary was guided by a chart with error codes for commentary set by the program administration (Appendix A) and focused primarily on morphosyntactic concerns.
At the beginning of the study, its purpose was explained to the students.The students were told that all the writing tasks were part of regular course work but that they had the option to release first and revised drafts of three different compositions they were going to write throughout the course for inclusion in the study.Those who agreed to do so were also asked to complete short pre-and post-questionnaires that collected biographical information and attitudes towards technology.Student participation was close to 100% but several students were absent during one or the other in-class writing session when draft texts were generated.Each of the three texts consisted of a one-paragraph narrative or descriptive composition on a pre-determined topic that the students had encountered in class through readings, discussion and pre-writing activities.The students were given time in class to compose their first drafts of a suggested length of approximately 200 words; revisions were typically completed as home work.
The two participating teachers were non-native but fluent speakers of English with extensive experience teaching ESL in Colombia.They were experienced users of various technologies.They were considered themselves as 'very comfortable' with technology; were enthusiastic about participating in the study.The teachers had been trained and were used to using the error codes chart provided by the administration but indicated that they sometimes also comment on more organizational or content issues.

Data base and analysis
All the first and revised draft compositions were available in electronic format as was the teacher commentary.The teacher commentary for each draft was coded according to its type and tone (see Appendix B for a listing).The teacher comments on the students' first drafts were then compared to the corresponding revised drafts to determine what revisions students had made in relation to the teacher commentary.These revisions were also coded following the categories listed in Appendix B. The written teacher commentary was compared to the screencasts to determine potential additions.As the screencasts typically consisted of contextualised versions of the written comments, i.e. delivered in full sentences and accompanied by hedges, pauses and similar features of aural language, but did not include additional points, the screencasts were not coded separately.The biographical questionnaire answers were compiled, quantitative responses tallied and qualitative responses organized into themes.It should be noted that the few qualitative questions included in the questionnaire were typically answered in a short phrase or left blank.

Findings
The question of whether or not video commentary was a promising alternative to more traditional written teacher commentary was explored through analyses of teacher commentary and student revisions, student questionnaires and informal teacher feedback.The study sought to determine the technical implications as explored through teacher and student reactions to the use of screencasts and through a comparison of the quantity and quality of student revisions in response to teacher commentary.The findings described in the following show that although the two student groups generated comparable texts in terms of length, the TG made more revisions compared to the CG.Both teachers and students were enthusiastic about the use of electronic commentary and screencasts.No technical obstacles were identified that would impede the commenting and revision processes.

Texts
Draft and revised electronic copies of three different texts, with corresponding teacher commentary, students in the course were required to complete were analysed.Analyses show that students in the TG and CG wrote texts of very similar length for drafts and revised drafts.Table 1 shows that the average length for the three compositions from each group ranged from 176.5 words for the first draft of composition 1 (TG) to 228.3 words for the first draft of composition 2 (TG).The length of texts did not vary greatly from one composition to the next or between draft and revised version within each group.Similarly, the text lengths generated by the participants in the TG and the CG were comparable, as shown in Table 1.
Although text length ranged from 102 to 712 words on first drafts and 191-221 words for second drafts, the vast majority of texts were close to the average number of words.Averages for the two groups were similar and no significant statistical differences were found between draft and revised version in either the TG or the CG group.In addition, comparison of draft and revision lengths between CG and TG showed no significant differences.The similarity in text length further supports the similarity of ability levels among the students in the TG and CG, identified as B1 by the administrative testing procedures.

Revisions
Analyses of revisions reflect consistently higher numbers of overall revisions in the TG texts compared to the CG texts, as shown in Table 2. CG participants made no revisions on the level of organization.A clearer picture emerges when the revisions made by students in the two groups are compared to the number of comments they received to determine whether the students in the TG received more comments, thus encouraged more revisions.Participants in the CG received fewer comments overall.The comments they received focused on grammar and mechanics with some vocabulary-related comments.The overall number of comments from each of the two teachers is, however, comparable, as shown in Table 3: Comparison of type of teacher commentary for each composition for TG and CG fered to students in the two groups is similar, the types of comments the two teachers made differs.Students in the TG received noticeably more comments on organization and vocabulary compared to students in the CG. Figure 1 highlights this difference across the three different texts each group produced.The teacher of the TG offered considerably more comments on organization and vocabulary compared to the CG teacher.It should be noted that while both teachers offered comments on organization and vocabulary, these were not part of the error codes provided by the program administration.
The next question investigated was how many revisions students made on their second drafts.Analyses of the students' revisions shows that only 2 revisions were made that cannot be related to teacher commentary.The revisions that were made are less numerous than the teacher comments received, especially for the CG, and the revisions tend to be limited to specific comment types.Tables 4 and 5 summarize the overall findings.
Comparison of the revisions made overall by the two groups show that students in the TG group revised just over 50% of teacher comments while students in the CG group revised 38% of teacher commentary received.In addition, teacher comments in both the TG and the CG were most frequently ignored when they revolved around organization and vocabulary.Students in the TG group addressed almost 86% of all teacher commentary relevant to grammar.By comparison, students in the CG revised 38% of all grammar comments  Analyses examined both type and tone of teacher commentary.Both teachers produced primarily statements and orders for their comments.Questions and requests were used for less than 1% of all teacher comments, while statements and orders occurred with similar frequency.An analysis of the relationship between the tone of teacher commentary and the commentary addressed by students suggests that the tone of teacher commentary did not determine whether students revised or ignored a comment.
In addition to quantitative analyses of student texts and teacher commentary, informal comments from the two teachers and short student questionnaires were also analysed to explore the feasibility of using screencasts for teacher commentary on student writing.The following provides a short report of these sources of data.

Teacher comments about technology
The two teachers who participated in the study did so because they were willing to allow their commentary to be used in the study, which suggests that they had a positive attitude towards the course and the technology used in the course at the beginning of the study.The teacher who used the screencasts, JC, was familiar with the technology and enthusiastic about its use and potential.The CG teacher was also positive about technology and would have been equally enthusiastic about being assigned to the TG rather than the CG.Informal requests to the teachers for comments on their experiences in terms of practicality, time involvement and convenience, elicited positive responses and thoughtful suggestions.JC found the screencasts convenient and easy to use.He was enthusiastic about the opportunity to elaborate orally on his written, often cryptic commentary, in full sentences and claimed that screencasts themselves did not require additional time for commenting.However, screencasts encouraged him to offer more or more detailed comments compared to what he would have offered if limited to written comments.This, combined with the need to store screencasts as JING files, indicate that the teacher who used screencasts took a little longer overall.JC, the teacher who used screencasts, seemed pleased to invest slightly more time in his screencast comments as he felt the practice more satisfactory compared to providing written comments only.The TG teacher argued convincingly that commenting on the student texts through screencasts did not take longer than more traditional comments.JC's practice was to read each student text, then insert the written codes while generating the oral commentary.A potential problem JC noticed was that teachers using screencasts such as offered by JING need to be in a quiet environment and speak directly into the microphone to ensure audible recordings.Neither the teacher nor the TG students reported any technical difficulties with the screencasts.
The teacher whose comments were limited to "track changes" in MS Word, AM, reported that commenting electronically was efficient and facilitated keeping the students' drafts on file for subsequent comparisons with the revised versions for final evaluation.Such record keeping was not readily available with the previously used handwritten comments on hard copies of student texts as the hard copies were returned to the students.AM cautioned that teachers need to ensure that they enable track changes or their comments will simply merge with student texts, making them difficult to identify.One difficulty AM encountered was that toward the end of the course, an increasing number of students had heard about the use of "new" technology in another course (i.e. the TG) and wanted the technology used in their class too.

Students' pre-and post-questionnaire responses
Open-ended questions asked TG participants what they liked about JING, why they liked it and how they used the teacher commentary received through JING.TG participants were also invited to indicate what they would change if they could change anything about the use of JING in writing classes and to add any additional comments they considered relevant.Student responses suggest that they appreciated different aspects of screencasts, found them helpful to learning and would like them used more widely.
All 19 students indicated that they liked JING but their responses were limited to short phrases such as illustrated in the following representative quotes: Fifteen (88%) of the 17 TG participants indicated that they liked that they could listen to the teacher commentary as often as they liked; 14 (82%) valued that it was easy to use and that they could see their texts as well as hear the teacher's comments; 13 (76%) appreciated that they could view and listen to the comments wherever they chose to do so.Comments that addressed how the participants used teacher commentary indicate that most of them (15 or 88%) viewed teacher commentary as correction: their comments invariably referred to "errors" or "mistakes" in their work that the teacher commentary enabled them to "correct".Eight (47%) participants expressed that screencasts make it easier for them to understand the meaning of the teacher commentary.Individual participants also indicated that JING was interactive, that it could be used in subjects other than ESL, that it is fast and practical.No comments expressed negative features of either screencasts or participants' experience with screencasts.However, just over half the students (53%) indicated that they would either like to receive more teacher commentary or more grammar explanations in particular.The same number of students also suggested that screencasts be made available in all their courses.
Quantifiable responses are summarized in Table 6 and show that the use of screencasts was perceived favourably by the TG participants.
The participants' responses show that all of them considered screencasts a helpful addition to written teacher commentary on their texts; the vast majority of the students found screencasts easier to understand than written commentary alone, motivational and suitable for their learning preferences.Similar numbers of respondents considered screencasts helpful for their writing development but they were divided on whether screencasts made revisions easy.Just over 76% of the participants indicated that they did not need more time for their revisions by using JING.

Discussion
Findings from the exploratory study suggest that the use of screencasts in addition to written teacher commentary on student drafts was considered favourably by the students and teachers involved.The teachers indicated that the technology in question is easy to use and that oral comments do not impose a burden on teachers' time.Although only one teacher used screencasts and one teacher used 'track changes' in MS Word in this study, the findings confirm previous findings (Nijhuis and Collis, 2003;Sun, Tsai, Finger, Chen and Yeh, 2008) that a positive attitude towards technology on the part of the teachers who use it tends to lead to positive classroom implementation of technology.In many learning environments, the implementation of technological features into the curriculum may necessitate teacher training to enable teacher to expand their range of technical competencies as well as their comfort zones (Nijhuis and Collis, 2003).
Analyses of teacher commentary and student revisions suggest that the addition of screencasts leads students to address more teacher comments in their revisions compared to when they receive written comments only.However, the analyses also indicated that the concept of revisions is interpreted more as editing: the majority of commenting categories (Appendix B) teachers were to use focus on a limited number of grammar issues rather than the expression of clearly organized ideas.Revisions entailed minimal changes, primarily at the morpho-syntactic or mechanical level, as indicated by the teacher commentary or code.Participants did not add, move or otherwise re-organize their texts.Teacher comments that addressed organizational or content matters were ignored in all but a handful of student revisions.
The absence of significant differences between draft and revised version lengths suggests that the participants in the two groups were at highly comparable levels of ability.Alternately, it may suggest that the participants had a limited understanding of drafting and revision purposes, a suggestion that is reinforced by the primarily surface revisions that occurred.The findings presented reinforce the notion that student writers need training in how to revise based on commentary they receive (McGarrell, 2012) and that the use of new technologies does not replace sound pedagogy (Peachey, 2012).
The question of what should be the objective of a writing class needs to be determined at the institutional level prior to the identification of suitable pedagogical tools.In contexts where English is a foreign language, teachers and students often focus on correcting mistakes, resulting in a narrow definition of 'writing' as producing error-free text (McGarrell, 2011;Morra and Assís, 2009) rather than expressing ideas clearly in relation to reader needs and topic.The latter would require a more formative technique to teacher commentary as discussed for digital technology in e.g.Denton, Madden, Roberts and Rowe (2008).One interesting consequence of this focus on grammar as opposed to development and expression of ideas is that over half the TG participants requested additional, more detailed teacher explanations on the use of grammar.Despite the fact that participants had, on average, experienced close to 10 years of grammar instruction, they felt that more talk about rules would lead to improved writing ability.
The TG participants liked screencasts and considered them easy to use and a helpful learning opportunity.They appreciated seeing and hearing teacher commentary at the same time, a combination that likely contributed to the fact that 94% of these participants indicated that they would like to see screencasts used more fully and more widely, i.e. they would like their teacher to give additional comments and more extensive grammar explanations in screencasts.They also indicated that they would welcome the availability of teacher commentary in the form of screencasts in all of their classes.
Despite the limitation of the exploratory study described in the above, finding suggest that screencasts as a tool for teacher commentary are viable from a technical and logistical perspective.They are attractive as they are easy to use, readily available.Screencasts increase input in English, provide an additional modality thus accommodates different learning styles.Studies involving additional teachers will determine whether screencasts suit different teaching styles.As the findings suggest, students need to understand the objectives of any writing assignment they are asked to complete within the context of institutional goals.If a multi-draft approach to writing is in place, the purpose of multi-draft writing and the differences between revising and editing students' work needs to be reflected in pedagogical approaches and include training relevant to potential error codes used in commentary.Finally, students need to be shown how to deal with teacher commentary in a manner that develops the purposes of the writing course.Future research exploring the applicability of new technologies might ensure that both learners and teachers have received relevant training to help identify any effects that can be attributed to the technology.Because it can take 3 hours, 5 minutes & 34 seconds to get to London from Boston.

WT
Wrong tense Yesterday I go to the movies.Yesterday I went to the movies.On her journal writing she argues that Joe get ill with cancer.
On her journal writing, she argues that Joe got ill with cancer.

SS
Sentence structure In the afternoon tomorrow he will travel to Israel.
He will travel to Israel tomorrow afternoon.In 1995, in Zipaquirá I live with my aunt.Zipaquirá, in 1995.

WO
Word order She has eyes blue.She has blue eyes.They told their parents where were they last night.
They told their parents where they were last night.
120405F2815: -liked JING; can listen to feedback many times 120405F2801: -it's fast, practical, efficient 120405F2809: -I like 120405F2804: -I like it because is interactive; I can see errors 120405M2805: -I like can listen to teacher and see my mistakes 120405M2816: -[JING] helped me in process of written text

TABLE 4
Totals of teacher comments compared to student revisions for TG

TABLE 5
Totals of teacher comments compared to student revisions for CG