ASSESSING SPEAKING

Assessing Speaking

Among the macro skills of language, it has been widely recognized that speaking, particularly in a second or foreign language, is the most difficult language skill to assess. The various directions and foci in the testing of speaking abilities of learners frequently lack solid grounding on theory and pedagogy and reliable test designs (Pennington, 1999; Celce-Murcia, Brinton, & Goodwin, 1996). This is due, for the most part, to the difficult matching of the testing goals and the appropriate instruments and tasks for assessment. Speaking as a major construct for testing is likewise divided into different criteria with highly diverse applications. The testing of pronunciation (both segmentals and suprasegmentals), spoken grammar, spoken vocabulary, and even sociolinguistic applications of speech all fall into the construct of speaking but largely require discrete test designs and measures. This is a challenge for classroom teachers and researchers of learners’ speaking abilities. Knowing what to test specifically and how to conduct the testing process require applicable theories and valid procedures that map out the direction of the assessment strategy. As a result, drawing upon applied linguistic theories on what is to be tested in speaking becomes essential for teachers and researchers. Sari Luoma’s book attempts to provide these necessary theories, concepts, and issues in the assessment of speaking.

Assessing Speaking is a part of a series of assessment in applied linguistics edited by J. Charles Alderson and Lyle F. Bachman and published by Cambridge University Press. Just like its predecessors, Luoma’s book intends to provide both classroom teachers and language researchers an adequate discussion of the issues in the assessment of learners’ language skills. Luoma’s extensive background in research on speaking in the ESL and EFL settings allows her to contextualize the presentation of topics and tasks that appeal to teachers and researchers of differing levels of competencies in the use of reliable assessment techniques in speaking. The discussion of applied linguistic theories on the nature of speaking (Chapter 2) offers interesting insights to experienced scholars and provides valuable background information to classroom teachers who want to assess their students’ ability to speak in a second or foreign language.

The book is divided into eight chapters that start with a description of the spoken language and provide summaries of applied linguistic perspectives on the assessment of speaking. Current research in test design and preparation of test tasks are reviewed (Chapter 3) and examples of rating scales used by established testing institutions (Chapter 4) are provided. At the same time, the book illustrates practical examples in the development of tasks for speaking tests (Chapters 5 through 7) and the final chapter focuses on the procedures for ensuring the validity and reliability of test designs and the overall conduct of assessment (Chapter 8). The organization of chapters and the coverage of topics and issues in the book are appropriate for its target audience. The choice of sample rating scales and test tasks in speaking are generally justified within the specific goals set forth in Luoma’s introduction and series editors’ preface.

The strength of the book relies on a considerable focus on construct validity in the assessment of speaking and a constant reminder of this concept is repeated in the discussions and chapter summaries. Luoma emphasizes the requisite need for teachers and researchers to first define the kind of speaking they want to test before they develop or adapt test tasks and rating criteria. This consideration often gets lost in classroom assessment procedures, especially when ESL/EFL teachers rely heavily on textbooks and materials that do not match the specific context of the kind of speaking intended to be assessed. For example, the testing of pronunciation is very different from the assessment of spoken grammar or the testing of speaking in meaningful interaction. However, oftentimes, similar rating scales and test tasks are used for these features in the language classrooms. Luoma suggests that because testing situations and participants vary considerably in language classrooms, it is necessary that a review of the construct validity of the testing process is conducted. She adds that examinees must be informed about what is being measured in the speaking tests and that the teachers and researchers need to make sure that the testing and rating processes follow a designed plan.

Although the book provides example tasks in the development of testing materials, classroom teachers might need more examples and detailed procedures in the application of these tasks to the assessment situation. The book offers guidelines and procedures in test development which could have been explored some more to include specific examples that could be easily adapted in the classroom or in training programs that target the development of fluency in the second language. More exhaustive sample tests that explicitly state the skill to be assessed, the tasks and procedures, rating scales or rubrics, and interpretation of results or test scores would further emphasize the importance of following a theory-based assessment process. In following this, classroom teachers will develop a heightened awareness of the components and steps of assessing speaking within their own classrooms.

The book has limited discussion of the learner in various assessment contexts. It is a relevant issue because the focus on reliability and validity of measures in speaking tests also requires proper accounting of learner factors in the acquisition of competencies in speaking a second or foreign language. Classroom teachers need to clearly control for the suitability of the measures to the proficiency level of the learners being tested in order to ensure that results are credible and generalizable to individuals in similar settings. It is important to note here that learner factors such as motivation, age, L1 background, social status, intelligence and talent, etc., have been studied in various classroom settings and have proven to contribute to the success or failure in the acquisition of effective speaking skills (Derwing & Rossiter, 2003; Dalton & Seidlhofer, 1994). Considering these factors in the assessment process, therefore, is also highly important.

Although Luoma briefly discusses inter-rater reliability (pp. 179-180), the procedures in judging performance in speaking are not thoroughly covered in the book. Teachers and researchers need to understand the difference between subjective or objective judgment in various applications and goals of the speaking tests. Moreover, credible procedures in arriving at a rating score whether the assessment calls for different raters or just one teacher are needed to be established in the test design.

Current trends in computer-assisted language learning (CALL) have provided the assessment of speaking skills newer perspectives and directions. CALL-based pronunciation training and testing are slowly being incorporated in the classroom and have evolved to include automatic computer raters and assessment procedures that could be done without the presence of a teacher (Pennington, 1999; Neri, Cucchiarini, Strik, & Boves, 2002). The role of CALL in the assessment of speaking is a pertinent field that could be added as a chapter in this book. Future applications of assessment in speaking will most likely involve CALL components in the development of tests and actual scoring of performance.

Luoma’s Assessing Speaking is an excellent starting point for teachers and researchers involved with the assessment of the different kinds of speaking skills. The book addresses important issues and provides a wealth of information that will equip the readers with sufficient grasp of the theories and concepts needed to develop tasks and rating scales in various assessment settings. The content and style of the book are presented in very clear, “down to earth” writing that will appeal to a range of audiences.

References

Celce-Murcia, M., Brinton, D., & Goodwin, J. (1996). Teaching pronunciation: Reference for teachers of English to speakers of other languages. Cambridge: Cambridge University Press.

Dalton, C., & Seidlhofer, S. (1994). Pronunciation. New York: Oxford University Press.

Derwing, T.M., & Rossiter, M.J. (2003). The effects of pronunciation instruction on the accuracy, fluency, and complexity of L2 accented speech. Applied Language Learning, 13, 1-18.

Neri, A., Cucchiarini, C., Strik, H., & Boves, L. (2002). The pedagogy-technology interface in computer assisted pronunciation training. Computer Assisted Language Learning, 15(5), 441-467.

Pennington, M.C. (1999). Computer-aided pronunciation pedagogy: promise, limitations, directions. Computer Assisted Language Learning, 12(5), 427-440.