HETL Note: We are pleased to present Austin Ryland’s research article “Worked examples: Teacher Practices” which investigates best practice in using “worked examples” – a problem-based teaching method especially suitable to the disciplines within the STEM (science, technology, engineering and mathematics) cluster of areas. “Worked examples” can be extended to online and technology supported collaborative teaching and learning models. The findings of this qualitative study provide useful insights into how “worked examples” is applied by expert educators and how it can be developed further and integrated into emerging educational formats such as MOOC (massive online open course). You may submit your own article on the topic or you send “letter to the editor” of less than 500 words (see the Submissions page on this portal for submission requirements).
Author Bio:
Austin Ryland is a graduate research assistant at the College of Education Office of Research & Service at the University of Alabama, U. S. A. He is currently a doctoral candidate in higher education administration. His dissertation focuses on the graduate gender divide in STEM (Science, Technology, Engineering, and Mathematics) fields. His research interests include STEM education, institutional research, and the gender divide in higher education. Contact email address: [email protected]
Worked Examples: Teacher Practices
Austin Ryland
University of Alabama, U.S.A.
Abstract
The purpose of this article is to present interview feedback from university teachers who have been acknowledged for their use of worked examples in their pedagogy. Worked examples are teaching techniques which can be used in fields that are currently being emphasized nationally in the United States STEM (Science, Technology, Engineering, and Math) fields. This work identifies best practices for the teaching method of “worked examples” (WE). Semi-structured interviews were conducted over the phone or through email by nine respondents from American and European universities. Responses indicated continued use of worked examples and student satisfaction with applying worked examples as a context-specific teaching method.
Keywords: worked examples, teaching best practices
Introduction
Worked examples (WE) are step-by-step demonstration processes of how to complete a problem or perform a task (Clark, Nguyen, & Sweller, 2006). WE seek to impart information. An instance of a worked example may be the process of how to perform a math problem correctly. Concepts are first introduced in their simplest form. A gradual progression of simple to complex step-by-step procedures are an integral part of WE (Clark et al., 2006). Knowing the appropriate levels of progression and starting points is referred to as “scaffolding” (Atkinson, Derry, Renkl, & Wortham, 2000; Glaser, 1976). Since WE are a way to impart information, the process is considered a form of lecturing (McKeachie & Svinicki, 2011).
There is potential integration of WE and recent advancements in technology to create an online “how-to” approach of imparting information (lecture) for a given task or problem. WE are also a form of problem-based learning (PBL). For WE, overlap with PBL occurs when the purpose is to provide a problem as a facilitating concept acquisition medium (McKeachie & Svinicki, 2011). For instance, in a classroom setting students may be organized into groups, and each group is given an applied statistics problem to solve. The overall problem is a PBL approach; however, the exact process used to approach the problem can be a sequential process synonymous with a worked example. One cited advantage of WE over PBL is the means by which WE allow for the learner to address the process of learning material in place of focusing on finding an immediate solution (Sweller, 1988).
A more recent interpretation of WE allows for a broad conceptualization where multiple technology media can be used to facilitate dialogue and discussion. This is a much more fluid and dynamic conceptualization of WE which presents an example of a continual process with learners collaborating (Barab, Dodge, & Gee, 2012a; Barab, Akram, & Ingram-Goble, 2012b). A community of learners comes together to create a worked example which acts as a stand-alone hypothetical scenario or problem. WE are intended for both the community of creators and for WE users.
Fields of study for which WE are appropriate include linear, structured tasks where there is a logical build-up of a knowledge base (Clark et al., 2006). While there are similar teaching techniques, such as POGIL (Process Oriented Guided Inquiry Learning) (National Science Foundation, 2012), WE are unique in their ability to be used with individuals or groups and, potentially, collaboratively. WE are the cornerstone, basic processes for the applied teachings strategies such as POGIL. However, POGIL does provide a related educational outlet for how a WE-related process can be applied in education and industry settings. One sponsor for POGIL is an industry, TOYOTA (National Science Foundation, 2012).
The focus of this paper is on worked example use in university settings. WE have relevance to teaching with technology and to “mission critical” fields (Obama, 2011). Academic areas in which WE fit include disciplines with logical, structured curriculum. Examples include sciences, technology (online media tutorials), engineering, and math, or related STEM fields. Given the recent expansion of technology in education, U.S.A. colleges and universities focus increasingly on teaching with new technologies, whether in face-to-face, hybrid, or online formats. Perhaps the most prominent technology promoted recently is the MOOC, or Massive Open Online Course, which seeks to reach a broad audience. A MOOC is a teaching platform offered in an online format, either for free or at a discount. The value of MOOCs when compared to face-to-face and hybrid class formats is a topic of daily debate in U.S. higher education.
Context for WE can be significant on two levels. The first level is the setting in which WE are used, i. e., the context of the classroom setting. The second level is the context of college teaching, including the changing dynamics of how external factors influence college teaching. Such external factors can influence how college teaching is conducted (changes in technology) as well as what is being taught (STEM fields being promoted).
The use of technology, including MOOCs, on a broad scale is significant because such courses need efficient ways to teach and to assess student learning. MOOC technologies and pedagogies serve disciplines where there are “solutions.” MOOCs are less suited to disciplinary practices that may not have concrete solutions or that have different views on what constitutes a solution. WE can be more beneficial to students on an individual level, where MOOCs teach individuals on a broad scale. The significance of these research findings may have as much to do with the timing and context of the study as well as with the actual findings.
Literature Review
The history of WE is grounded in theory on teaching students to remember concepts. Examples have always been used as the main medium for teaching conceptual learning (Atkinson et al., 2000). This history is linked to the history of cognitive psychology, which started in America during the 1950’s (Bruner, Goodnow, & Austin, 1956). A development of cognitive psychology, cognitive load theory, is given credit as starting during the 1950’s with a groundbreaking study on the amount of information people can remember in general relative to numerical digits (Miller, 1956). From the 1950s to the 1970s, WE history remained synonymous with the history of cognitive psychology. Another layer was added in the 1970s by providing for complex WE. The use of ordering and layering of presentations (scaffolding) techniques used in conjunction with WE increased (Atkinson et al., 2000; Glaser, 1976).
In the 1980’s problem–based learning was a main framework for learning theory (Atkinson et al., 2000). WE that were embedded in PBL became prominent. The interplay between WE and PBL continues currently. POGIL is an example in a traditional classroom setting (National Science Foundation, 2012).
The connection between WE and PBL consisted of the level of learning appropriateness, or the ability for the teaching method to meet learners at their level (Tarmizi & Sweller, 1988). This connection between WE and PBL regarding meeting the learners at their level remains true today. Depending upon the scenario, WE alone, or combined with PBL, was considered more advantageous than conventional problem solving when the goal was to increase student learning. The purpose was to take a pragmatic approach to increasing learning (Ward & Sweller, 1990). WE were also found to be consistent with 1980s research looking at prototypes, archetypes, and similar mental models (Cooper & Sweller, 1987). This finding indicates that the mental model of the individual was integral to showing how WE were an effective approach to teaching (Sweller, 1988).
During the 1990’s WE use in the classroom grew as a stand-alone practice (despite the continued overlap with PBL) (Reed & Bolstad, 1991), reflecting the trend of applied research (Shuell, 1996). Self-explanations (the process of talking out loud while working through a problem) was also a term that arose during this era (Mwangi & Sweller, 1998). During the 2000’s WE grew in prominence with the re-emergence of STEM fields, as shown by recent highlights of STEM fields nationally (Obama, 2011). Continued research growth involving WE is expected due to the national emphasis of fields in which WE are appropriate as a teaching technique.
Advances in technology and changes in curriculum theory have continually influenced WE (Mayer, 1997). Technological approaches which seek to enhance WE have resulted in a paradigm shift in the notions of expert and learner as related to WE. This paradigm shift has led to a new conceptualization of Worked Examples presenting the Worked Example as an overall outline for a step-by-step process. The elements that fill this design–including content, instructional design elements, scaffolding, and related contextual factors–contribute to the WE. Instead of an expert-learner dyad, there is a community of experts and learners coming together to create an instructional, often problem-based process.
There is an entire website devoted to WE (workedexamples.org) (Barab et al., 2012a; Barab et al. 2012b) with a continued website (workingexamples.org) (Gee, 2012) which allows for the promotion of WE that use this new technology framework (Barab et al., 2012a; Barab et al. 2012b). For instance, navigating to the online website for WE, workedexamples.org, and finding the worked example titled “Worked Example About Worked Examples” conveys the scaffolded approach to the delivery of content. For this specific “example,” the content also provides a deeper look into the process of adapting WE into a technology medium (Barab et al., 2012a; Barab et al. 2012b). Working examples are a newer, improved version of WE. Working examples aim to make WE more audience-appropriate (Barab et al., 2012a; Barab et al. 2012b; Gee, 2012). This attempt to meet learners at their level is a key concept of working examples (Gee, 2012).
There is evidence that WE are successful for lower-level students, yet not for higher-level students with select mathematical concepts (Carroll, 1994). This evidence provides insight into the critical implementation process for WE. There appears to be an inverse relationship for effectiveness regarding learner capability (novice through expert) interacting with the level of difficulty of the WE. The exact level of a novice learner needed to maximize the effectiveness for a WE depends on the context. A novice learner may potentially benefit more than an intermediate learner (Tuovinen & Sweller, 1999). While WE may not have either a positive or a negative impact on an intermediate learner, they may have a negative impact on an advanced learner. Finding out the appropriateness of fit with a WE and the learner is key (Salden, Aleven, Schwonke, & Renkl, 2010).
Scaffolding is a technique where concepts are introduced at select points in a given sequence or format in order to enhance learning (Atkinson et al., 2000; Glaser, 1976). Crossover between scaffolding and WE is common, given the shared intent to maximize the ability of the “expert” to impart information to the learner through additional learning techniques and aids in a prescribed step-by-step manner. Scaffolding is related to learning processes which seek to break down data into manageable parts (chunking). There is evidence that chunking approaches are more effective than processes which do not break down data into manageable parts (van Gog, Paas, & van Merrienboer, 2008).
The fields of cognitive psychology and instructional design offer support of learning theories for WE (Atkinson et al., 2000). A specific learning theory supporting WE is cognitive load theory. Differences in processing for short-term memory vs. long-term memory are noteworthy. Cognitive load theory is at the core process of WE, which takes portions of a problem and adapts them to the learner’s cognitive load and uses instructional design to facilitate knowledge acquisition (Sweller, 1994).
One individual-level learning attribute used in conjunction with WE is that of self-explanations (Atkinson et al., 2000). Self-explanations refer to individuals taking an active role in processing and verbalizing their learning experiences (talking or writing). Self-explanations (elaboration) elucidate self-monitoring and abstract generalization skills. Evidence reveals how much self-explanations seek to provide a way for learners to apply their newly acquired concepts to new situations (Chi, Bassok, Lewis, Reimann, & Glaser, 1989).
The work by Chi, Bassok, Lewis, Reimann and Glaser (1989) reveals how much WE can be an individual-level approach dependent on the individual’s mental model. Self-elaboration is a means of communicating this mental model, demonstrating how a student is learning, and articulating ways a student can apply new concepts. Application of knowledge in the same repetitive process reveals near transfer of knowledge. Application of knowledge in new ways reveals far transfer. Near and far transfer of knowledge can reveal an individual level characteristic which holds more influence than generalizations from cognitive load theory (Woltz, Gardner, & Gyll, 2000).
One study of teacher training in a higher education setting revealed differences by employing product vs. process WE settings. A group of teachers reviewing a worked example by viewing tasks only (the product condition which consisted of only powerpoint slides) created better design projects than a process group (which included a video of the process). The task was how to perform a literature review for teachers who were being trained in instructional design. Expert ratings of the teachers undergoing training were used to compare the two groups. Explanations for the higher performance of the teachers with the product condition (powerpoint only) included a simpler task with less cognitive load (Hoogveld, Paas, & Jochems, 2005).
Augmenting instructional design is the main way that technological adaptations have changed with WE. Since instructional design elements can be so vast, there will be three elements briefly mentioned which have paralleled the development of technology. These three instructional design elements are as follows: gaps in WE, prompts, and ability to offer help. All three elements are based on the assumption that instructional design increases the quality of self-explanations, which are used by learners to grasp new concepts (Atkinson & Renkl, 2007).
Gaps in WE include instances where a literal gap in the WE process is used to try and facilitate concept gain by students. Anticipating the next step in the WE process is the rationale behind gaps. Prompts are used to try and engage students throughout the WE process to develop better self-explanation processes (Atkinson & Renkl, 2007). The ability to offer help is perhaps the technological adaptation which has influenced WE the most. The ability for learners to ask for help at the touch of a button through an introductory-level worked example would seem to be a highly advantageous development for WE as a result of technology.
Tests and quizzes can be commonly associated with WE. Additionally, self-explanations act as a means to provide elaboration at select critical time points within assessment frameworks. Synthesis of viewpoints (Carroll, Booth, & Cooper, 2011) within an assessment framework can facilitate WE assessment.
One instance of this occurrence was in a chemistry setting. Quizzes were used to assess performance throughout a course. Self-explanation prompts were used in addition to WE with positive results (Case & Gunstone, 2003). Given the focus on engagement on a cognitive level of WE, there is a tendency towards individual-level learner assessments. In contrast to individual-level approaches to assessment are satisfaction-oriented focus groups (Folkard, 2004). Focus groups are a group-level form of self-explanation.
The definition of WE affects assessment approach. The traditional definition of WE as a step-by-step process with an expert and learner(s) lends itself towards tests and quizzes through simple knowledge acquisition. The tests and quizzes within this traditional framework reemphasize individual-level focus (Atkinson et al., 2000). This emphasis on individual-level cognitive approaches reinforces self-assessment strategies (Kostons, van Gog, & Paas, 2012).
WE can be thought of as an assessment process for cognitive-based self-assessment learning paradigms (self-regulation) (Kopp, Stark, Heitzmann, & Fischer, 2009). The exact medium for assessment is open to self-reflection, self-explanations, or guidance through a scenario using technology. The self-reflection, self-explanations, or guidance through a scenario can be considered an assessment process.
These individual cognitive-based elements stand in contrast to a broader conceptualization of WE as a practice of a community of learners using technology as a facilitator (Barab et al., 2012a; Barab et al., 2012b). This broader conceptualization of WE presents ways in which online WE can be assessed. Focus groups can be conducted as in traditional satisfaction-oriented cases. The exact manner in which WE must be assessed in this broad-based conceptualization using technology is highly contextual. The manner in which the WE must be assessed depends on the exact WE being used. The assessment approach must match the context of the WE.
This broader approach is more problem-oriented. As WE in this framework provide an outline (schema facilitation) (Meier, Reinhard, Carter, & Brooks, 2008) for a single-instance of an example or a problem on a larger scale, program evaluation measures might be more appropriate to use as an assessment framework. Program evaluation may be a better fit because the program evaluation concept is larger and thus fits with the broader approach. Program evaluation can include learning outcomes or learning objectives, but it would also include the entire program. A worked example may be a small part of a classroom or program evaluation (Barab et al., 2012a; Barab et al. 2012b). Student (learner) feedback is key to assessing a worked example regardless of the type of WE used, whether the conceptualization is traditional or newer.
A problem with generalizing WE is the highly context-specific nature of their effectiveness. This problem is augmented by the context-specific nature assessment approaches must have in order to match the context of each worked example. The broader conceptualization of WE via technology exacerbates this problem when trying to assess WE scenarios. The context of a specific technology being used may be so individualized that generalizations may be difficult to make if there are not comparable technology contexts.
Using learning approaches familiar to students helps knowledge acquisition (Hilbert & Renkl, 2008). Test scores can be one way in which WE effectiveness can be measured. A classic example of a setting that resulted in higher test scores for WE compares a WE setting to a traditional setting with the subject material of calculus (Miller, 2010). Results have been replicated in high school algebra settings (Carroll, 1994).
The rationale behind the process of how select aspects of WE are advantageous for test scores can be a closer, more authentic replication of real-world scenarios (Boekhout, van Gog, van de Wiel, Gerards-Last, & Geraets, 2010). An additional explanatory factor can be prior knowledge. Prior knowledge is indicative of overall positive main effect in WE (Tuovinen & Sweller, 1999). Less time to complete problems has also been shown with the use of WE (Paas & Vanmerrienboer, 1994).
Evidence for WE over strictly PBL approaches has been shown at earlier stages of PBL regarding efficiency (Kalyuga, Chandler, Tuovinen, & Sweller, 2001). Efficiency for WE over exploration practice alone has also been demonstrated (Tuovinen & Sweller, 1999). One study revealed how the ability to meet learners at their level promotes WE over PBL through higher immediate and post-test scores for WE (Salden et al., 2010). Increasing guidance within problems has shown to yield mixed positive results (Stark, Mandl, Gruber, & Renkl, 1999).
Types of explanations have also been shown to be correlated with higher test scores of WE (Renkl, 1997). The context of WE influences the degree to which self-explanations increase WE effectiveness. Generalizations are hard to support (Renkl, 1997). However, there is evidence that adding self-explanations to WE can increase average test scores (Crippen & Earl, 2007). The ability to find the right amount of cognitive load with WE when using self-explanations surfaces with the following study where self-explanation conditions were compared on test scores. The ranking of groups is as follows: self-explanations prompts with no instructions, self-explanation prompts with instructional prompts, no self-explanation prompts with instructional explanations, no self or instructional explanations of any kind (Schworm & Renkl, 2006).
Levels of participation have been shown to be greater with WE (Darabi, Nelson, & Paas, 2007). Knowledge gain and skill retention has been shown to be higher for WE as compared to a control group in a game setting (Shen & O’Neil, 2006). Instructors trained in a WE process setting had higher attitudes towards regarding applicability compared to teachers being trained in a WE product setting (Hoogveld et al., 2005). Qualitative data indicate that WE reduce student anxiety (Folkard, 2004).
Research Approach
The main purpose of this research project was to learn how expert faculty teach using WE. This study used a qualitative approach. A small sample size was acceptable. Respondents were chosen due to their recognized use of WE. This recognition often came in the form of published journal articles, book chapters, or both. Teaching awards or conference presentations were also acceptable forms of recognition.
To be eligible, a respondent had to have actively used a worked example process recently. For example, professionals who teach in training and research university settings using WE were acceptable. Respondents could have come from any discipline. The main medium for identifying respondents was through a review of recent literature and identification of authors who used WE.
Interviews were used to collect responses and followed a semi-structured format. Each interview had a set of questions which, in general, were asked in the same order. In select interviews over the phone, the questions were asked out of order to create a sense of flow for the conversation. Questions were asked in the same sequence as much as possible. Often a respondent would answer more than one question in one response.
Interviews were sought in two formats, synchronous and asynchronous. Both interview formats used the same predetermined set of questions. Overall there were 15 questions; five questions contained follow-up sub questions (Table 1). A generic greeting was included with the questions. (See Appendix for a complete view of the greeting and questions.)
Table 1. Interview Instrument
Interview Questions
1. Tell me about a course you regularly teach using Worked Examples.
- a. What is the course about?
- b. What level course is it?
- c. How many students typically enroll in the course?
2. What are your primary teaching goals for the course?
- a. Content goals?
- b. Skill goals?
- c. Affective goals? (e. g., satisfaction, appreciation of subject matter)
3. Please define the method, in the way that you use it in your course.
4. Do you also regularly use other methods in the course in conjunction with worked examples?
- a. If so, what are they?
- b. How do they work with Worked Examples?
5. How do Worked Examples help you accomplish your teaching goals?
- a. Content goals?
- b. Skill goals?
- c. Affective goals? (e. g., satisfaction, appreciation of subject matter)
6. Why did you chose to use the worked examples?
7. Please tell me about a story, incident, or experience you have had while using Worked Examples that conveys why you feel the chosen instructional method is effective.
8. How do you prepare to teach using Worked Examples?
9. What do you do during a typical class session in which you use worked examples?
10. What problems do you typically encounter when using Worked Examples and how do you deal with them?
11. How can you tell that students are learning during the process?
12. How do you grade students?
13. How do you gather feedback about the course from students?
14. How do you reflect and improve for the next course?
15. Have you honed your skills in Worked Examples for teaching?
- a. Read?
- b. Attended workshops?
- c. Talked to colleagues?
Data Collection and Analysis
Invitations with a tailored message based on the generic introduction and 15 interview questions were emailed to potential respondents. The list of questions was included in the email message. An IRB-approved consent form was attached to the invitation email.
All synchronous interviews were conducted by telephone. Taped interviews were transcribed. Interviews were open-ended in length, so that respondents could take as long, or as short, a time to respond as they wanted. Asynchronous participation was possible if a respondent chose to answer the questions and return them, along with the consent form, to the interviewer.
Nine interviews were conducted. These nine interviews were out of 50 email invitations. There was approximately an 18% response rate. A broader approach such as survey methodology in future research can be used to learn more regarding the use of WE across university teaching populations.
Results from the interview transcripts were reviewed and analyzed for themes. Highlights of transcripts are presented according to each theme. An introduction to each theme is presented, along with supporting quotes from the transcripts. Often a block quote from a single respondent provided support for each theme.
Since this is a qualitative study, the implications of results are limited. The goal of qualitative research for this study was not external validity. Rather, the analytic focus is on interview transcripts and characteristics of participants.
Respondents represented a range of disciplines and academic ranks, from graduate students to full professors. Two respondents were graduate assistants in research labs which emphasized teaching methods. Both graduate assistants worked alongside professors who had been recognized in the use of WE through publication. Both graduate assistants taught using WE. Two respondents were long-tenured faculty members in psychology departments. One of the psychology professors taught research methods and statistics coursework, and the second psychology professor taught classes in cognitive psychology and problem solving. Two professors were faculty members from Computer Science programs. Two faculty members taught Statistics, one of which taught graduate statistics in a university Education department. One respondent dealt with the new conceptualization of WE where the WE had been adapted to a technology setting. Two respondents worked in Europe, one in Germany, and one in the Netherlands. The remaining seven respondents worked in the United States.
Themes emerged in response to the interview questions. Themes were an attempt to summarize respondent feedback concerning a certain question. Often respondents indicated they had already answered a question in a prior answer. For example, question two and question five have a similar focus on teaching goals. Question nine, asking about an outline of WE use in class, contained repeated elements from question three regarding defining WE and how WE are used in the course. Given the potential for repetitive information, themes were, when possible, drawn out from multiple questions. There was not always a one-to-one relationship with themes and questions. The question with the highest degree of overlap was question seven. Question seven concerned a story which revealed WE were effective. This story often contained elements from related questions. For instance, question two and question five included goals, so a story focusing on the effectiveness of WE may have included goals. Additionally, question seven overlapped with question eleven, which asked about how a teacher could tell if students were learning.
The following is the way that themes were identified in responses to the interview questions. The first theme to emerge comes from question three which asks respondents to define WE. The definition of WE as a theme is presented first because how respondents defined WE forecasts the framework of respondents for remaining themes. Theme two, technical settings, was in response to question one which asked about which classes were taught using WE. Theme three, novel approaches, was a response to questions two and seven, which included teaching goals and a story illustrating how WE are effective. Theme four, learning through failure, emerged from questions four and seven; theme five, novice learners, from questions five and ten; theme six, value of WE, from questions seven and fifteen; theme seven, satisfaction, from questions five and eleven; theme eight, individualism vs. collaboration, from question four. The themes are presented below and are supported by excerpts from the participant responses.
Theme 1: Definition
Defining Worked Examples (WE) displayed variation across respondents. Eight of the nine respondents had a traditional definition of WE. One respondent referenced a collaborative, sociocultural approach using technology.
WE were defined as “… the application of a problem-solving procedure that is illustrated in a particular case.” A more exact definition articulated WE as “…an explicit example showing the steps for how to solve a problem or the work for how to solve a problem.” A broader interpretation defined WE “very broadly, that would include demonstration, activities, those are sort of examples to me.”
Given that instructional technology can have such an influence on WE, participants were careful to differentiate between WE and guided design because “guided design can be more interactive and less explicit.” Guided design is a process which is similar to WE in that instructional design approaches can provide a sequential process for walking a learner through a problem or task. However, guided design does not have the level of structure and is not as precise in methodology. A difference between WE and guided design was noted: “guided design is a problem a learner is trying to solve, and is given help in how to do it.” One respondent commented: “I think of examples as…structured scaffolding.” These quotes illustrate how WE are a more structured, precise process than simply guided design using technology, or even helping a learner with a given concept in a problem-based learning environment.
Theme 2: Technical Settings
Overall, WE were used in technical disciplines in a traditional class environment. As one respondent said: “[WE are] a natural approach for topics of a technical nature, since many of these topics involve a good deal of problem solving.” In every setting, the intent was to facilitate knowledge acquisition. Groups were often used in a collaborative manner. Formats differed in getting learners to articulate through discussions or respond to problems individually.
The theme of technical settings was in response to the six class types that can be considered technical in nature. These classes include the following: four statistics classes, one computer science data structures & algorithms class, one class on computer programming languages. There were two classes which focused on teaching methods: one in educational psychology and one on teaching competency. For the educational psychology class, WE were used as an instance of teaching math concepts. Since math concepts were one of the subjects where WE were used, this educational psychology class had a technical emphasis, although this emphasis was not enough for the class to be considered a technical course. One respondent used WE in presentations related to using WE as software. Following is a quote which reveals the most prominent type of course across the nine respondents, statistics:
Statistics, the thing I want them to really understand is how to select proper statistical analysis, so they have to be able to identify how to identify variables and based on the design know what the appropriate analysis is to analyze the data. So I really emphasize that in the examples.
Theme 3: Novel Approaches
The responses reflected idiosyncratic teaching styles. Two respondents noted having online WE “problem banks” where students could practice on their own. One professor took the class outside and had diagrams drawn.
Since environment diagrams are pictorial, I take the students outside with sidewalk chalk and let them draw the diagrams on the sidewalk. I have other students walk around asking questions and pointing out problems with the diagrams. Since I started doing that, I feel that average case mastery of the subject has improved.
Theme 4: Learning Through Failure
One of the consistent reports spoke to the degree to which learning was achieved through “working” the example. This process included allowing students to take multiple approaches, right or wrong, and find their own avenues which can lead to better understanding, or deeper learning, than simply showing a solution. One respondent even went so far as to remove the answers so that a large class (180 students) would be forced to become more engaged when solving problems in group tutorial sessions.
I present a problem and then probe the class for approaches to solve the problem. Sometimes, I let the students go down a wrong path, since it is important to recognize such paths and which choices led to the wrong path….the process is the ultimate goal.
Theme 5: Novice Learners
Almost all respondents used WE with novice learners. One respondent even dealt exclusively with a research approach that used WE for novice learners: “There is an immense amount of evidence that learners, especially novices, prefer to learn from examples.” That WE are seen as positive for novice learners introduces a problem with the method. The problem is whether WE are understood by students. One instructor captures the issue precisely: “If you have kids on the really low end or really high end WE are almost too overwhelming or too easy. Tailoring the WE to the student’s prior domain knowledge is really, really important.” One respondent found a way to tailor lessons to novices: “Some students tend to dominate. I say at the beginning that once you feel you get it, you should refrain from answering questions.” Fading was also growing in prominence.
There’s the idea of fading where you maybe give them a complete worked example or two and then you give them a problem to solve they just have to do a small part of it. Then you successively give them more and more of the uncompleted part of the problem and have them do it.
Theme 6: Value of WE
The value of WE was consistent in responses. This positive bias towards WE could reflect teacher characteristics that lead instructors to use WE. In general, the positive impact (and potential) for WE was promoted.
The importance of creating good examples that learners can use to generalize beyond those examples is going to be absolutely crucial. Otherwise learners memorize steps to examples and don’t know what to do with it. If they’re created well, you can generalize, that’s the important thing. The use is inevitably going to grow.
Theme 7: Satisfaction
WE are received positively by students. Motivation was noted, especially as WE are part of a problem-based learning technique. Students show a preference for an example.
Well students love it. Most students, and the research shows this, they pay more attention to examples, especially math classes, than text explanations. As soon as they have a problem to solve they look for an example they can identify. Examples are very valuable to students.
Theme 8: Individualism vs. Collaboration
Collaboration with WE may have taken place between an instructor and a large class, group work, or pair work. Additional approaches start out working individually, then collaborate with class feedback. Pairs or smaller groups were better because they were closer to individual approaches.
One approach was purposefully individual: “students are handed out the learning materials and work with them by themselves. Typically, in these sessions the teacher is only there to support students or clear up misunderstandings and problems with the materials.” This individualistic method is in contrast to the conceptual, collaborative approach of WE, which intentionally seeks to promote collaboration. One respondent promoted technology and WE to focus on the individual: “WE are beneficial if there is some kind of or explanation or activity happening with them…if you were using a computer or interactive design, it would be easier to do with individuals.”
Discussion
Eight out of nine respondents held a traditional framework of WE. Hence, the traditional and individual-level framework in using WE can be expected throughout the study conclusions, implications, and discussion. For instance, the use of tests and quizzes to assess WE was predictable. Tests and quizzes aside, finding agreement among respondents in order to articulate a theme was difficult. Respondents seemed to have their own approach to testing or quizzing; however, the simple use of tests or quizzes seemed common among respondents. Some new avenues emerging from these data can provide avenues for future research. One such avenue was the use of online problem banks. The use of such techniques shows a transition to using technology, albeit a gradual process still based in a traditional framework of WE.
Overall, the emphasis on structured processes based in cognitive, individual frameworks reinforces the literature. There was a virtual one-to-one relationship with assessment approaches used in the literature and those used by respondents. Types of tests and speaking out loud, including discussion, were used, yet in ways which were unique to the respondents.
Some respondents designed features to enhance WE. These refinements include fill-in-the-blank questions or erroneous examples. Use of these approaches seems to be context-specific to the learners and the teachers. This context-specific nature of WE is consistent with the literature. Future research can look at the disciplinary background of instructors and how this background influences the WE context, as well as the degree to which background influences technology use.
Perhaps broader approaches by disciplines can uncover the ways in which the novel approaches of each respondent may be a result of disciplines in place of individual respondents. Even though each respondent brought his or her own style to the process of teaching WE, the responses indicated the importance of WE as a process. This remained true regarding the ability of the respondents to learn through failure, as well as the importance of creating WE that meet learners at their level, as shown through the theme of novice learners. The continued emphasis of WE as a process was consistent with the literature.
The manner in which technical settings were emphasized conveyed the significance of WE to the STEM fields. This could be a result of the nine respondents who chose to respond to the questions, or a result of the focus of the types of professionals who were being interviewed and the way in which respondents were identified as “experts.” Publishing journal articles or book chapters, like receiving awards, may be valued in select disciplines of the respondents, or by select types of individuals in university settings, even institutional types. In this way respondent characteristics act as the raw data upon which conclusions and insights can be drawn.
Overall, according to feedback from respondents, there is promise for the use of WE. While this promise may result directly from respondent bias, there were positive themes for student satisfaction and value of WE. The growth in STEM fields and the overlapping process WE use with these fields is promising for future developments of WE in STEM and related sequential, structured disciplines. This forecasting of positive growth in the use of WE remains true in as much as the feedback from respondents can be generalized in future research.
The final theme of individualism vs. collaboration is perhaps the most intriguing. Due to the traditional WE framework of most respondents, there would be an expected traditional emphasis on the individual approach to WE, and to a certain extent this proved true. However, there seemed to be room for using pairs or groups in different ways to facilitate learning while using the WE process. The way two conceptualizations of WE, one step-by-step traditional and one collaborative technology, evolve has promise for future research. This evolution includes using technology to enhance learning of WE on an individual level, as well as findings ways to facilitate the use of WE in groups, even using technology. Respondent feedback suggests that most of the approaches remain grounded in an individual-level approach enhanced by group work. Perhaps this group enhancing of the WE process can be combined with a focus on novice learners to provide new ways for fading of WE to facilitate knowledge acquisition. This fading process could be true for moderate or expert learners, as well as for novices. Finally, respondent focus was mostly on knowledge acquisition on an individual level, in order to enhance the arrival at a solution, rather than on working together to form new knowledge while facilitating the arrival at a solution. This focus could be the result of the fields that use WE as shown through the respondents or simply the bias of respondents.
Given that WE can be a process as well as an immediate technology medium, WE can be considered the “building block” of curriculum. While this curriculum can certainly be technical, as with STEM fields, any online curriculum can be conceptualized as a sequence of WE. This includes the step-by-step approach to creating a lesson, class or even curriculum.
Given the growth in MOOCs, there is a readily available illustration of how WE as a process and technology can be applied to a large audience. Additional online tools also feature this WE process implicitly when a step-wise “example” is given. There is significant room for interaction between the teaching process of WE and related teaching approaches.
Future research can focus on collaboration with WE and technology. The potential for collaboration of professionals in technical (STEM) disciplines can be explored with WE. This includes the new WE technology mediums and similar collaborative technologies. Since respondents taught WE for technical fields, and WE translate well to these STEM fields, enrollment and funding trends in STEM fields could influence the manner in which WE are taught in the future.
Additional use of WE may be considered outside of the university setting. One respondent was a consultant who used WE in the private sector. This consultant respondent did not report significant differences between using WE in the private consulting sector vs. the university setting. WE may arise more in training scenarios in the private sector in addition to the university setting. A broader review may be needed to compare the two settings. No respondents worked in for-profit university settings, which represent another area for further exploration and may provide a middle ground between the private sector and nonprofit university settings.
Given the potential of the paradigm shift with WE being used as a stand-alone “example” in only technology mediums (a sociocultural approach as described by James Gee (2012) on the working examples website), there is still room for development of WE beyond the traditional paradigm. WE have been noted as a form of problem-based learning (PBL). Using the new concept of WE allows for new possibilities for learning via technology and PBL.
A possibility for new WE uses might reside in the field of instructional assessment. Given that WE are inherently an assessment process, providing WE as case studies of assessment is a possible applied use. A report of a case study of assessment “best practice” can be presented via a problem-based learning format through an online medium such as a WE. Assessment is included through the actual content of the assessment case study, as well as the structured process of the online WE. This can provide an online resource for assessment. An online resource for assessment is only one potential application of WE which is possible via a new paradigm shift.
Conclusion
The significance of this project has as much to do with the timing and context of the study as the actual findings. The initial purpose of this work was to discover how WE are currently being used. The study then focused on where to move forward, based on the qualitative data collected. The analysis concludes that there are more contexts to be explored with the use of WE and forecasts ways to explore these contexts. These contexts include highlighting one way in which WE have been tried as a technology medium, the use of teaching with technology to broad audiences, such as MOOCs, the use of WE outside of the traditional non-profit university setting, as well as industry settings. Additionally, there is potential for overlap among MOOC’s, industry, and education.
The qualitative data revisit how WE are being used in a potentially changing teaching landscape. This teaching landscape includes which disciplines are being promoted nationally as well as how courses are being taught, which institutions may lend themselves to using teaching approaches for mass audiences and which institutions may include STEM fields. One example includes select courses that are required to be taught in select ways to allow for financial savings (i. e., MOOCS. Certificate programs comparable to MOOCs may also grow in number). Such growth may include free coursework, certificate programs or badges which can then be applied to universities to obtain course credit. Again, types of teaching techniques that may be used in this process need to be revisited to learn about teaching effectiveness; one of these teaching techniques is WE.
The study sample is small, so the results are more suggestive than generalizable. A quantitative method approach such as survey methodology can be used to generalize on a broader scale. Perhaps this review can be used as a resource to move forward with developing a survey to gather more information from faculty who teach using WE. Alternatively, this study may prompt a survey to send to faculty who teach in STEM fields to gather feedback about the potential use of WE.
For the purposes of this study, technical fields seem synonymous with STEM fields. There is no universal definition of STEM fields, and the question of which fields are STEM is a matter of ongoing discussion. There was a recent conference presentation which highlighted different definitions of STEM fields (Moore & Sapp, 2013). Also, for this study, technical courses do not have to be taught in a STEM field, as in the case of a statistics course that is taught in a non-STEM field, such as psychology.
Future research may seek to review the use of STEM fields where there is overlap of industry and education. Future studies may seek to learn about the use of WE in the U.S.A. community college setting. There are courses which can be considered technical in nature which are not in a STEM discipline. For example, WE may be useful in non-STEM disciplines that aim to train students. Programs may include training or certificates which are vocational in nature.
There is a current demand to revisit how a technique such as WE remains relevant to changing student demographics and the use of technology among these students. Students may use technology in new ways or to varying amounts outside of class, and consequently, students may desire more or less technology use in their courses. As this qualitative study provides insight for future quantitative work, it is also a call to forecast how WE can be used as higher education advances with technology.
There is a new technology which is being developed which uses the WE name. This process of transforming from a teaching process into a literal software is the framework for a new way to use WE. The manner in which this technology is being developed is one way in which the traditional teaching technique of WE has been moved into a new context, from the classroom into an entirely online format, underlining the significance of the use by one respondent of WE as software. Currently, the use of this software is undergoing change. The effort put into trying to transform WE into a software is significant because this effort reveals the extent to which WE can be used as a teaching method in new ways.
The hope is that this qualitative study provides insight into teaching methods regarding WE in order to provide avenues for future exploration. This research is especially important during a time when technology (such as MOOCs) is changing how courses are being taught, as well as which disciplines are being emphasized nationally in the United States. Perhaps future research can further investigate trends of teaching WE.
References
Atkinson, R., Derry, S., Renkl, A.,, & Wortham, D. (2000). Learning from examples: Instructional principles from the worked examples research. Review of Educational Research, 70(2), 181-214.
Atkinson, R. K., & Renkl, A. (2007). Interactive example-based learning environments: Using interactive elements to encourage effective processing of worked examples. Educational Psychology Review, 19(3), 375-386.
Barab, S., Dodge, T., & Gee, J. (2012a). The worked example: Invitational scholarship in service of an emerging field. Retrieved from http://workedexamples.org/mission
Barab, S., Akram, S., & Ingram-Goble, A. (2012b). Worked examples: An invitational scholarship for promoting scholarly dialogue. Retrieved from http://workedexamples.org/mission.
Boekhout, P., van Gog, T., van de Wiel, M., Gerards-Last, D., & Geraets, J. (2010). Example-based learning: Effects of model expertise in relation to student expertise. British Journal of Educational Psychology, 80(4), 557-566.
Bruner, J.S., Goodnow, J., & Austin, G. (1956). A study of thinking. New York: Wiley.
Carroll, C., Booth, A., & Cooper, K. (2011). A worked example of “best fit” framework synthesis: A systematic review of views concerning the taking of some potential chemopreventive agents. BMC Medical Research Methodology, 11(1), 29-37.
Carroll, W.M. (1994). Using worked examples as an instructional support in the algebra classroom. Journal of educational psychology, 86(3), 360-367.
Case, J., & Gunstone, R. (2003). Approaches to learning in a second year chemical engineering course. International Journal of Science Education, 25(7), 801-819.
Chi, M.T.H., Bassok, M., Lewis, M.W., Reimann, P., & Glaser, R. (1989). Self-explanations: How students study and use examples in learning to solve problems. Cognitive Science, 13(2), 145-182.
Clark, R.C., Nguyen, F., & Sweller, J. (2006). Efficiency in learning: Evidence-based guidelines to manage cognitive load. San Francisco: Pfeiffer.
Cooper, G., & Sweller, J. (1987). Effects of schema acquisition and rule automation on mathematical problem-solving transfer. Journal of Educational Psychology, 79(4), 347-362.
Crippen, K.J., & Earl, B.L. (2007). The impact of web-based worked examples and self-explanation on performance, problem solving, and self-efficacy. Computers and Education, 49(3), 809-821.
Darabi, A.A., Nelson, D.W., & Paas, F. (2007). Learner involvement in instruction on a complex cognitive task: Application of a composite measure of performance and mental effort. Journal of Research on Technology in Education, 40(1), 39-48.
Folkard, A.M. (2004). Mathophobic students’ perspectives on quantitative material in the undergraduate geography curriculum. Journal of Geography in Higher Education, 28(2), 209-228.
Gee, J. (2012). Working Examples – About. Retrieved from http://workingexamples.org/about
Glaser, R. (1976). Components of a psychology of instruction: Toward a science of design. Review of Educational Research, 46(1), 1-24.
Hilbert, T.S., & Renkl, A. (2008). Concept mapping as a follow-up strategy to learning from texts: What characterizes good and poor mappers? Instructional Science, 36(1), 53-73.
Hoogveld, A.W.M., Paas, F., & Jochems, W.M.G. (2005). Training higher education teachers for instructional design of competency-based education: Product-oriented versus process-oriented worked examples. Teaching and Teacher Education, 21(3), 287-297.
Kalyuga, S., Chandler, P., Tuovinen, J., & Sweller, J. (2001). When problem solving is superior to studying worked examples. Journal of Educational Psychology, 93(3), 579-588.
Kopp, V., Stark, R., Heitzmann, N., & Fischer, M.R. (2009). Self-regulated learning with case-based worked examples: Effects of errors. Evaluation and Research in Education, 22(2-4), 107-119.
Kostons, D., van Gog, T., & Paas, F. (2012). Training self-assessment and task-selection skills: A cognitive approach to improving self-regulated learning. Learning and Instruction, 22(2), 121-132.
Mayer, R.E. (1997). Multimedia learning: Are we asking the right questions? Educational Psychologist, 32(1), 1-19.
McKeachie, W.J., & Svinicki, M.D. (2011). McKeachie’s teaching tips: Strategies, research, and theory for college and university teachers. Belmont: Houghton Mifflin.
Meier, D.K., Reinhard, K.J., Carter, D.O., & Brooks, D.W. (2008). Simulations with elaborated worked example modeling: Beneficial effects on schema acquisition. Journal of Science Education and Technology, 17(3), 262-273.
Miller, D. (2010). Using a three-step method in a calculus class: Extending the worked example. College Teaching, 58(3), 99-104.
Miller, G.A. (1956). The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review, 63(2), 81-97.
Moore, M., & Sapp, T. (2013, May). STEM defined: Understanding the implications of STEM classification systems. Electronic paper presented at the meeting of the Association of Institutional Research, Long Beach, CA.
Mwangi, W., & Sweller, J. (1998). Learning to solve compare word problems: The effect of example format and generating self-explanations. Cognition and Instruction, 16(2), 173-199.
National Science Foundation. (2012). POGIL-Process Oriented Guided Inquiry Learning. Retrieved from http://www.pogil.org/
Obama, B. (2011). State of the Union Address. Retrieved from http://www.whitehouse.gov/the-press-office/2011/01/25/remarks-president-state-union-address
Paas, F., & van Merrienboer, J. (1994). Variability of worked examples and transfer of geometrical problem-solving skills – a cognitive-load approach. Journal of Educational Psychology, 86(1), 122-133.
Reed, S.K., & Bolstad, C.A. (1991). Use of examples and procedures in problem-solving. Journal of Experimental Psychology-Learning Memory and Cognition, 17(4), 753-766.
Renkl, A. (1997). Learning from worked-out examples: A study on individual differences. Cognitive Science, 21(1), 1-29.
Salden, R., Aleven, V., Schwonke, R., & Renkl, A. (2010). The expertise reversal effect and worked examples in tutored problem solving. Instructional Science, 38(3), 289-307.
Schworm, S., & Renkl, A. (2006). Computer-supported example-based learning: When instructional explanations reduce self-explanations. Computers & Education, 46(4), 426-445.
Shen, C. and O’Neil, H. (2006, April). The effectiveness of worked examples in a game-based learning environment. Paper presented at American Educational Research Association, San Francisco, CA.
Shuell, T.J. (1996). The role of educational psychology in the preparation of teachers. Educational Psychologist, 31(1), 5-14.
Stark, R., Mandl, H., Gruber, H., & Renkl, A. (1999). Instructional means to overcome transfer problems in the domain of economics: Empirical studies. International Journal of Educational Research, 31(7), 591-609.
Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12(2), 257-285.
Sweller, J. (1994). Cognitive load theory, learning difficulty, and instructional design. Learning and Instruction, 4(4), 295-312.
Tarmizi, R.A., & Sweller, J. (1988). Guidance during mathematical problem-solving. Journal of Educational Psychology, 80(4), 424-436.
Tuovinen, J.E., & Sweller, J. (1999). A comparison of cognitive load associated with discovery learning and worked examples. Journal of Educational Psychology, 91(2), 334-341.
van Gog, T., Paas, F., & van Merrienboer, J.J. (2008). Effects of studying sequences of process-oriented and product-oriented worked examples on troubleshooting transfer efficiency. Learning and Instruction, 18(3), 211-222.
Ward, M., & Sweller, J. (1990). Structuring effective worked examples. Cognition and Instruction, 7(1), 1-39.
Woltz, D.J., Gardner, M.K., & Gyll, S.P. (2000). The role of attention processes in near transfer of cognitive skills. Learning and Individual Differences, 12(3), 209-251.
This academic article was accepted for publication after a two-round blind peer review involving five independent members of the Reviewer Board of the International HETL Review (IHR), and three subsequent revisions. Editors: Beth Kalikoff, University of Washington (USA), and Angels Velez-Solic, Indiana University Northwest (USA), members of the IHR Editorial Board.
Suggested Citation
Ryland, A. (2013). Worked Examples: Teacher Practices. International HETL Review, Volume 3, Article 8, URL: https://www.hetl.org/academic-articles/worked-examples-teacher-practices
Copyright © [2013] Austin Ryland
The author(s) assert their right to be named as the sole author(s) of this article and the right to be granted copyright privileges related to the article without infringing on any third party rights including copyright. The author(s) retain their intellectual property rights related to the article. The author(s) assign to HETL Portal and to educational non-profit institutions a non-exclusive license to use this article for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. The author(s) also grant a non-exclusive license to HETL Portal to publish this article in full on the World Wide Web (prime sites and mirrors) and in electronic and/or printed form within the HETL Review. Any other usage is prohibited without the express permission of the author(s).
Disclaimer
Opinions expressed in this article are those of the author, and as such do not necessarily represent the position(s) of other professionals or any institutions.
