Use Webb’s DOK Levels to coach teachers, not critique or criticiz
Recently, I met with a school who is in “Targeted School Improvement” (or what I call “Double-Secret Probation” because you can be in targeted and comprehensive school improvement at the same time) according to the criteria of the Arizona Department of Education.
The school leadership team informed me one of the program “specialists” from the School Improvement Agency gave the school a poor evaluation because, according to the evaluators, “Teachers are only teaching at a DOK-1 level.”
What’s worse is the Arizona Department of Education, like numerous state and local education agencies, are using this document to judge the depth of knowledge teachers are teaching.
(I’ve written numerous articles on why the DOK Wheel is inaccurate and won’t go into detail here. If you’d like to know why it’s inaccurate, please read http://inservice.ascd.org/what-exactly-is-depth-of-knowledge-hint-its-not-a-wheel/ or http://inservice.ascd.org/why-the-d-o-k-wheel-does-not-address-depth-of-knowledge/).
I’m not surprised the Arizona Department of Education school improvement “specialists” are evaluating teachers on the DOK level they are teaching. Principals and instructional coaches have been doing it for years ever since we educators were first introduced to Norman Webb’s concept over ten years ago as part of our training on the Common Core State Standards. I have heard numerous stories of school administrators who have directed their teachers to teach only at a DOK-3 or DOK-4 and evaluators who have penalized teachers for their lesson only being a DOK-1 or DOK-2.
It’s bad enough local education agencies and their site administrators are using the DOK Wheel and evaluating the DOK Levels of a teachers’ instructional delivery on one specific lesson on one particular day. Now a state education agency like the Arizona Department of Education is evaluating teacher effectiveness and overall school performance of a school in improvement status based on the DOK level of a single lesson they observed during a compliance monitoring – and the school is being penalized for this.
My worst nightmare has come true.
It’s bad enough we educators have been so grossly misinformed about depth of knowledge and Webb’s DOK Levels thanks to the DOK Wheel of Misfortune. It’s bad enough so many of us educators have been using this to develop and deliver our instruction. Now our schools are being evaluated and penalized using this false document.
Webb’s DOK Levels is a resource for coding and comparing the alignment of the alignment between the learning expectations of standards and the assessment items that address them (Webb, 1999). That’s why Webb (1997) initially created depth of knowledge – as a criterion (one of 12) for alignment studies.
Webb’s DOK Levels can also be superimposed with to determine the cognitive rigor of curricular activities and test items (Hess et al, 2009a, 2009b). To determine the cognitive rigor, first identify the cognitive action verb (or verbs) that indicate the level of thinking – or behavior – students must demonstrate and where they are categorized in Bloom’s Revised Taxonomy. Then look at the words and phrases that clarifies the context in which students will perform the behavior. That determines the cognitive demand and designates the DOK level. The Hess Cognitive Rigor Matrix is an effective and excellent tool to determine the cognitive rigor of learning expectations (Hess, 2013).
Webb’s DOK Levels can also be used as a model for teaching and learning that sets the ceiling of assessment based on the most cognitively demanding objective within the learning expectation of a standard, identifies potential gaps in learning, and guides students to demonstrate and communicate their learning up to and beyond the standard (Francis, 2016; Francis, unpublished – for now).
Webb’s DOK Levels should NEVER be used like this.
Teaching and learning for depth of knowledge is not Dancing with the Stars (or DOK with the Staff) where you’re up in front of the room dancing (teaching) your heart out and the judges (the evaluation team consisting of the principal, assistant principal, and / or instructional coach) judge (evaluate) your performance (teaching) like this. Webb’s DOK Levels should NEVER be used as a resource or tool to evaluate teacher effectiveness.
However, Webb’s DOK Levels can be an excellent resource and tool to coach and support teachers in developing and delivering deeper teaching and learning experiences for their students. To coach teachers on how to develop and deliver teaching and learning experiences for depth of knowledge, we must have a deep understanding and awareness of what exactly depth of knowledge is.
Depth of Knowledge – and Webb’s DOK Levels – are all about context. They designate the depth and extent students must demonstrate and communicate their learning in a given teaching and learning experience. Also, one DOK level is not better or more ideal than the other. Each DOK level has its focus and purpose in the scope and sequence of a teaching and learning experience.
A DOK-1 teaching and learning experience is when students develop and demonstrate foundational knowledge and functional understanding of concepts, content, and procedures. This is where and when students acquire and process the data, facts, information, and procedures they need to know, understand, and use to address and respond to simple and complex questions, problems, tasks, texts, and topics. It’s also when and how we teachers assess background knowledge and check for basic understanding. Do students have the knowledge? Do they understand the concept or content? Can they use the procedures to answer simple questions, solve simple problems, or complete simple tasks that have only one correct answer, outcome, result, or solution. That’s teaching and learning at a DOK-1.
A DOK-2 teaching and learning experience is when students demonstrate and communicate their conceptual and procedural understanding. Students are challenged to establish and explain reasons, relationships, or results using their knowledge and skills. The emphasis is whether students can demonstrate and communicate not only correctly but also clearly how to apply subject-specific knowledge, concepts, and skills or use information and basic reasoning to achieve and explain specific answers, outcomes, reasons, results, or solutions.
A DOK-3 teaching and learning experience is when students examine and explain answers and arguments – be it their own or those made by others – with evidentiary support. Students must think strategically how they could use different methods, strategies, or techniques to attain and explain answers, outcomes, results, or solutions. Students are also engaged to use complex reasoning supported by evidence to defend, explain, justify, or refute arguments, claims, conclusions, decisions, hypotheses, ideas, or reasoning. The emphasis is how well students can use the knowledge and skills to explain and justify or validate reasoning and results.
A DOK-4 teaching and learning experience takes not only demands extensive thinking but also time to address and respond to the question, item, problem, or task. This is when students are encouraged to transfer, use, and share what they have learned in different contexts, new situations, or to address, explain, and respond to a real-world scenario or situation. Teaching and learning will go deep within a subject area, across the curriculum, or beyond the classroom.
Yes, the complexity and demand of DOK-3 and DOK-4’s make them “quality” learning experiences and evidence of “good teaching”. However, they are not “better” or even “more difficult” than a DOK-1 or DOK-2. In fact, a DOK-1 or DOK-2 might be harder than a DOK-3 or DOK-4 as students engage in productive struggle and creative frustration to develop and demonstrate the foundational, conceptual, and procedural knowledge and understanding of subject-specific texts, topics, and techniques. DOK-3 and DOK-4 learning experiences may become easier than the DOK-1 and DOK-2’s as students demonstrate and deepen their knowledge, understanding, and awareness and develop their knowledge and skills into personal expertise.
The DOK levels and the descriptors of each level SHOULD NOT be used, however, as a rubric for evaluating teachers. However, they can be used as a guide to help teachers understand develop and deepen their instructional delivery. For example, if a teacher is teaching at a DOK-1 or DOK-2 – AND THAT’S OKAY! – we can coach and support teachers to expand their instructional delivery to a DOK-3 by asking them to consider how they can engage their students to explain and justify an answer or argument. We can also coach and support our teachers to extend their instructional delivery to a DOK-4 by prompting them to consider how they could connect the academic text, topic, or technique to another topic within the subject area, to another content area (e.g. How could the standard units of measure or the metric system be used in science or geography? How could statistics be used in history / social studies when doing polls or surveys? What’s the impact on or influence between science fiction and science and history – or vice versa?). Coach and support teachers in taking their teaching to a deeper level. Don’t penalize them if they aren’t or don’t because maybe they don’t understand – just like our students might not understand what we’re teaching them.
The coaching and planning conversations between educators should address the following:
- What is the goal of the learning experience?  Ask the teacher to clarify the student learning expectation.   Is it for the student to develop and demonstrate foundational knowledge and functional understanding by recalling information or procedures (DOK-1)?  To demonstrate and communicate conceptual and procedural understanding by applying concepts, knowledge, concepts, and skills or using information and basic reasoning to establish and explain reasons, relationships, or results (DOK-2)?  To think strategically or use  use complex reasoning supported by evidence to examine and explain answers and arguments?  To think extensively how they could transfer, use, and share their learning in different contexts, new situations, and over an extended period of time (DOK-4)?
- What is DOK level the learning expectation of the standard? Â The learning expectation of a standards can include multiple objectives that demand students to demonstrate their learning at different levels of complexity. Â Which part of the standard is being addressed? Â How did you deconstruct the standard to determine the DOK level? Â Â What is their understanding of depth of knowledge and Webb’s DOK Levels?
- What DOK Level do you feel the learning experience was and why?  This can be a very sensitive conversation because many educators see the DOK Levels as a value judgment, which it’s not.  Why does the teacher believe their lesson was at a certain DOK level?  Â
- What’s the next step in the DOK teaching and learning experience?  If the learning experience is a lower DOK level – AND THAT’S OKAY! – how will students be engaged and encouraged to expand and extend their learning?   How could you as the coach support the teacher in getting to the next level – or way beyond?
We need to have these conversations about depth of knowledge and check for understanding of Webb’s DOK Levels to make sure we’re both on the same page.
If you’re a school leader who wants to make depth of knowledge and Webb’s DOK your schoolwide initiative this year, please do not say this to your teachers at the staff meeting at the beginning of the year.
You are placing unreasonable expectations on your teachers. Demanding a teacher to make every lesson a DOK-3 or DOK-4 is not only impossible but also unnecessary. The majority of the learning expectations of the academic standards addressed in the classroom and the items on the state assessments are at a DOK-2. You will also deny yourself essential information about how and why your students are performing. The DOK-1 or DOK-2 learning experience can be used as a check for understanding and to identify and fill potential gaps in learning if your students are not meeting the standard at its DOK ceiling of assessment.
(This from the Arizona Department of Education’s blueprint of their AZMERIT literacy assessment. Notice how the majority of the test items are at a DOK-2 and only the writing items are coded a DOK-4. Check your state’s assessment blueprint to see exactly what percentage or how many items demand students to demonstrate their learning at a certain DOK Level.)
Depth of knowledge and Webb’s DOK Levels can be a useful framework to develop and deliver deeper teaching and learning experiences for our students. However, it must not only be implemented correctly but also supportively. To evaluate educators using Webb’s DOK Levels is not only inappropriate but also unfair. Some lessons will be a DOK-1 and DOK-2 – AND THAT’S OKAY! However, don’t critique or criticize teachers based on the DOK level of their instruction or lesson using Webb’s DOK Levels. Coach and support.
Erik M. Francis, M.Ed., M.S. is an author, educator, and speaker who specializes in teaching and learning that promotes cognitive rigor and college and career readiness. . He is also the author of Now THAT’S a Good Question! How to Promote Cognitive Rigor Through Classroom Questioning published by ASCD. He is also the owner of Maverik Education LLC, providing academic professional development and consultation to K-12 schools, colleges, and universities on developing learning environments and delivering educational experiences that challenge students to demonstrate higher order thinking and communicate depth of knowledge (DOK). He has also been named one of World’s Top 30 Education Professionals for 2019 by the research organization Global Gurus.
