Wayfinder Learning Lab

"Learning is about living, and as such is lifelong." Elkjaer.


3 Comments

Making Feedback Visible: Four Levels in Action

Five years ago I was starting to become concerned with the difference between marking and feedback. What was making a difference to my students’ learning and was the effort I was putting into detailed marking worth it in terms of their improvement? In reading Hattie’s Visible Learning for Teachers, Wiliam’s Embedded Formative Assessment and the pdf of The Power of Feedback (Hattie & Timperley), I developed a four-levels feedback template for use on student work.

This post is to share an updated version – I still really like this method of giving timely, actionable, goal-focused and student-owned feedback. It definitely saves me time, but puts the focus of feedback on what’s most important for the student to take the next step. I’ll keep updating, editing and adding to this post.

When giving feedback on a piece of work, I paste this at the top of the student’s assignment, give some comments in the work and check their self-assessed rubric. Before we open individual feedback, I summarise whole-class feedback.

A copyable GoogleDoc version of the grid (and teacher explanation) is here, and to export it as pdf, click here.

……….o0O0o……….

Why present feedback this way?

Screen Shot 2013-11-03 at 9.22.37 PM

Hattie & Timperley, Four Levels of Feedback. Click for the pdf of ‘The Power of Feedback.’

Feedback addresses three questions:

  • Where am I going?
  • How am I going?
  • Where to next?

Feedback is timely, actionable and needs to be more work for the learner than the teacher.

  • Clarity of achievement so far: goal-referenced, tangible & transparent.
  • Understanding “the gap” between where the learner is and where they need to go next (not necessarily the top bands)
  • Timely. Using a system like this saves time   in grading/giving feedback, makes it more  accessible to digest (is user-friendly) and can be easily reviewed for the next time the student works towards similar goals.
  • Feedback first, then grades. Not presented together, to enforce student reflection & action.

 

Making The Four Levels Work

  1. Goals and outcomes need to be clear – do students & teachers have a shared understanding of what success looks like at different levels of achievement?
  2. Feedback needs to be ongoing. Students are taught to self-assess in the drafting stages and feedback (not grading) given on the drafts with plenty of time to take action before submission.
  3. Students self-assess before submission. Even better – they can peer-assess and give feedback. If tasks are differentiated, this does not present a collusion challenge.
  4. Teacher gives feedback in the grid, on the front page of the work (or in an accessible place):
    1. Check the student’s self-assessment against descriptors
    2. Check the assignment, making comments only on actionable next steps – not an overwhelming number, as this can increase the perceived “gap” for students. Students who want and will take action on very detailed marking can request this in follow-up.
    3. Summarize feedback in the grid: task-level, process level and self-regulation level.
    4. Link to support resources where appropriate
    5. Record grades out of sight of student.
  5. Teacher places value on interaction with feedback by giving class time to digest & reflect
    1. Give “whole class” feedback on common issues and note needs for later workshops
    2. Students read their feedback: table and comments.
    3. Students synthesise this into a “feed-forwards” note to self. Showing this to the teacher and a shared agreement on the next steps releases the grade, not before.
  6. Next time the task type is attempted, the first thing students do is open the feedback and set achievable, specific goals to “level up” based on the feedback & feed forwards.

……….o0O0o………..

Reflections in Practice

I worried initially that the pushback from students would be that I wasn’t grading enough. This didn’t happen for a couple of reasons:

  1. We made explicit the reason for doing this and I keep no secrets about the “magic” of learning from students. I explain and demonstrate what works in learning and why we do things this way.
  2. Most students like seeing the next steps really clearly. We’re not all aiming for top levels right away – we’re aiming for progress upwards.
  3. We talk about “the gap” a lot, and our quest to close the gap in prep.
  4. I already know what the grades are likely to be, as we invest time in class for drafting, feedback and conferencing. I expect students to show their work and take action on feedback.
  5. I will happily take a piece of work back and sit with a student, giving really detailed marking and justification if they request it. This rarely happens and it is usually one or two who are working at the very top of the rubric. This is far more efficient and effective doing acres of marking for large classes, the bulk of which won’t have an impact.

References

Hattie, J. & Timperley, H. (2007) The Power of Feedback. Review of Educational Research. Vol. 77 no 1 (pp 81-112). https://www.jstor.org/stable/4624888 (includes diagram above)

Wiggins, G. (2012) Seven Keys to Effective Feedback. Educational Leadership Magazine. Vol. 70 no. 1. (pp 10-16). www.ascd.org/publications/educational-leadership/sept12/vol70/num01/Seven-Keys-to-Effective-Feedback.aspx   (and related: EL Takeaways Poster http://inservice.ascd.org/seven-things-to-remember-about-feedback )

Dylan Wiliam Centre: Ten Feedback Techniques That Make Students Think (poster). https://www.dylanwiliamcenter.com/wp-content/uploads/2015/02/10-Feedback-Techniques.pdf

……….o0O0o………..

More Resources on Feedback & Grading

 


5 Comments

Webb’s DOK4 & Transfer

115013bI recently took part in a fabulous Bold Moves Curriculum Mapping Bootcamp, by Dr. Marie Alcock at ISKL. I was there to think about next steps for curriculum planning at CA, and it was a great opportunity to pick the brains of a true expert (and get lots done). I like the bootcamp model for PD: short, focused and with the opportunity to take immediate action with great feedback from colleagues in similar positions.

DOK is not a wheel of command terms

dokwheel

Not a Wheel. [John R. Walkup]

Through one of the discussions about high-quality assessment, Marie dug into Webb’s Depth of Knowledge (DOK) framework. She asserted that it’s not a “wheel” of command terms as is often presented, but a way of framing how deeply students need to know and use information, skills and concepts.

Similarly, DOK is not the same as Bloom’s Taxonomy, and is not a pyramid or a hierarchy of knowledge that “peaks” at DOK4. DOK4 can be accessed from any of the other three levels, and effectively sits in parallel. For a decent explainer of how DOK levels work, see this by Erik Francis for ASCD Edge – I used his DOK descriptors in my rough teacher plansheet tool below.

In practical terms, as explained by Marie, students should be able to access DOK4 from any one of the other DOK levels. This means that DOK4 can act as a filter for transfer.

How else can the student use the knowledge, skills and content at this level? 

So… in curriculum and task design and differentiation, teachers can set up situations for all students to pull their learning (even if only at a recall/DOK1 level) through to DOK4 by applying it in a new context – as long as it is the same skill/target. For example, this might mean taking a scientific skill and applying to a new experiment, or a writing technique applied to a new genre. This is knowledge augmentation.

MYP Teachers will see the immediate connections here to level 7-8 objective descriptors in the criteria (“correctly applying x in unfamiliar contexts”). This calls for some careful task design.

……….o0O0o………..

Teacher Plansheet: A Practical Use

Transfer is a notoriously difficult skill to teach, even though it is included in the ATL framework, and so I sketched up this planning tool (pdf) in the hope that it can visualise how DOK4 can be used as a filter to make transfer explicit. Follow the arrows as you think about putting a target standard or learning outcome to work. What level (DOK1-2-3) is expected of the student? How else (DOK4) could it be used? For some excellent, practical resources on applying DOK in the various disciplines, check out Dr. Karin Hess’s Cognitive Rigor and DOK rubrics and resources.

……….o0O0o……….

Transferring the Transfer: Thinking Collaboratively

How else might this tool be put to use? Here are some quick thoughts on how this might work with the collaboration of the relevant experts or coaches in the school.

  • Technology Integration: using the DOK4 filter as an opportunity to amplify and transform (RAT model) the learning task (but still meet objectives).
  • Service Learning: In moving from “doing service” to service learning, could this be used to help frame students’ focus on planning, or post-service reflection? As students learn about issues of significance, how can they put it work through transfer to meaningful action? As they reflect on their learning, can they connect new and existing disciplinary knowledge?
  • Interdisciplinary Learning: How can students take their learning and use it meaningfully in a context that requires transfer between disciplines?

……….o0O0o……….

 


8 Comments

Capturing the Curriculum, Criteria & “Zooming In”

Shortlink to this resource: is.gd/mypassess

Update Sept. 5 2017 based on edits summarized in this update from the IB.

Big Update Dec. 2 2017: added subject group overview (curriculum articulation) tabs for each subject group, with data validation for key, related concepts, ATL. 

If you find this useful, please consider making a donation to one of my chosen charities through Biology4Good

……….o0O0o…………

Capturing the Criteria (and the Curriculum)

After some parent-teacher conferences recently, I was asked to show all of the MYP assessment criteria together and realised I couldn’t find something that met our needs for a single-reference, quick overview of the MYP assessment objectives and criteria.

Screen Shot 2017-04-27 at 17.52.59Here is an attempt to put the big ideas and rubrics together in one place, so that colleagues can quickly see vertical and horizontal articulation and connections, and so that parents have a resource to hand to help understand assessment.

You might find it useful.

To make your own copy, click “File –> Make a copy”.

This slideshow requires JavaScript.

 ……….o0O0o………..

Disclaimers:

  • This involved a lot of clicking and is bound to have some errors. Big thanks to Mitsuyo-san, our data secretary, who helped with this. 
  • Descriptors in bold did not make it across from text to spreadsheet. Use original descriptors in student assignments.
  • This is intended only as an overview of the programme. Teachers must exercise caution with this, and default to the published guides on the OCC for assessment rubrics, clarifications, rules and guidance.

………o0O0o……….

Zooming In: Focus on what’s important in assessment

Edit: added 3 May 2017

Why the green bands? 

In each of the subject-area bands, you’ll find the Level 5-6 row accented with green. This is part of something I’m trying to work on with colleagues and students in terms of zooming into the objectives-level of assessment, and was something I used in #HackTheMYP.

The basic idea is this: 

  1. As a model of a 4-band rubric, we typically see the third band as ‘meets objectives‘. This means that the rows below are approaching and above are exceeding.
    • Try it: add up the scores for all 5, all 6 or a combination thereof. What does it come to when you apply the total to the 1-7 conversion chart? This is the kid that meets the outcomes of our core curriculum. 
  2. When we focus only on the top-band descriptors we may inadvertently end up doing one of two things:
    • Causing students to get stressed by default as they’re aiming for the ‘exceptional’ descriptors first. “The gap” between where they are and want to be is too big; or,
    • Falsely making our core expectations for all students fit the 7-8 band, thus leaving nowhere to go from there – creating a “low ceiling” and no room for extension into genuinely meeting those top descriptors.
  3. If we zoom into the 5-6 band first – in task design and as a student – we are able to set an appropriate expectation for all learners, see how and where to scaffold and support those who need it, and provide a “high ceiling” for innovation, application, analysis, synthesis, etc.
  4. It should then become easier to create the task-specific clarifications. If we can clearly describe the 5-6 “core” band first, we should then make sure that the levels above and below can be really clearly distinguished. In my experience, this is easier than starting at the top and working back.

If you’ve tried this idea (or similar), how did it go?

……….o0O0o……….

For a similar discussion and great resources, but in an SBG context, check out Jennifer Gonzalez’s (@cultofpedagogy) posts on the “single point rubric”:


2 Comments

Standardization: Cycling away from the moderation “event”

A quick post to share a resource, based on some of our work at CA. I love cycle diagrams and was thinking about the process of moderation, planning and the challenges of effective collaboration when there are grades (and a big pile of ‘done’ grading) at stake. 

If you’ve ever tried to ‘moderate’ work that’s based on two or more teachers’ hours of effort in grading, you’ll recognise the challenge. The proposal here is to reframe stadardization as a cycle – various points of entry to working together on a common understanding of assessment – so that teachers align their assessment standards more closely. Post-hoc moderation events may tend towards defense of our own grading work; who wants to go back and change all that work?

Do you think you could put the cycle to work in your own context? 


2 Comments

The IMaGE of an International School

It’s crunch time for my MA International Education studies at the University of Bath, with a big literature review in progress and some data collection coming up, aiming to submit by the summer break. As much as I’ve loved the study, I’m looking forward to reclaiming some balance. 

………o0O0o……..

My plan for the dissertation is to update and pilot-test my web-chart of the international dimension of a school, aiming to tackle the challenge of defining a nebulous concept through visualisation, based on self-reporting, to generate “the IMaGE of an international school“. (IMaGE = international mindedness and global engagement). The small-scale case-study will generate an IMaGE for my own school, and the pilot study will help evaluate the usefulness of the visualisation and metrics.

Web8Sample (2)

A sample of the web chart in use, with the IMaGE showing the evolution of a school or a change in perception. The eight radials are still under development, and there will be descriptors for each in the final project. At first glance, where would you rate your won school? What do you see, think, wonder about the results of this (imaginary) school?

The idea of trying to evaluate or measure the ‘internationalisation’ of a school is not new: we already have metrics, practices or handbooks from various organisations, including the IB, CIS, ISA, ACE, OECD. This project aims to learn from, adapt and distil these qualities into an accessible tool that will generate a ‘visual definition’ for a school, as a starting point for further investigation.

Although some of the ideas within the chart have evolved a lot since the initial idea in 2012 (and I have found many more studies), here is the original assignment.

 

 

 

 


4 Comments

How NOT to be ignorant about the world.

Hans Rosling, TED.com

“Fame is easy to acquire. Impact is much more difficult.” 

Update, 2018: As a Rosling fanboy, using their work on i-Biology since about 2008, I was saddened to hear of his death in 2017. However, his recent book #Factfulness, is fantastic and well worth reading. More below. 

……….o0O0o……….

Another great Hans Rosling TED Talk, this time with his son, Ola.

Here Dealing with misconceptions, bias, ignorance of global issues and a little formative assessment*, they discuss how we can be better informed about the world, with a fact-based world view… and how we could (eventually) perform better than chimps on a global issues quiz. I have blogged about how this might be used in IBTOK or science classes on i-Biology.

……….o0O0o……….

Should a fact-based world view be the core curriculum of an international school?

Early in the talk, Ola recognises the influence of early bias of students and outdated curricula on the world view held by students – and how these are compounded by an ill-informed media. Through their project, they are trying to measure these misconceptions and propose a ‘global knowledge certificate’ that candidates (or organisations) might use to stay informed, to be competitive and to think about the future.

It seems to me that the fact-based world view would make for an excellent set of content-knowledge standards for an international school, and might pair nicely with the IB programmes as we seek to create knowledgable young inquirers who seek to make a positive difference to the world around them. How can they achieve this if they are learning outdated concepts of development or using stereotypes to paint the world in an ugly shade of ill-informed?

Hattie’s meta-analyses note that the power of prior learning (including prior mis-learning or misconception) has a very high impact on students’ future learning (d=0.67). As we generate scopes and sequences for courses or set up units of inquiry, should we be looking to the research not only on misconceptions in our own content domain but in global literacy in order to give students the tools they need to inquire in a changing and often-misunderstood world?

Is globally-literate the same as internationally-minded?

It is hard to define international-mindedness, though we can recognize it in our own settings. We might observe the behaviours of a globally-engaged student (or teacher), and might use assessments of students’ fact-based world-views as a measure of their international-mindedness. To this end, a globally-focused national school might be a more effective ‘international school’ than a more narrow-focused overseas expatriate school.**

You read about the ignorance project here on CNN, or find more classroom resources (including a world-view card game) on Gapminder’s education page. The Guardian also has a selection of global development quizzes, which you can take for fun or in class.

……….o0O0o……….

*making great use of the audience-response clicker system pioneered by Eric Mazur.

**this is part of the idea of my web-chart of the IMaGE (IM and Global Engagement) of a school in my MA work.

……….o0O0o……….

“Please return your brain for a free upgrade.”

Edit 2018: Rosling’s posthumously-published book (Factfulness: Ten Reasons We’re Wrong about the World – and Why Things Are Better Than You Think)is out. Here is an excerpt in the Guardian.


4 Comments

Faculty PD: Assessment Principles & Practices (and a stretched golf metaphor)

Over the last couple of weeks we’ve put a lot of work into developing Wednesday afternoon PD sessions for middle and high-school faculty on Assessment Principles and Practices. We’ve chosen to do this now, as this is an easy entry point for work on MYP: Next Chapter and it is always valuable PD to think about, evaluate and strengthen our practices. It builds on a lot of the good work that CA has been doing in recent years to improve assessment.

The inspiration for the theme came from Ken O’Connor’s (@kenoc7) blog post on “23 reasons why golf is better than classroom assessment and grading,” as well as some of Rick Wormeli’s (@RickWormeli) great series of videos on Assessment in the Differentiated Classroom. The aim was to emphasise the importance of the connection between the objectives of the unit and the assessment tasks, resulting in strong, worthwhile assessment taking place. As part of the discussion we connected the objectives of the MYP subjects to their respective assessment criteria and strands.

So far we have completed two of three (or more) sessions on this, with the first being a general overview, including revisiting our Assessment Policy and a Socrative Space Race, as well as sharing with colleagues from different departments. The second session focused on scaffolding tasks and was kicked off with Wormeli’s provocative talk on redos and retakes, before having some exemplary teachers show their scaffolding and student support tools to teachers. The second half of the session was devoted to further developing our own tasks and in later sessions we’ll evaluate these and think more carefully about what to do with ‘broken’ assessments and how to make best use 0f learning data.

The curriculum team, including Tony (@bellew), LizD (@lizdk), LizA and I were impressed by the feedback given in a one-minute essay between sessions and in the quality of collegial conversations taking place. It is clear that CA has come a long way in assessment philosophy and practices in recent years.  I am grateful to be in a place where we can work towards progress, share our practice and improve together as a faculty.

Here is the presentation. Apologies for stretching the golf metaphor to breaking point, but I wanted to also use it as a way to model use of CC images from Flickr. I’m trying to find a happy medium here between attractive ‘presentation zen’ for the PD sessions and functional informational flipbooks for teachers to refer to and use in their later work as they’re embedded to the faculty guide.