Using Student Learning Data to (try to) Improve Student Data Processing: A Department Goal

This year saw the start of a school-wide push to become more data-driven in our decision-making and evaluation. As a result, one of our goals as teachers at the start of the year was the Student Learning Goal, a target set by groups or individuals based on data collected in school. As a science department we opted to investigate Criterion E: Data Processing in the MYP. We picked this criterion as it is something we have all experiences student difficulty with, it feeds directly into success at IBDP and even with the Next Chapter changes the skills and conventions are likely to remain intact as it is such a fundamental part of the scientific method. Data analysis on data analysis; that’s how we roll. 

Summary

• Our initial goal was to analyse the Criterion E scores given in 2011-12 and take actions to improve them in 2012-13, though we realised early on that there were too many uncontrolled variables within the department to make comparisons valid and reliable. Scientists, eh?
  • As a result we shifted this year’s focus to setting up a greater vertical departmental understanding of the elements on Criterion E, as well as preparing resources and exemplars for students.
  • We are confident that the semester 2 data from this year are more reliable, and so will be useful in comparison next year (2013-14) as we take action on our work.
• To work towards this common goal we needed to have a lot of discussions about data processing, presentation and analysis. This involved:
  • Identifying and discussing common student errors and misconceptions.
  • Unpacking the rubrics to make sure we all shared a common understanding of what is expected.
    • This included a lot of vertical discussion about what elements are appropriate for MYP 1 and 3, and I think this was the most powerful part of our work.
  • Identifying commonalities and differences in expectations in MYP and DP Biology, Chemistry and Physics courses in terms of conventions for data processing.
  • Moderating exemplars of student work.
  • Looking up research, journals or articles on student issues in data processing and sharing these with the group, to further discussion and develop strategies to use in our classes.
• Towards the end of the year we were able to produce a GoogleSlides set of resources to give common advice to students, with exemplars. This is to be copied and edited to MYP3 and MYP1, to meet the adjusted expectations of the interim objectives. As you can see, there is still some work to complete, though it will be ready for action with students in August.

Read more…

………o0O0o……….

Outcomes

It was essential that early in the process we got over the ‘students can’t do it’ discussions and started to really explore the why, from both sides: what do they find difficult and what can we do to teach it more effectively. We can’t fix the world in one academic year, but have made headway, and are looking forward to developing it further in 2013-14.

• We certainly worked more cohesively as a department in terms of vertical understanding of expectations and teaching these skills. The quality of collegial discussions was stimulating, and all were able to be involved, from Grade 6 to Grade 10/IBDP. This was of particular use across the MS-HS transition.
• It became apparent that we needed to do some work on clarifying the interim criteria for Grade 6, which will be done in time for
• The presentation above shows some of the process so far give us a clearer set of common understandings and exemplars to show students.
• Our expectations for lab reports in MYP 4-5 feed very strongly into good labs for IBDP. We were able identify areas where we were perhaps expecting too much from students in MYP4-5, in terms of meeting the descriptors. Many students exceed the MYP descriptors, and get the ‘6’; those whose work is of a lesser standard but still meets the descriptors still deserves a ‘6’.
• A new section on our department website for academic journals and papers, so that we can make better use of these resources in our discussions and practice.

It is clear that this criterion – and even the strands within the criterion – cannot be taught as a single entity. There are many sub-skills that need to be explicitly taught and practiced, including:

• Distinguishing between quantitative and qualitative data and presenting both appropriately.
• Conventions in tables and graphs; titles, units, axes, layout.
• Choosing and using appropriate table structures and graphical representations.
• Appreciating and noting uncertainty and error.
• Appreciating and evaluating reliability and validity in our data and their ability to support a conclusion.
• Using and presenting appropriate significant digits.
• Choosing, correctly applying and presenting calculations and examples.
• Identifying trends, patterns and relationships in data sets.
• Explaining the science behind our results.
• Evaluating hypotheses based on our data, avoiding the temptation to think of our predictions or data as being ‘right or wrong’.

Literature in science education is rich with ideas and research on the many elements identified above. As we become more research-oriented as a department, we will have a lot of fun exploring how to strengthen our practices.

………o0O0o……….

Practices

As a result of our discussions we plan to implement the following in our classes, in an attempt to improve student data processing, presentation and analysis:

• Get back to pen/paper or whiteboards before jumping into Excel or graphing where possible. Students plotting graphs by hand, at least in the early stages of a lab (and before the final lab report), should help them get more scale competence and visualise patterns as they emerge.
  • This is a focus on processing: what do the numbers look like and what can we do with them? 
• Show exemplars and give better guidance or tutorials on how to make Excel work effectively. One example I have is the IBBio Statbook, which can be adapted. The purpose of this is not to take away the need for thought in how to present and process data, but to cut down the sometimes excessive amount of ‘clicky-clicky’ time that can dominate students’ attention in lab lessons.
  • This is a focus on presentation: how do we make the data look right in our lab reports so that they obey conventions and feed into IBDP? 
  • We can explore the use of GoogleSheets for this, especially in MYP1-3, where error bars are not needed.
• Explicitly teach and practice skills in identify and interpreting trends, patterns and relationships in the data.
  • This is a focus on analysis: how can we draw conclusions based on our data and how do we know how confident we can be in those conclusions? 
• Drafting stages of student work will allow us to head off problems before the final submission; I have found that students who get work in early are better able to self-assess and take action on the deficits they spot.
• Continue as a department to discuss data processing, presentation and research regarding students’ abilities and issues, and continue to build on this student learning goal in 2013-14.

………o0O0o……….

Influential Readings

• Hattie’s Visible Learning meta-analysis, of course.
• Qualitatve Data Analysis, and expanded sourcebook
• The ups and downs of interpreting graphs, by Peter Newbury (graphic below)
• Teaching the disciplines in the MYP: Nurturing big ideas and deep understanding (OCC, pdf, Sciences chapter)

……….o0O0o……….

Science Education Journals

I set this up for the department, but may as well share it here. The links below take you to a GoogleScholar search for each of a selection of science education journals. To search within the journal archive, click on the advanced search arrow in the search box and enter your search parameters. The results will show up, and in most cases you’ll get to read the abstract. If you have a department member who has university access they might be able to get you the paper.