Improving Measurement and Display of Student Behaviors with Rick Kubina
/Dr. Richard Kubina is Professor of Special Education in Penn State’s College of Education where he teaches courses on reading methods, informal assessment, and behavior analysis. Dr. Kubina also conducts wide-ranging research in the areas of Applied Behavior Analysis and Precision Teaching. Previously, he taught special education in schools specializing in post-acute brain trauma rehabilitation. Dr. Kubina is a Board Certified Behavior Analyst, Doctoral-level and has served as the editor of the Journal of Precision Teaching and Celeration.
3 Problems With Data Measurement in Special Education
Thriving Schools: Prof. Kubina, in the first part of our conversation you mentioned 3 big problems related to data measurement in special education. Could you recap for us what those were?
Rick: The 3 big problems that we started to discuss were: the use of operational definitions in IEPs, dimensionless quantities to track progress, and improperly displaying student improvement on graphs. As you’ll recall, we briefly discussed all three of these items. We also discussed in depth the ramifications of the first – problems around operational definitions.
"I really try to emphasize to teachers that when we do a few small things correctly – pinpointing behaviors, using good dimensional measures, and putting the results on a standardized ratio chart – you’re whole world will open up!"
THE PROBLEM WITH DIMENSIONLESS QUANTITIES (AND HOW TO FIX IT)
Thriving Schools: So let’s take a deeper dive into these second two issues. Can you remind us what a dimensionless quantity is and why it’s a problem?
Rick: The second big issue (that stems in part from our discussion of operational definitions) is how we count behaviors. Teachers, by and large, use what we call “dimensionless quantities.” An example of this might be a teacher monitoring a student’s “percent correct” on a particular task. And this causes all kinds of problems! For starters, it ignores time. For instance, let’s say you have 2 students with scores of 90% on some math work they’ve done. Which one is better? Well if the first student completed 9/10 problems in the same time it took the 2nd student to do 18/20, you’d say the 2nd student is doing way better (because they’re completing twice the number of problems in the same amount of time). So having these dimensionless quantities can mask the true nature of a student’s skill.
Thriving Schools: So how can teachers get better at counting behaviors?
Rick: The answer is using better units of behavior. I’ll give you four ways to achieve this! The first is to include frequency, a count of something over a given period of time. And this one is huge! If teachers more readily adopt frequency in measuring behavior, it’ll dramatically improve the work they’re doing with their students. Like we talked about before, a simple “% correct” doesn’t do that.
Thriving Schools: What are some good examples of using frequency to improve the measurement of behavior?
Rick: There’s something called Curriculum Based Measurement and DIBLES that many teachers already use. Those systems use frequency quite a bit! For example, “How many words can you read per minute?” And when we do this it allows us to compare a student’s progress to norms, other students, and their own performance over time. A lot of reading fluency measurements are like this. So teachers are already doing this to some degree, but they could benefit by thinking this way in all their measurements!
Thriving Schools: What are the other three units of behavior?
Rick: Next you have duration, which is just the extent of time in which you’re doing something. For example, let’s say a student has a stereotypic movement disorder which causes them to rock back and forth in their seat. The duration would indicate how long they’re doing that. Then we have latency, which is how long it takes a student to start a behavior. Let’s say I give the instruction, “It’s time to clean off our desks.” Latency would refer to the time it takes a student to actually start doing that. The last one is what we call “count only.” This one is simple in that you’re just counting how many times a particular behavior is happening.
A BETTER DISPLAY OF PROGRESS
Thriving Schools: Can you also remind us what the 3rd big problem with IEPs and progress monitoring was?
Rick: The third issue is how we graphically display the data for the behavior in question. Most people are accustomed to only using linear graphs, with time on the x-axis and a count on the y-axis. But this chart is very basic – it can only show us whether we have more or less of something. The problem with this is that it forces teachers into linear thinking. But behavior changes and skill acquisition don’t progress linearly. They’re better shown on what we call a ratio graph because it’s able to show the rate of change.
Thriving Schools: How do we explain this in a simple way that educators can understand and use?
Rick: Ratio charts display student behaviors proportionately, whereas, linear graphs do not. As an example, let’s say you have 2 individuals trying to lose weight. A man who weighs 250 pounds and a woman who weighs 150 pounds. If both lose 30 pounds, who’s achievement is better? While the absolute amount of weight lost by each individual is the same, proportionally, the woman’s weight loss is far more impressive.
Thriving Schools: Can you give us an example related to student performance?
Rick: Let’s say you have two students who just improved their math fluency task performance by 1 question. One student starts at just 1 question correct and goes to 2. The other student starts at 50 correct and goes to 51. Clearly, the change means very different things for these two students.
Thriving Schools: Why else should educators be using ratio charts?
Rick: If you’re using a ratio chart it’s so much simpler to quantify a student’s rate of change. Obviously, a simple linear graph is not geared to show you that and so the conclusions we end up making aren’t always optimal.
Thriving Schools: Anything else you’d like educators to know about ratio graphs?
Rick: The third item relates to making our graphs and charts standardized. Again, let me give you an example. When individuals go to have their EKG charts read, the graphs will all look the same. And we do that so that anyone who reads it will instantly know what that pattern is telling them. If, on the other hand, you’re using non-standard graphs that vary the scaling from one teacher to the next, you don’t have any way of realistically comparing the two.
In the ratio chart above, one can clearly see the rates of change for the 3 different groups (students with autism, students with intellectual disabilities, and total students with disabilities) have been remarkably different. Again, ratio graphs better depict and show these rates of change relative to linear charts.
Thriving Schools: What other suggestions do you have for teachers when they start to make these shifts? What problems should they anticipate?
Rick: We haven’t really talked in depth about interventions yet. But I should make clear that when you’re trying new things, if you’re not using good data science tools, it’s impossible to make correct conclusions about the resulting student results. So I really try to emphasize to teachers that when we do a few small things correctly – pinpointing behaviors, using good dimensional measures, and putting the results on a standardized ratio chart – you’re whole world will open up! You’re going to know exactly how well a given intervention is working.
Thriving Schools: And if we’re able to do these things, what would the outcomes be for our system?
Rick: Here’s the story I like to tell. When I was a kid I had a mini-bike where we basically rigged up an engine and installed it on the bike. What my parents did, though, is they installed a device called a governor that basically limited or controlled the bike’s speed. And I was pissed at them because I could never go more than 15 miles per hour. By analogy, I like to say our entire education system has a governor on it – we’re only going 15 miles per hour when our kids need us to be going 100 mph! Especially if we’re working with students who are 2, 3, or 4 years behind, they need us as teachers to be using the best systems that’ll allow them flourish. And I think when we start looking at things through that lens, we’re going to discover all kinds of interventions that we could change and improve.
Thriving Schools: Do you have any thoughts about the extent our superstar teachers might be following your recommended best practices on a more intuitive level? For example, they try something, it doesn’t work, and because they have a very keen sense of awareness, they’re able to try something different more quickly.
Rick: That’s a very interesting thought and I think I’d agree with it. If you’re a very good teacher and you discover something that works well, you’re going to use it with more of your students. But if you find that such an activity only works for 70% of your students, you’re going to dig deeper and look for something else that’ll help the other 30%. Often that means you’re going to trial, iterate, and personalize solutions for the remaining students. And they’re going to keep discovering more and more things that work.
RECOMMENDED RESOURCES
Thriving Schools: What books, videos, or resources do you like to recommend to teachers to help address the items we’ve chatted about today?
Rick: I know this will sound like a shameless plug, but I wrote a book called The Precision Teaching Book that covers all the items we’ve discussed today. And I would argue there’s nothing out there like it because nothing else has been published in this area for quite some time. On Facebook, there’s a group called the Standard Celeration Society, which allows teachers to connect with other educators worldwide and share very practical resources on these issues. And for teachers looking for specific recommendations on writing front, Prof. Shawn Datchuk is one of the amazing folks out there who has a foundation in the Precision Teaching framework and is doing some really amazing work to support teachers.
You can check out our writing intervention interview with Shawn by clicking here!