Do skills tests work?

As I've written about before, a large part of the students' grades this year in Algebra 2 are based off of the skills tests. The method I'm using is based of off Dan's, but I've modified it quite a bit. I'll save reflecting on the details of the method, and what should be kept/changed for the end of the year. I'm still getting a feel for the process, and what I've been doing has worked well enough that I don't want to significantly alter it until next year.

The crux of the method is that students are primarily assessed on smaller bits of information, more frequently. They are also encouraged to try and try again at the same concepts until they master them. Since students learn at different rates, and have different things going on in their lives that may prevent them from learning at a certain point in time, they can relearn and retake the skills tests whenever they want, before the end of the semester.

Instead of assessing each skill individually, I've been grouping them into clusters of 4 or 5 related skills. If a student gets, for example, the first 3 out of 5 correct, the score is 3/5. If they retake it, and get the last 4 right (but miss the first this time), I'll raise the score to 4/5, not 5/5 - even though the first one was "mastered" the first time around. This promotes lots of retaking, which is what I want, since my students really need to practice and practice in order to retain concepts.

It took students a while to understand how this system works, but as they figure it out, they love it, because it gives them a chance to really improve their grade when they fall behind. I've had a handful of students bring their grades up from Fs to Cs or Bs just in the last two to three weeks before finals, where this never would have been possible before.

My big fear, of course, is that this style of "micro-testing" would lead to artificially high grades, and that students' retention of material would not pan out. I've been eagerly anticipating the results of the final exam to get some relevant data. The final consisted of 50 questions that were compiled from the skills tests, though of course with different values. First off, here is the distribution of grades on the final exam:



Though this may not look like something to cheer about, for a DCP final exam, this is actually quite good. The average score was a 70 and the median was a 72. But, I was more interested in thinking about the relationship between students' skills test percent and the final exam percent. If the system works as it is meant to, the skills test score should strongly predict the final exam score. The next graph shows a scatterplot of this relationship.



The purple dotted line shows what a y = x relationship would look like, and clearly (as I expected) there are more dots below the line than above - indicating students who performed better on the skills tests than on the final. But how much of a difference was there? I added in the best-fit line, and though it deviates from the purple line, it actually strikes me as not that bad. It's clear that all but a handful of students who failed the skills tests (i.e. didn't do well the first time, and didn't bother retaking them) also failed the final exam. While these students concern me greatly in terms of the task we have in motivating and educating our target students, they actually support the idea that the skills test scores are predictive of the final exam score.

The section of most concern to me is that in the red box. These are the students who had a passing score on the skills tests, but failed the final exam. Are there enough students in that section to show that the system doesn't work? I'm not really sure. Of the students who passed the skills tests, many more of them passed the final exam than did not, and I find this encouraging. And, the 24 dots in the red box all did better than 50% on the final, which means they didn't have catastrophic failure (which is not that uncommon on our final exams). But, they didn't show what we typically consider "adequate" retention, since they didn't get at least 70% of the questions right.

I'm posting this because I would like feedback and impressions from other teachers. What does the data say to you? And for those of you using a concept quiz/skills test method, what kinds of results are you seeing?

Sold out or bought in?

We're back from break, and it's time to gear up for finals. Since DCP is a California public school, my course is standards-based. I use the standards as a guideline for what to teach, but of course I must pick and choose, modify, add, and subtract in order to meet my students' needs and get them ready for higher level classes. Though it's not fun for anyone, the STAR test must be faced head-on, and I want my students to show that they really are learning math (even if it is hard to see on a day-to-day basis). To that end, I am giving a fully multiple-choice final exam. I copied the language and even the formatting of the STAR test. I feel (somewhat) justified in doing this, since none of the quizzes or cumulative exams have had any multiple choice on them. And, if they don't practice the all-or-nothing multiple choice format, they will do much worse on the STAR test (and the ACT, and the ELM, and the CAHSEE, etc.).

Most DCP students simply don't study. We do our best to teach them, but it takes a long time for students to first believe that studying helps, and then to learn how to do it effectively. On our first day back, I gave the students a practice final exam without any warning. They were not thrilled with it, but they accepted it and actually put in real effort. My purpose was to show them what their score will likely be on the final if they don't study at all. It was time well spent, because before giving them back today, I asked students to write down what percent they think they got on the test. Almost every student guessed way higher than their actual scores, and many were quite shocked. Hopefully, this will help students make wiser decisions regarding studying between now and finals (which start next Wednesday).

Here is the practice final, if you are interested.

Algebra 2: Lines and Systems

I'm trying to cram in a unit on systems of equations and inequalities before break. It's hard, since so many students are still not totally comfortable with graphing linear functions. But we're making progress. We're up to solving 3x3 systems with linear combination, and most of them have got the idea. These problems are huge, and are probably the longest routine problems my students have ever done. This is cool, because it makes them feel smart and accomplished when they get one right. Unfortunately, a single arithmetic or copying error (which happen all the time) can crumble the whole thing, and then the frustration is back again, eating away at their self-confidence. I'd like to get to graphing systems of linear inequalities before break. When we get back, we need to review for the final, but I'd really like to do some work with linear programming problems first. Here are the files from the last few lessons.

Lesson 1 (Linear Functions) / Keynote / Quicktime
Lesson 2 (2x2 Linear Combination) / Keynote / Quicktime
Lesson 3 (2x2 Word Problems) / Keynote / Quicktime
Lesson 4 (3x3 Systems)
Lesson 5 (Systems Practice)

Algebra 2: Parent Functions

We ended the functions unit before Thanksgiving. I'm not giving a comprehensive test until just before winter break, and I think that is good, so they can have more time for it to sink in. The new unit is on systems of equations and inequalities, but I'll post about that later on, when I have more time. Here are the last files of the unit.

Lesson 15 (Practice and Skills Test)
Lesson 16 (Translating Parent Functions)
Lesson 16 Keynote
Keynote Quicktime

Algebra 2: Horizontal Shift and Review/STAR Problems

Translation and transformation have continued to prove extremely difficult for my classes. Even my strongest students have been struggling. I'm still trying to work out what is making it so hard to understand (if anyone has insight on this, I'd really love to hear it). I think they are starting to get the hand of it, but for mastery, we'd need at least another full week, and that is time we just don't have - especially for something that is only tangentially in the standards.

I did incorporate the idea of texting in lesson 14, to introduce what I'm calling "translation notation". We're not talking about vectors or anything like that, but I wanted to give them an efficient way to describe the translations and calculate with them. The kids thought it was really funny; I did play it up, calling it "math chisme" (gossip) and pretending I was texting it under my sweatshirt to my friend. You wouldn't want to type out that whole sentence, right?

Anyway, here are the files from this week.

Lesson 12 (Horizontal Shift) Keynote Quicktime
Lesson 13 (Translation and Transformation Practice) No Keynote for this one
Lesson 14 (More Translation and Transformation) Keynote Quicktime

Algebra 2: Vertical Translation and Transformation

I've been really good about my timing all year, until this lesson... I wasn't able to finish it in any of my classes. We almost got to the end of the Keynote, and didn't have any time for independent practice. But that's why I don't really create more than one lesson at a time - so I can adapt as needed. Well, that and it takes a huge amount of time, and keeping afloat is what it's all about. I'm still not sure why this lesson took so long; some students were tearing through the class notes, figuring it out on their own and finishing before we even go there. And some students were struggling to keep up. I know it's always kind of like that, but we are working with a very visual representation right now, and it has shifted some of the dynamics of the classes.

Coming soon will be horizontal shift, but not horizontal stretch. I don't want to overload them, and the standards in Algebra 2 really only require that students be able to graph things like f(x) = a(x - h)^2 + k, or to say how one vertex form parabola got shifted to another one. They can learn horizontal stretch in pre-calculus with the trig functions. At least this will give them a good foundation for the tedious work of grinding through f(x) = -2sin(3x-pi/2)+5.

Lesson 11 (Vertical Shift / Stretch)

Lesson 11 Keynote
Keynote Quicktime

Algebra 2: Graphical Analysis Practice

We have Veterans' Day off on Tuesday, but we still have school on Monday. Would have been nice to get a four-day weekend. How many absences do you think we might have tomorrow? I decided to do a review lesson, both because my students are really struggling with graphical analysis stuff, and because I don't want to move ahead with potentially many students gone. Hopefully that doesn't happen. But there are at least 4 teachers who are taking a personal day, so...

I found a site with some good resources on understanding domain and range graphically, and have included some of those animations in this lesson's Keynote.

Lesson 10 (Graphical Analysis Practice)
Lesson 10 Keynote
Keynote Quicktime

Algebra 2: Graphical Analysis

Here are the latest files... more work with domain and range (which continues to stump some students) in interval notation form, and my favorite, solving equations and inequalities graphically. These are very challenging concepts for students, even though they don't seem like they would be, compared to some of the other material. But Keynote really shines through for clearly showing how this works.

Lesson 8 (Domain and Range)
Lesson 8 Keynote
Keynote Quicktime

Lesson 9 (Analyzing Graphs)
Lesson 9 Keynote
Keynote Quicktime

Brilliant!

Students were asking why we have to learn interval notation. I was going on about ease of communication and writing things more simply, but I wasn't getting anywhere until one student piped in with this gem: "Oh, it's just like texting". As soon as she said that, the rest of the class produced a collective "ohh...". Why didn't I think of that? I used it in the following class, and it worked well.

Algebra 2: Interval Notation

I went back and forth on whether or not to spend time on this, and in the end I decided to go with it. It will be helpful to students who go on to pre-calc and beyond, and it gives us a good opportunity to review solving linear inequalities and to keep working on finding domain and range of graphs. Plus, it's good to have a lesson every once and a while that is pretty easy for students to master right away - someone said today, "This is the easiest thing we've learned in like 50 years!".

Lesson 7 (Interval Notation)
Lesson 7 Keynote
Keynote Quicktime