Blog

Assessment

Through the looking glass

Sometimes you come across an idea/process/presentation and it is Ah.Mazing.

Today for me, that presentation was given by Dr. Clifton Franklund from Ferris State University at the 2018 IUPUI Assessment Institute.

Amidst what sounds like a discouraging and chaotic situation on campus, Dr. Franklund has developed a beautiful, intuitive and actually useful method of collecting, analyzing, and disseminating general education data for a campus of around 14 thousand students.

The beauty in the system is its simplicity and leverage of (free) statistical and data tools to create a way of meaningfully engaging faculty (and students) in understanding assessment data and using it to make informed choices.

The entire system was borne from a shift in perspective on the assessment process – rightfully, Dr. Franklund suggests that too often, educators focus all their efforts on “planning” and “collecting” assessment data.  Everyone is then too exhausted to spend time learning from the data or engaging in meaningful visualization and interpretation of what the data actually tells us.

Instead, Dr. Franklund turns the focus towards the discussion about what assessment data tell us – whether that leads to specific changes or simply staying the course, the entire process is geared towards faculty conversation about student learning.

If that isn’t the heart of assessment, I don’t know what is.

In the details

I could describe the process used at Ferris State based on the presentation I saw.  But the BEST PART is that not only is their new system set up to be transparent on their campus, but it’s totally open source.

You (and I and anyone else) can visit the data, report and all the files and documentation online.  Yes, I just said open source and online access to anyone in a sentence about assessment data.

When I live in a world where I feel like we do assessment so we can say we put a report in a drawer, this feels like a breath of fresh air.

The nuts and bolts of the system are this:

  1. Assessment artifacts are classified into one of 14 types.  Faculty evaluate work on whatever scale they choose and then translate this score to a rubric score from 0 – 4.
  2. Faculty enter the student IDs and scores into a spreadsheet sent to relevant faculty each semester.
  3. Faculty record the data and send it directly to the database.
  4. The Gen Ed coordinator (Dr. Franklund currently) downloads and compiles the spreadsheets based on learning outcomes.  Courses are deidentified into 100 and 200 level by department.
  5. The coordinator analyzes the data using R, interprets the data and write an assessment report.
  6. Discussion prompts are included in the report and a Disqus discussion board is embedded directly into the report.
  7. Faculty have one year to review the report, play with the data if they choose, and discuss the findings.
  8. At the end of the year, the discussion board is closed and the discussion is archived with the report, a beautiful way of closing the loop and promoting institutional memory.

Finally, an innovation

As has been an underlying theme behind some of the presentations today, there’s nothing much novel about the way we do assessment.  The problems and solutions identified decades ago are still with us.  So, what have we been doing?

Even when I go to a presentation about the experiences on a campus, it still seems like we’re doing the same things that should have been done, I don’t know, a decade ago.

I’m not sure the extent to which I can run with what was presented today, but I am certainly going to be thinking about what he did and percolating ways I can implement the Ferris State model either in the department or in other ways.

Because I am shouting the brilliance of this idea from my own tiny rooftop – check out their resources here.

Let assessment be new and move beyond the typical – finally!

Teaching

I threw out my grade book

It happened to me like a bolt of lightning.  I was grading work, sitting in the public library, while my kids ran around pretending they were taking a time travel rocket back to the age of the dinosaurs.

While I might have preferred to be reading one of the many books on my Goodreads list, I wasn’t.  I was grading.

And I wasn’t having a ton of fun.  It seemed like I might be having more fun than humans who time-traveled back to the Jurassic era and found it difficult to return to my own time when my ship was damaged by a T-Rex.  But, whatever.

I. Don’t. Like. Grading.

Educators, breathe a collective sigh of relief because I said it for you.  I don’t think anyone likes grading.

Don’t get me wrong, I LOVE giving feedback.  I love thinking about words and ideas, and how concepts fit together and create new ideas and knowledge and inspiration.  That stuff I love.

But I do not love taking words and ideas and inspiration and stuffing them into either a 7-point box or an 8-point one.  Oh wait, maybe that should be in the 7.5-point box instead?

Forget it.

Stages of Grieving

However, I felt a little trapped.  I didn’t know what to do with this frustration that had built inside me countless prior semesters and was erupting in the middle of the children’s area of a library.  So, I literally Googled, “professor doesn’t like grading”.

And I found a community of people out there who had already thrown out their grade book.  They were liberated from fighting about points (can’t you just bump me up one more for XYZ grade?!?!?!) and able to reinvest their energies into well, actually helping people learn stuff.

As any great professor, I took this opportunity to dive headfirst down into a rabbit hole, avoiding the very work I hoped the rabbit hole would help me resolve.  You know, instead of just knuckling down and doing the grading. Ironic, no?

Take the plunge

After several hours of reading, deep contemplative thought (in between hollering at children sparring in the backseat), and at least a week of rolling it around in the back of my mind.  I went for it.  All in.

I found some great resources along the way – if anything I’ve said caught your attention, check these out.

My ultimate contract, after reviewing several, is based largely off Jennifer Hurley’s example – her blog details both her journey into gradeless grading and what she’s learned since.  Plus, I like her writing.

Jane Danielewicz and Peter Elbow have also written an essay about their grading contract lessons – a paper that has been widely referenced among the gradeless graders.

Also, check out this overview of different forms of contract grading that can be used from Oberlin and lessons learned from John Warner.

And then I sat down to write my syllabi for the semester.  Point-free syllabi.

I had to think carefully about several things.  Handling participation and course absences in both face-to-face and online courses.  Prior to contract grading, participation points were mega huge for students’ grades in my classes.

I also had to have specific enough language in the contract that students knew what to expect (objectivity was still embedded in the process) but to allow for flexibility enough when determining whether students had reached the benchmark for acceptable work.

I realized I was coming at the contract from the perspective of myself as a student – how can I get an A – but wasn’t thinking clearly about students who might be at the other end – how can I get a B.

The current version for my Communication Theory course, based largely on Hurley’s contract, is here.

Bracing for Impact

So, this little pipe dream of mine, a little dream of which I was now desperately in love.  What would the students think?  Was the semester going to turn into an episode of Game of Thrones?  Was everyone going to think I had lost it?

I went to class.  I opened my online modules for my online classes.  Students heard me talk about the pointless craziness.  Students took a quiz on the syllabus.  No questions.  No complaints.

But don’t worry, I’m prepared should that other shoe drop.

It seemed a bit too good to be true.  Until I sat down to do battle with our learning management system, Canvas.

Your LMS Number Management System

Let’s be real right now.  Your LMS may be wonderful (after using them all Canvas is truly a delight).  It may make everything about education one zillion light years better than it would have been otherwise.

But it HATES gradeless grading.  It craves numbers.  Like that little frog guy from Number Munchers.  Nom nom nom.  Give me numbers and calculate on them and share them with your students.

arts-and-crafts-blue-background-concept-1314536

Nom nom – wait what?  You don’t want to give points?  You want to just give feedback?  To mark work satisfactory or unsatisfactory?  Possibly as excellent?  NO.  YOU CANNOT DO THAT.

I’ll be honest, it took me several days of pondering and fighting with Canvas before a discussion with a colleague illuminated the problem.  Grade assignments on a scale of 0 – 3.  Three is “outstanding” work, two is “acceptable for the contract”, one is “insufficient”, zero means it wasn’t submitted.  And magically, the problem was solved.

And the verdict is…

From my perspective?  It. Is. Amazing.  I spend SO MUCH LESS time and effort grading.  My feedback is targeted to the specific things students do well or need to improve on in their work.  To me, the grading system is simple and easy to understand.  And now that my LMS and I have learned to work together, all is good.

From the students’ perspective?  I haven’t heard anything positive or negative at this point, but students are doing their mid-term evaluation now so I’ll know in a few days where they are with it.

I will say this change was both freeing and posed a serious challenge to the way I talk about and express standards with the students.  It changed the way I think about grading.  And I believe put this task into a more manageable box – it takes less time and is less tedious, rubric or no.

I’m proud to report, barring some disaster or significant student opposition, I’m proud to be a gradeless grader.  Sometimes, you just have to take the plunge.