How do we know that students “got it”? What lets us know that they understand the material, or that our assignments were focused on the learning objective in question? In the physical classroom, discussion, questions, and even non-verbal cues can let us know more about where the students are within a given set of material or objectives. But online, we miss some of that information– we grade discussions over the course of several days and might miss patterns that would appear more clearly in synchronous conversation, blog posts might trickle in over the course of a week, and we lack all of those non-verbal cues online.
But one of the benefits of having things online is that you have data to work with! Discussion boards, blog posts, journal entries, and even longer assignments are a wealth of text that can be analyzed to see what students are talking about collectively. Here’s an example: the now-common word cloud.
Word Clouds and Word Frequency:
For this example, I took the text of an entire class’ discussion board and dumped it in Wordle, a free online tool that generates word clouds based on the frequency of a word’s occurrence. The title of this discussion was “Initial Ruminations” and since that was the title of everyone’s posts, you can see they’re the most common words in the discussion.
But look at some of the others: teachers, school, PLC, learning, results, standards, improvement, team. Are these the things the students should be talking about? In the case of this discussion, talking about professional development, they are right on the mark.
But what if the words had been different? Uncertain, questions, didn’t, wonder: these words indicate less certainty, and their appearance might signal to an instructor that there might be some aspect of the assignment or learning objective that was unclear.
Word clouds and word frequency give summative feedback about the assignment, which can be used both during the course to help the instructor focus on the topic for longer, and used after the course, to redesign the assignment for future semesters.
Grade Information and Graphs:
Trends in grades are easy to miss. When faced with a list of numbers, I can’t always see what they mean in the bigger picture. However, Blackboard lets you download all of your grades into an Excel file, and Excel has some excellent tools to visualize what those data mean.
In the case of this image, I took all the quizzes from a course and graphed each student’s performance to look and see if there are any trends. To do this, I simply removed all of the columns I didn’t want to have graphed, changed the student information to a A## format, and clicked on the Insert Tab > Line > Line with Markers. This generated a graph for me that I was able to tweak a little (for scale, so that you can easily see the trends here, and to switch the axis so that I would end up with a horizontal graph). It was a pretty automated process.
Each student is listed on the right side as a number, and the last one is the class average. I can see each individual student’s path as well as the distribution of grades in each assignment. As an example, Quiz 5 has a much lower set of grades than other quizzes. This raises questions: was this quiz particularly hard? Was there a set of materials covered here that wasn’t covered enough? Are there things in this quiz that the students should have learned in a prerequisite course? Is it part of a larger trend in the course? Each of these questions can help improve the class the next time it’s taught.
These are just two examples of ways you can take your class data and look at it in a new way, which I find helps get me out of the usual ruts of my thinking.
If you have another way of looking at data, I’d love to hear about it! Please email me (Lily) and let me know about it.