Retention and Intention in Massive Open Online Courses

For educators used to thinking about student attrition in a traditional university setting, the "retention funnel" in a massive open online course (MOOC) often causes considerable alarm. For example, in 2012, the typical Coursera MOOC enrolled between 40,000 to 60,000 students, of whom 50 to 60 percent returned for the first lecture. In those classes with required programming or peer-graded assignments, around 15 to 20 percent of lecture-watchers submitted an assignment for grading. Of this group, approximately 45 percent successfully completed the course and earned a Statement of Accomplishment. Thus, in total, roughly 5 percent of students who signed up for a Coursera MOOC earned a credential signifying official completion of the course.

To a university professor accustomed to the traditional audience of committed, paying students in a brick-and-mortar classroom, the image of continuously-emptying lecture halls—where only one in every twenty students remains to the end—is an understandably frightening prospect. But is this really the appropriate framework for thinking about student success in MOOCs?

We believe that retention in MOOCs should be evaluated within the context of learner intent, especially considering the varied backgrounds and motivations of students who choose to enroll. When viewed in the appropriate context, retention in MOOCs is often quite reasonable.

Understanding Learner Intent

The vast majority of students who enroll in traditional university classes enter with the explicit intent of earning a credential. MOOCs, however, cater to a substantially more diverse audience. For MOOC retention metrics to be useful, they thus must be defined and interpreted with the learner's goals in mind. Passive lecture-watchers, for example, may go through an entire course without ever touching an assessment, yet they often derive substantial value from a MOOC without contributing to completion-based notions of retention. The lectures in typical MOOCs differ significantly from the lectures delivered in the standard lecture-hall setting, in that MOOC students are continuously forced to interact with the material in the form of frequent in-video quizzes. Furthermore, because of the online format, many professors can choose to pack more material into a concise ten-minute lecture segment, knowing that the student will have the option of re-watching the videos multiple times. Across a range of Coursera classes, students watch the typical lecture video an average of 1.7 times, with one in every ten students watching a video more than 2.7 times on average. Given that the proportion of students who watch at least 90 percent of the available lecture videos is generally around twice the number of students who earn a Statement of Accomplishment, this important subpopulation cannot be ignored.

Because of the variability in student intent, it is important to study the completion rates among those students who actually begin the course intending to complete it. In general, students do not declare their intent from the beginning, making this rate difficult to estimate. To determine student intent, Stanford Professor Kristin Sainani asked students in her "Writing in the Sciences" course to fill out a pre-course survey about their planned amount of effort for the course. Of the students who responded to the survey, roughly 63 percent planned to do all the work necessary to earn a Statement of Accomplishment. Considering that only around one-third of the students who registered for the class actually filled out the pre-course survey, and that the students who completed the survey were more likely to be the ones engaged in the class, clearly not all students entering the course were highly invested in finishing. But among students who intended to finish, roughly 24 percent successfully completed the course, compared with fewer than 2 percent in the remaining population of registered students.

An even more compelling indicator of intent can be found in Coursera's Signature Track, announced in January 2013. This track, a recently developed optional program available in selected MOOCs, provides students with a way to earn a more official credential for their accomplishments by participating in keystroke biometric and photo-based identity verification. Students enroll in Signature Track early in the course (week two or three), pay a fee ($30–$100) for the identity-verification services, and earn an identity-verified, university-branded credential if they pass the course. Signing up for Signature Track is a clear statement by the student that he or she intends to complete the course and earn a credential. In the first Signature Track class, the completion rate among paying Signature Track students was 74 percent, compared with 9 percent in the non–Signature Track population. Moreover, among students who indicated a strong intent to finish from a survey administered one month into the course, completion rates were higher in the paying group (96% vs. 84%, p = 0.0009), suggesting that having a financial stake may provide additional incentive to finish.

The Relevance of Retention in MOOCs

We have argued that for discussions of retention in MOOCs to have real meaning, retention should always be considered within the context of student intent. Still, a higher-level question remains: is retention even the right metric by which to measure success in a MOOC?

For students whose explicit goal is to earn a credential in a MOOC, retention-based metrics probably do provide a reasonable proxy for course success, which is capped to some extent by the natural rate at which students disengage from any online activity due to unplanned life events, shifting personal priorities, and the low friction involved in stepping out of an online course. As noted above, we believe that MOOC retention rates among students who are committed to completing the course are quite high. Yet we also believe in the importance of striving to improve retention among all highly motivated learners, whether by exploring new and more engaging forms of pedagogy, taking advantage of social networking to help maintain students' interest in lifelong learning, or simply adapting existing MOOCs to better fit the needs of the working adult online learner.

At the same time, in discussing MOOC retention rates, it is also important to keep in mind the bigger picture. In traditional university courses, lack of retention is a serious problem. Currently, almost half of the students who begin college at a two- or four-year brick-and-mortar institution fail to earn a degree within six years, according to a report by the National Student Clearinghouse Research Center.¹ In some universities, the completion rate is close to 15 percent.² With the average cost for undergraduate tuition, room, and board estimated to be $13,600 at public institutions, $36,300 at private not-for-profit institutions, and $23,500 at private for-profit institutions for the 2010–11 academic year,³ students who drop out have invested a considerable amount of money, with little return. Many of these cases involve significant costs to taxpayers as well, whether via state support for public institutions or financial aid. Non-completing students have reduced chances of getting a good job and hence considerably lower chances of repaying their loans.

 By contrast, students who enroll for free in a MOOC and do not complete the course incur zero financial cost to themselves or to taxpayers. Some critics correctly mention the cost in terms of students' time; however, given the amount of time that people spend on activities such as watching television, having some time "wasted" on education, even by non-completing students, seems quite inoffensive. Indeed, one can relate the act of enrolling in a free online class to that of checking out a book from a public library: it would be absurd to measure the success of the book strictly by the proportion of individuals who read its contents cover-to-cover within the standard loan period. Some people may read a few chapters of a non-fiction book and then stop after getting enough information to suit their needs. Others may read more deliberately and renew the book a couple times before finishing. In both cases, few people would consider the lack of completion or the extra time taken to be a waste or a failure on the part of the book.

In fact, the ease of non-completion in MOOCs can be viewed as an opportunity for risk-free exploration. Think, for example, of high school students who are trying to decide where to go to college. Through MOOCs, a student can explore different topics that she finds intriguing and pick the ones that are actually a good match for her interests and skills. Or a student can try courses at different levels of difficulty, perhaps discovering that he is capable of more than he thought and thus trying for a more selective school than he had originally intended. Thus, MOOCs may help alleviate the problem of college undermatching, which occurs when students attend less-selective colleges than their skills would allow—an issue that particularly affects the success of minority and first-generation college students.⁴ Interventions aimed at increasing retention by reducing students' freedom to try out multiple classes actually do little to improve the true social impact of an online course, despite the fact that they may lead to increased retention rates by artificially selecting only the most dedicated students.

Ultimately, retention is only one of many factors underlying success in MOOCs and is, arguably, far from the most important for many students. The goal of education is to provide students with the skills they need to achieve their life goals, whether or not they finish a course. Given the broad range of motivations for students participating in MOOCs, the true challenge of online education will be to identify what it is that students are looking to get out of the virtual classroom experience, so that we can provide them all—non-completers and completers—with the learning experience that will help them achieve those goals.

1. Doug Shapiro et al., "Completing College: A National View of Student Attainment Rates," National Student Clearinghouse Research Center, November 2012.
2. Bryan J. Cook, "Incomplete Completers: Analysis of a Comprehensive Graduation Rate," American Council on Education (ACE).
3. "Tuition Costs of Colleges and Universities," National Center for Education Statistics.
4. The problem of undermatching is addressed in William G. Bowen, Matthew M. Chingos, and Michael S. McPherson, Crossing the Finish Line: Completing College at America's Public Universities (Princeton, N.J.: Princeton University Press, 2009).

A longer version of this article is available here.

Daphne Koller, co-founder of Coursera Inc., talks about the ways in which analyzing student and instructor behavior can contribute to better learning outcomes in MOOCs.