More than 5 million college students took an online class during the 2013-14 school year (AP Photo/Jacquelyn Martin of a college student in a St. Mary’s College dormitory in Lexington Park, Md.)

Does online learning work? Do college students learn better, or at least as well, from computer instruction as they do from a human teacher? That’s a question asked over and over again by not only students, parents and professors, but also by academic researchers. It’s especially important because universities are offering more and more of their courses online. Almost 5.3 million university students took at least one course online in the 2013-14 academic year, a 4 percent increase from the previous year, according to a Babson Survey Research Group report citing federal data.*

A new paper sheds some light on this question. The author sorted through the best studies on online university courses published in the past couple years, and concluded that online education, or partial online instruction, is neither worse nor better than traditional face-to-face instruction.

“Students are achieving the same outcomes in either format,” online or in person, said Martin Kurzweil, director of policy at Ithaka S+R, a non-profit research and consulting organization which published “Online Learning in Postsecondary Education: A Review of the Empirical Literature (2013–2014)” on March 11, 2015.

None of these three studies employed educational software that was at the cutting edge of so-called “adaptive learning,” where algorithms can tailor instruction for each student, as a personal tutor would. Essentially, the same content was taught to all the same students through interactive videos and worksheets. If a student got a question wrong, the computer system could help the student review what he didn’t understand.

Still, it’s important to see that independent scientific research, reviewed by peers, shows a far humbler picture for online learning than the purported learning gains touted by software vendors to university administrators and professors. “When a vendor cites evidence of impact and it’s based on their own analysis, you always have to take that with a grain of salt,” said Kurzweil. “Online learning providers aren’t special [in this]. Textbook publishers do the same. It’s based on research that hasn’t been validated.”

And it’s also noteworthy that anti-technology naysayers would be incorrect to assert that online learning is categorically worse than tradition instruction. However, there are signs in this research that weaker students aren’t learning as well through the kinds of computerized instruction available today. More on that later.

Derek Wu, the author of the Ithaka paper, combed through every research paper published between 2013 and 2014 that studied online education at universities. His parent organization runs JSTOR, the library of digitized academic articles. So Wu scanned everything. Only 12 research papers passed minimum scientific muster, where online learning was compared to face-to-face instruction. And only three of these studies randomly assigned students or had enough information about student’s academic abilities before the start of course to compare how much students learned. Too often, researchers look at students who had voluntarily registered for the online course. But students who choose online versions tend to be different. They’re often older, for example. And they may be better at learning on their own than the average student is. So it’s not fair to compare a self-selected group of mature autodidacts against the general population. They might learn more in any classroom, online or not.

In the best of the three studies, more than 700 students at Baruch College (part of the CUNY system) were randomly assigned to either a conventional introductory economics class with a professor or a purely online course, covering the same material. Students took the same midterm and final exams. The face-to-face students did much better than the online students on the midterm, but by the finals, there was no statistical difference between the two groups.

However, lower-performing students did worse online than face-to-face. That’s a sign that online instruction may not be working as well for struggling students, and echoes a 2011 Columbia University Teachers College study finding that community college students were more likely to drop out of an online version of a course and to get lower grades.

In the other two studies, hybrid instruction (part human, part software) was put to the test against conventional face-to-face instruction. In one, there was no statistical difference between to two teaching methods, in terms of students’ exam scores and homework grades; they were essentially the same. In the other study, it depended on how the authors did their math. Using one statistical technique, they found the hybrid students scored worse on weekly quizzes than the face-to-face students did. Using another statistical technique, the students performed the same on those quizzes, regardless of the teaching method.

Not all experts are convinced online instruction does no harm. For example, David Figlio, an economist at Northwestern University, conducted one of the first studies to randomly assign students to online video versus live lectures in an introductory microeconomics class. In his 2010 paper, he found no statistical difference in student outcomes on average. But lower-achieving, male and Hispanic students were worse off with the online version. These students did better with face-to-face instruction.**

“When I look at the weight of the evidence, it looks like online education might come at some sacrifice to student learning,” said Figlio. “Thoughtful administrators will need to weight those sacrifices against the cost savings. You can see a situation where schools for the haves will continue with face-to-face instruction, perhaps enhancing it with technology. And the have-nots will get this mass online instruction. That can be potentially problematic from an equity perspective.”

* Babson Research Group ceased its independent count of the number of students taking online university courses with its most recent report, released Feb. 2015, and is now using federal data from the Integrated Postsecondary Education Data System (IPEDS). 

**Futher complicating matters is that Figlio’s face-to-face control group used some supplemental online materials. “Face-to-face plus” is how Figlio describes it. Others might call that hybrid, and argue that Figlio’s study proves that hybrid is marginally superior to online instruction for low-achieving students. Now that hybrid, or so-called “blended,” learning is becoming more and more popular, it’s going to be tough to study it properly.

This article also appeared here.