This is a bit of a follow up to the paper I wrote last semester that I posted a few weeks ago. I’m going to go off on a bit of a rant (like I didn’t vent enough in the comments of a post on ReadWriteWeb) about a research study by McKinney, Dyck, & Luber at SUNY-Fredonia about the effectiveness of iTunes U in delivering lectures to an introductory Psych course. The title of the paper is provocatively titled: “iTunes University and the classroom: Can podcasts replace Professors?”

In this study, two groups of students listened to a 25 minute lecture. One group listened in class, the other listened via podcast. When given a test on the subject, the podcast group scored 71% and the in-class group scored 62%. Within the podcast group, those that took notes while listening scored significantly better than those that didn’t (who performed more or less the same as the in-class group). The study generated significant controversy among their colleagues because “the podcast condition was not used to enhance a college lecture (perhaps giving students who attended the actual lecture a chance to listen to it again), but rather was in place of attending the lecture.”

On the one hand, this study engages the issue of podcasting in a serious manner. Lots of investment is being made in this area by universities, which is strange given the lack of research or data regarding its effectiveness. In comparison to some of the other studies I’ve read, it’s incredibly thorough.

On the other hand: Stop it.

Just stop it. For the love of god, it means nothing. Why must we continually have these studies where professors take old ways of teaching, dress it up with technology, and ask if technology is really worth our time? About half the time, the results go towards the technology, half the time they don’t. Does it really matter? You’re still talking at students and seeing how good they are at regurgitating it back to you. All these professors are getting up in arms because students can use the podcasts to avoid going to class. News flash: not having podcasts has never stopped them before.

So who cares? Why must people keep doing the same damn media comparison studies over and over again? Why the same old lame questions asking if technology will replace teachers? All these things ever seem to do is bring out the same comments:

“Technology is crap, you’ll never replace a teacher!” (goodbye, tenure) or

“We have to change the way we work to accommodate digital natives!” (so they can be bored with technology too) or

“Podcasts are great because you can rewind them!” (as if rewinding is the panacea for all our educational woes).

I’m not trying to disrespect the lecture as a way to deliver instruction. I’m not trying the disrespect the podcast as a medium to deliver instruction. In fact, a couple days ago I listened to a podcast lecture by Michael Wesch that blew my mind it was so awesome. The reasons I liked it had nothing to do with my ability to rewind it or listen to it on my commute.

The reasons I did like it were three-fold: the content, the speaker, and the access. The fact that the lecture took place a month ago in a different location, and required attendees to pay some exorbitant registration fee did nothing to prevent me from hearing it.

Technology is a game-changer, but these researchers are still playing the same old game.

I’m not referring of course, to the bullets that come out of a gun, but rather the ubiquitous bullet point: •

I was told 4 years ago to have my wisdom teeth taken out, which I put off (a brilliant move on my part) taking care of until today, when I found myself sitting in an oral surgeon’s office watching a video that is to inform me all about the process. At the point that the presenter is discussing possible complications, a bullet point list comes up on the screen. Here’s what I see:

• blahblahblahblahblah
• blahblahblahblahblah
• blahblahblahblahblah
• blahblahblahblahblah
• blahblahblahblahblah
• blahblahblahblahblah
• POTENTIALLY FATAL

WHAT???!!!!

At its best, the bullet point list is an easy shortcut that helps to properly organize your ideas – particularly for presentations, which are all about talking points. At its worst, the bullet point list is for lazy bastards with no imagination.

Dan and Chip Heath had a related column in Fast Company a few months back (yes, I realize I reference this magazine a lot, but it’s the only one I read on a regular basis):

We’ve all had the experience of being in the audience as a presenter clicks to a slide with eight bullet points. As he starts discussing the first one, we read all eight. Now we’re bored. He’s lost us.

He’s lost us? Or killed us?

This past week, I received a strange piece of mail. It was a paper I wrote for one of my classes, one that I have been finished with for almost 2 months.

This is the first time that I’ve ever had anything like this occur, so I don’t want to blow this up and pretend that stuff like this happens all the time. It really doesn’t. I also don’t want to sound like I’m totally trashing this professor. He made it fairly clear (admittedly not until before the midterm) that he was not really down with delivering the class online, and much preferred lecturing. This became evident the few times I saw him in person, when he was engaging, passionate and funny – really an all-around great lecturer. The Blackboard site for the class, however, was a bit of a mess. Content was scattered, disorganized, and confusing.

I’ve come to understand that professors are busy: they’re under enormous pressure to do original research, teach classes, answer questions. It’s a 7 day a week job where they have to constantly be at the top of their game, demonstrating their expertise. With other such lofty concerns, I can imagine organizing Blackboard content and grading papers probably falls pretty low on the priority list.

I think it just may be a symptom of a systemic problem within higher learning that really has never been addressed. Andre Malan, a student at the University of British Columbia, gave this really fascinating talk about the future of education, where he discusses the paper grading problem.

His argument is that the traditional mode of doing so relies on one person – the professor, who may or may not grade papers reliably or accurately (his picture of papers thrown down a staircase, with the heaviest papers at the bottom and most deserving of an A – is freaking hilarious). He cites Jon Beasley-Murray’s Murder, Madness and Mayhem Wikipedia project as an example of a community-based grading system. In this project, the articles students wrote stayed relevant, even months after their completion. In many cases, student articles became top search hits for particular topics.

I appreciate the 100% the professor gave me on my paper. But 2 months out from finishing it, that grade just doesn’t seem relevant anymore.

Another mile long paper that I wrote last semester for my Foundations of Educational Research class, presented here for the purposes of personal archiving. In my next post, I’ll go into all the insights I’ve gained since turning it in.

(The goal of the paper was to produce a research proposal, but since I am not an educational researcher there are no plans in place to actually produce such research).

Podcasting and the computer-aided in-class presentation: a comparison of their relative influence on student motivation in a graduate Education class.

Abstract

Podcasting is a relatively recent technological phenomenon that has garnered widespread popularity among educators. Large investments have been made, particularly among colleges and universities, to implement podcasting for instruction. Based on existing research, however, it is not quite clear that this investment is justified, given existing models of use within education. Previous studies have shown that instructor generated podcasts are only being used by a small minority of students. Learner generated podcasts, on the other hand, have shown some indications of increasing student motivation, though these studies are frequently found to be lacking in a number of ways. The research proposed in this paper seeks to better gauge the motivational reaction of students creating podcasts and compare it to the (relatively) older but much more prevalently used computer-aided presentation. Participants in the study will be graduate-level Education students, which differs significantly from existing studies that tend to study populations of undergraduate Information Technology students. A quasi-experimental counterbalanced design will be used to study an existing class group, with students creating projects utilizing both technologies. Administration of the Academic Motivation Scale and focus groups following the completion of each project will be used to compare student motivation.

Introduction

The student-led presentation has been a staple within Western education since ancient Greece. In recent times, however, the development of several different technologies has altered the way in which these presentations are delivered. I propose a study that contrasts two of these technologies (podcasting and computer-aided in-class presentations) and examines their effect on student motivation within a graduate Education class.

In the past month, 23 million Americans have downloaded and listened to an audio podcast (Edison Media Research, 2008). This massive use has been in part facilitated by the explosive growth of Web 2.0, a category of technologies which podcasting is apart of. These technologies offer its users greater flexibility in the consumption of its content and the ability to interact with other like-minded individuals. These are trends not lost on advertisers, large media corporations, or educators, who have all been moving to meet this shift in the content delivery paradigm.

One category of podcasts that have garnered significant attention are those dealing with learning and instruction. Educators have been so enamored with the use of podcasting for education that schools such as Duke University, the University of Maryland and Abilene Christian University have given iPods to incoming freshmen (Glater, 2008). What is surprising is that these schools have chosen to spend large amounts of money on technology that has not thoroughly been investigated empirically in the education setting. Gura (2006) makes the argument that this entertainment-based technology has been merely “tweaked” to fit into an educational setting, and little thought has been given to the ways to best exploit it. Consequently, most instructors use podcasting only for the dissemination of class lectures, the idea being that students will be able to review the lecture at a later time.

In contrast to podcasting, in-class presentations aided with computer programs such as Powerpoint have been around for a (comparatively) long time and have seen significant usage throughout education at all levels. Despite this, Powerpoint has drawn serious criticism from several sources. Most noted is outspoken designer Edward Tufte, who argues that the program “elevates format over content, betraying an attitude of commercialism that turns everything into a sales pitch” (2003). This is not particularly surprising, given its origins in corporate America, but Tufte argues against its use in education, where he believes students might learn more by writing.

Despite these criticisms, Vallance & Towndrow (2007) make the argument that Powerpoint’s critics take little consideration of the things that it affords instruction. Central to their argument is the absence of “informed use” of technology within education; that is, the use of technology to accomplish an instructional task rather than the use of technology for technology’s sake. They describe several of the interactive features that Powerpoint offers that go beyond the linear progression model of presenting that most users are familiar with.

Literature Review

In spite of the theoretical debate that rages on over the use of these technologies, little empirical research has been generated regarding their effectiveness. A majority of what research does exist focuses primarily on instructor-generated materials with very little discussion of student-generated materials. This is in spite of a general consensus that “in order to achieve meaningful technology integration, learning must be designed from a constructivist approach that encourages students to learn in a social context and help them to develop an ability to readily create new knowledge, solve new problems and employ creativity and critical thinking” (Sadik, 2008).

Lee, McLoughlin, & Chan (2008), have produced a qualitative study that seeks to consider learner-led podcasting from a collaborative and knowledge-building perspective within a population of first year college students. Content analysis was performed on two focus groups following completion of the podcasts, which led the researchers to conclude that there was evidence of collective and progressive problem solving, knowledge building discourse, and in-depth engagement. They attribute these not only to podcasting, but also the pursuit of a common goal and a commitment to sharing ideas on the students’ part.

Lee et al. do qualify the findings of their study by acknowledging the extremely small sample size (5 out of 8 total students participated in the focus groups), but this is not the only factor contributing to the incompleteness of the study. Absent from the data is information regarding the characteristics of the research participants, who attend a relatively small university in rural Australia. Participants were volunteers, who may already have had extremely high intrinsic motivation in gaining experience working with this new technology (all were students in the Information Technology program). Additionally, there is no data regarding what grades the students obtained in the class or information on what prior knowledge students had about podcasting. Though this study raises extremely pertinent questions, it must be taken as only a first step towards the investigation of student-created podcasts.

Similar omissions were found in another study by Lee & Chan (2007), which appears to be an inquiry into the use of these same student-created podcasts by other students. In a survey of students in an online distance learning class, 83% listened to 7 out of 9 podcasts provided in their entirety and generally found them to be useful to them in their studies. It is important to note that the sample size was 18, and the response rate was only 47% – it is reasonable to assume that students responding to the survey were more engaged and motivated by the class in general, and not specifically to the podcasts. Much like the study discussed previously, no data is given on student grades, characteristics, or location.

While studies in student-created podcasts are rather rare, there have been large-scale implementations of podcasts of lecture recordings that would seem to point towards their ineffectiveness. In 2005, the University of Washington configured several large lecture halls for the automatic recording and podcasting of lectures, the results of which are presented by Lane (2006). In a survey distributed to over 7,000 students in these podcast-enabled courses, only 5.5% responded. Of those students, nearly half listened to 25% or less of available podcasts, while 20% listened to 76% or more. Though the extremely low response rate makes these findings somewhat inconclusive, they indicate a significant diversity in student usage, though with a definite skew towards non-use. The automatic implementation of the technology may have also contributed to results – responses by participating professors seemed to indicate an apathetic attitude towards podcasting, which may have been passed on to their students.

Grabe & Christopherson (2007), found similar non-use in their study of 329 Psychology students who were provided lecture outlines, lecture notes, and lecture podcasts. They found participants accessed only 3% of available podcasts, in comparison to 61% of lecture outlines and 19% of lecture notes. The researchers posit that compared to printed material, podcasts cannot be reviewed or accessed as efficiently. It is possible that the provision of outlines and notes (a luxury not provided in most college courses) aided in the non-use of podcasts by students.

Bell et al. (2007) approached instructor created podcasts from a different angle, providing supplemental podcasts for two freshman Computer Science courses. The podcasts provided informal discussion of topics covered in class, current events related to those topics, answering student questions, and interviews. Much like the other studies mentioned, the response rate to a questionnaire was low (38%), and of those that responded, 66% had not downloaded any of the podcasts at all. Requests for feedback were also ignored, for the most part. The researchers make the argument that if students were listening to podcasts while occupied with another activity, responding immediately may have been difficult – though based on the data, it seems students are simply not listening to the podcasts to begin with. Despite all this, the researchers still make the claim that podcasting is an “attractive tool to help engage students.”

Frydenburg (2006), found student enthusiasm for lecture podcasts to be similarly low in his study of 54 Information Technology students. Though students downloaded podcasts enthusiastically at the beginning of the semester, downloads dwindled as the class neared midterm. For the second half of the semester, students were offered extra credit to create video podcasts based on class topics. Though early podcasts were relatively simple, students soon made discoveries and developed innovations based on previous podcasts, leading to a “friendly competition” among students on who could make the most creative podcast. Though Frydenburg omits actual documentation of this effect (such as hyperlinks or transcripts of the podcasts), the existence of such competition would certainly indicate an increase in motivation and engagement of learners when introducing student-created podcasts. A survey was taken at the end of the class to gauge student usage of the podcasts, but no questions pertained to the motivation or the engagement of students as a result of creating podcasts, certainly the aspect of the study that seemed to merit the most exploration.

There have also been some studies done on the use of student-created multimedia in teacher education. Though the technology is slightly different, the concept of student-generated media and the use of non-linear digital editing tools are in the same vein as podcasting. Sadik (2008) studied student creation of digital stories (a basic type of video created with audio and still photos) by 13-15 year old Egyptian students. Though the results of assessments on the students’ projects indicated personal reflection on the topic, what is not made clear is how the use of technology was an improvement over more conventional constructivist instructional techniques, such as writing. There is brief mention of negative issues with the technology (such as distracting special effects) that would not occur in such traditional mediums, but these are downplayed. An attempt was made to control the variable of teacher experience with technology by only selecting teachers who already made use of technology in their classrooms, but during the study it was discovered that the teachers (it is not made clear what percentage of them) lacked proficiency with multimedia editing software.

It is clear that there are several significant issues with the research that has been done on this topic. The first is the low response rates in some of the studies using surveys. Lane (2006) faced distribution problems due to lack of interest from instructors, Lee & Chan (2007) had problems of distance, and Bell (2007) could not elicit feedback even when offering free coffee. Additionally, there is the sole reliance on the survey – with the exception of Sadik (2008), no other study uses triangulation to lend credence to research findings. Then there is the complete lack of data regarding the characteristics of the learners. Considering the fact that few studies had controls for extraneous variables (control groups, random assignment, etc.), it is surprising that potentially influential characteristics were not taken into account. Several studies, such as Bell (2007) and Lee et al. (2008), also expend a considerable amount of space discussing the nature and history of podcasting. Though the technology is relatively new, there are a significant number of articles that provide background and anecdotal evidence. A majority of the studies also focus their research on students within Information Technology programs. This would seem to be a logical decision, given the basic technological skill required, but podcasts can potentially encompass any academic subject. Finally, with the exception of Grabe & Christopherson (2007), there is very little comparison of podcasting with other forms of instruction and instructional technology.

Methodology

Based on existing research, several questions have been left unanswered. The results of Lane (2006), Grabe & Christopherson (2007), and Bell et al. (2007) seem to point towards a lack of student engagement when it comes to instructor developed podcasts, while Lee et al. (2008), and Frydenburg (2006) point to the potential motivation benefit of peer and learner developed podcasts. In this proposed research study, the focus will be on studying the motivational response of learners when developing podcasts as part of their class.

There is also the issue of studying podcasts in a vacuum. Without comparisons to existing modes of communication, much of the existing research does not shed any light on whether or not podcasting offers any kind of benefit over what already exists. In this study, a quasi-experimental counterbalanced design will be employed to compare podcasts with computer-aided presentations. Students will create one project of each of the two types, with random assignment determining the order in which they complete them. In order to measure motivation, two measures will be made after the completion of each project: First will be a focus group involving all the students in each group, with content analysis used to identify levels of motivation and amotivation. Second is the administration of the Academic Motivation Scale by Vallerand et al. (1992). With a fairly high test-retest reliability of .79, the AMS can be counted on to reliably measure changes in motivation.

Computer-aided presentations were selected because they are similar in format (one is presentation in digital form, the other is presentation in person), difficulty, complexity, and the use of non-linear editing software in their development. Though computer-aided presentations are relatively new, it does not quite match the newness of podcasting – thus there will need to be compensation for a novelty effect, which may affect internal validity. The instructor(s) participating will need to be aware of the potential for this, so that one technology is not presented in a more positive light than the other.

The development of podcasts and computer-aided presentations is an involved task, and they do not feature in a large number of instructional settings. Therefore, intact class groups of graduate Education students will be used to simplify implementation of the study. This population was also selected because a majority of the previously mentioned studies (Frydenburg (2006), Bell et al. (2007), Lee & Chan (2007), and Lee et al. (2008)) focused their attention on students in Information Technology or Computer Science undergraduates with mixed results. In comparison, graduate Education students may lack some of the technical skill, but be more adept at the presentation, content, and research aspects. In fact, thanks partly to easy to use software and readily available tutorials on the web, the technical aspects of these technologies are rapidly becoming a non-issue.

The class selected to take part in this study will have to meet several criteria. First is the class will need to be taught primarily in a classroom environment in order to facilitate the computer-aided presentations. The content of the class will also need to be broad enough so that students can choose project topics that do not significantly overlap. Though it may seem counterintuitive, the main focus of the class cannot pertain to technology, as one would expect students in such a class to be motivated to use technology to begin with. Potential classes for this study at the University of South Florida will most likely come from the “Educational Foundations” set of courses that include approximately 30-40 students from all majors in the College of Education. Examples of possible courses include Foundations of Curriculum and Instruction (EDG6627) and Foundations of Educational Research (EDF6481).

Conclusion

Based on the amount of money being invested in podcasting in schools around the country, the amount of research that has gone into studying best practices of using the technology has been woeful. Though it is touted as the latest thing to revolutionize education, it is apparent that current incarnations of the technology really provide nothing new. The ultimate goal of this proposed study is to reverse this trend, in order to determine if podcasting really does offer educators any real benefit not found in existing technology and where it might best be utilized.

References

Bell, T., Cockburn, A., Wingkvist, A., & Green, R. (2007). Podcasts as a supplement in tertiary education: An experiment with two Computer Science courses. In D. Parsons & H. Ryu (Eds.), Mobile Learning Technologies and Applications (MoLTA) 2007. Auckland, New Zealand: Massey University. Retrieved October 27, 2008 from http://molta.massey.ac.nz/massey/fms//Molta/Bell.pdf

Edison Media Research. (2008, April). The podcast consumer revealed 2008. Retrieved October 16, 2008, from http://www.edisonresearch.com/home/archives/2008/04/the_podcast_con_1.php

Frydenburg, M. (2006, November). Principles and pedagogy: The two P’s of podcasting in the information technology classroom. Paper presented at ISECON, Dallas, TX. Retrieved September 18, 2008 from http://www.isedj.org/isecon/2006/3354/ISECON.2006.Frydenberg.pdf

Glater, J. D. (2008, August 20). Welcome, freshmen. Have an iPod. The New York Times, p. C1. Retrieved October 19, 2008, from http://www.nytimes.com/2008/08/21/technology/21iphone.html

Grabe, M., & Christopherson, K. (2007). Optional student use of online lecture resources: Resource preferences, performance and lecture attendance. Journal of Computer Assisted Learning, 24, 1-10.

Gura, M. (2006). Moving instructional podcasting beyond coursecasting. Perspectives: New York Journal of Adult Learning, 4(2), 31-33.

Lane, C. (2006). UW podcasting: Evaluation of year one. Seattle, WA: University of Washington, Catalyst Research and Development, The Office of Learning Technologies.

Lee, M.J.W., & Chan, A. (2007). Reducing the effects of isolation and promoting inclusivity for distance learners through podcasting. Turkish Online Journal of Distance Education, 8(1), 85-105.

Lee, M.J.W., McLoughlin, C., & Chan, A. (2008). Talk the talk: Learner-generated podcasts as catalysts for knowledge creation. British Journal of Educational Technology, 39(3), 501-521.

Sadik, A. (2008). Digital storytelling: a meaningful technology-integrated approach for engaged student learning. Education Technology Research & Development, 56, 487-506.

Tufte, E. (2003, September). PowerPoint is evil. Wired, 11(9). Retrieved October 26, 2008 from http://www.wired.com/wired/archive/11.09/ppt2.html

Vallance, M., & Towndrow, P.A. (2007). Towards the ‘informed use’ of information and communication technology in education: a response to Adams’ ‘PowerPoint, habits of mind, and classroom culture.’ Journal of Curriculum Studies, 39(2), 219-227.

Vallerand, R. J., Pelletier, L. G., Blais, M. R., Brière, N. M., Senécal, C., & Vallières, E. F. (1992). The academic motivation scale: A measure of intrinsic, extrinsic, and amotivation in education. Educational and Psychological Measurement, 52, 1003–1017.

That’s a billboard I must pass a dozen or more times a month, but I only recently noticed it. Clearly in these hard economic times, the U is really trying to push distance learning… but this attempt feels more like a feeble shove than a real push.

There’s several other billboards around town, most noticeably the one on I-275 downtown that makes me feel like I’m about to launch into near-Earth orbit. They also had a real big outdoor ad campaign during football season, where they tried to implant the respective images of Matt Grothe and George Selvie into everybody’s brain. During any type of USF game on TV, there’s also a commercial they run made with still photographs (remind you of anything?) and music that sounds like Kraftwerk (unfortunately no one has deemed it interesting enough to post it to YouTube).

I bring this up in order to contrast it with the commercials that Kaplan University have been rolling out:

Really, they’re just breathtaking. Others have reacted similarly, some even go so far as to call it e-learning’s equivalent to Apple’s 1984 ad. It did come with a hefty price tag: the cost for the campaign was supposedly in the millions.

I’m not asking USF to spend that kind of cash. I only ask that they heed some advice from Uncle Phil: