Category Archives: Pedagogy

The LMS and Feeling Good

All the talk about learning management systems (LMS) around the office lately reminded me of a dataset a couple of colleagues and I put together last year. Dr. Florence Martin, Dr. Yuyan Su, and I undertook the task of validating an instrument to measure LMS self-efficacy.

Bandura (1997) defined self-efficacy as beliefs in one’s abilities to carry out a desired course of action. I’ll spare you the details of orthogonal exploratory and confirmatory factor analyses.

One of the many variables we decided to examine was whether student LMS self-efficacy was a predictor of course performance. After all, is not learning the primary motivation for using a learning management system?

Reported self-efficacy was generally low. However, students enrolled in hybrid courses reported significantly lower self-efficacy than students in face-to-face or fully online courses. In addition, for students enrolled in hybrid courses, we found a significant positive correlation of LMS self-efficacy with course performance.

It is perplexing that a significant positive correlation occurred only for the hybrid learners. One would think that the use of the LMS as a supplement to face-to-face instruction would require less confidence with the system than in a course in which all content is delivered though the LMS.

Hybrid learners often had the option to enroll in a fully online version of the course but self-selected into the hybrid version. Is this due in part to their lower self-efficacy with the LMS? Or does it mean there is a baseline competence with LMS use required for success, but once that level is perceived to be reached, greater self-efficacy with the system is not required?

Finally, the only instrument category that did not yield a significant difference between modes of delivery was “Accessing Information.” This section included items like logging in to the LMS, navigating a course site, accessing text-based class materials and grades, etc. This was also the highest rated category for self-efficacy. We hypothesize that this finding is an indication of the predominant use of a LMS throughout each student’s experience. As suggested by Bandura (1997), the formation of self-efficacy beliefs is based primarily on reflection on and interpretation of past performance.

In my pervious post, I referenced the Raslton-Berg & Nath (2009) report that says students are uninterested in the bells and whistles in online courses. But consider further the abundance of media-comparison “studies” and no-significant-difference studies that essentially nullify each other.

Is it possible that students actually do like the bells and whistles but lack the confidence to learn from them?

What a pickle.

References

Bandura, A. (1997). Self-efficacy: The exercise of control. New York: Freeman.

Ralston-Berg, P. & Nath, L. (2009). What Makes a Quality Online Course? The Student Perspective. Paper presented at Annual Conference on Distance Teaching and Learning, Madison, WI.

Avatar photo

Teaching Frustrations: Why Don’t Students Follow My (Clearly-Labeled, Logically Organized, and Bold/Highlighted/Flashing) Instructions?

Instructors who teach in online environments often devote extensive time and energy into designing a Web space that is inviting and useful to students. But frustration inevitably ensues when, despite the careful consideration given to the most logical placement of a discussion forum and the “clearest” instructions provided to students on how to post to the forum, the instructor still receives e-mail from students asking, “So, where is this discussion forum? And what am I supposed to do?” Why has this gap in communication occurred?

One reason for this may be the typically linear design of course sites. Often, learning-management systems adopted by universities have default settings that establish some of the design considerations for the instructors—i.e., the location and style of course navigation. These linear designs generally have the best intentions, since they try to organize information so that students can navigate course material easily, following step-by-step instructions and information.

However, with recent developments in eye-tracking software showing how users really view content on the Web, we can see why this linear design isn’t quite ideal. This video shows a user’s eye movements when scanning IKEA’s Web site, and several other examples available online confirm this rapid pattern of eye movement that jumps all over the page. It’s no wonder, then, that students miss the carefully placed, bolded, and highlighted instructions for turning in an assignment that you were sure everyone would see and follow—considering how the brain ingests and computes information from the screen, it’s easy to see how a linear design style for course materials might not match the ways in which users view the content.

So, what is the solution? Unfortunately, there isn’t a Band-Aid design scheme that addresses this issue, and because instructors are often working within an institutionally mandated learning-management system, course design happens within set boundaries. One important step is usability testing, which can reveal issues that designers can’t see once they are invested in their design decisions. This may seem like an onerous and time-consuming task, but it doesn’t need to be—usability guru Jakob Nielsen recommends five users for testing, but as this data shows, even finding two or three people to look at your course and perform key tasks can give you helpful information to improve your course design.

Another important step is realizing that, just as in face-to-face classrooms, your goal (for students to follow instructions) needs to clearly align with your assessments:

  • Include instructions in a logical location, as determined by your course design.
  • Ensure that students have seen these instructions. One effective method is to give students a graded quiz at the beginning of the term that asks them to locate important information throughout the online course.
  • Show students that following instructions is important by grading them on it. Depending on your class, you might make part of an assignment’s grade based on following the assignment’s instructions, or you could refuse to accept an assignment until the student has followed the directions.

Again, there isn’t a one-size-fits-all solution for designing courses that adhere to the ways users view information on the screen. This also isn’t a “lost cause” for instructors—just because users naturally view Web content in a nonlinear way doesn’t mean that the design of online course materials needs to be completely overhauled. Thoughtful design can help students, but supporting your design with clear expectations and assessments can also help students navigate your course more effectively.

Avatar photo

Putting a Concrete “Why” in Front of a Necessary “How”: Ideas for Faculty Technology Training

“Often faculty don’t need more training on the tool, they need more training on the affordance of the tool and how to use it to support learning.” Patricia McGee, associate professor from the University of Texas, made this statement while offering tips for training faculty on teaching with technology in the newsletter Higher Ed Impact: Weekly Analysis, published by Academic Impressions.

What she said about learning the tools versus learning the affordance of the tools reminded me of a lot of trainings and conference presentations I have attended, which are usually made up of a lengthy PowerPoint presentation followed by a little bit of product/project demo. The PowerPoint usually covers vendor introductions, the tool’s primary functions displayed as bullet points, a theoretical framework or the background of the product/project (sometimes), the implementation process, and eventually, student feedback. If I am lucky, I might be able to get a few screenshots of the site or a quick run-through of the final project, but often these come at the very end. While a big introduction does help build expectations, without any concrete examples, it is hard for me to understand what exactly this particular technology could bring to my own teaching practice.

Compared to academies, tool providers seem to do better at addressing the issue of affordances up front. If you’ve read Melissa Koenig’s blog entry Story-Telling Tools—Beyond PowerPoint, you might have noticed that almost all of the tool sites incorporate a good number of samples on their home pages (check out PhotoPeach, Gloster, and Toondoo). This shows that the tool producers have figured out the best way to capture the attention of today’s busy and impatient Web visitors—by showing (instead of “telling”) them what has been done by and with the tool. The only challenge here is that many of the examples are for a “general” audience instead of being targeted at educators. Examples of faculty and student use of technology for instructional purpose are usually not presented in one collection. However, that does not mean that they cannot be found (Isn’t it a general rule that you can find anything on the Internet?). It is up to the trainer to locate the appropriate examples that could get instructors thinking, “How should I use this in my class?”

Speaking of selecting appropriate examples for faculty, Patricia McGee provided another practical tip in the article—adopting a tailored approach. Offering generic examples of educational use of the technology is not good enough, since faculty in different disciplines will have different needs. One type of technology that works well for one content area may not work for another. Given the various needs of different disciplines, Patricia McGee pointed out that campus-wide training might not be the ideal option. This is exactly why we developed a tailored DePaul Online Teaching Series (DOTS) program with a well-matched combination of technology, pedagogy, and content knowledge (TPACK) and implemented a liaison model to embed technology consultants in schools and colleges. Now it is time to bring the same tailored mode beyond the systematic program (such as DOTS) and implement it into all training events.

According to the  CDW 21st-Century Campus Report, faculty’s lack of technology knowledge remains the greatest campus technology challenge perceived by students, and training is the type of support most needed by faculty. Whether faculty training is useful has become a determining factor for how successful technology integration on campus is. The answer to this could be as simple as a tailored training curriculum structured in a meaningful sequence. The one I’d like to propose includes the following three easy steps:

  • Step 1: Provide concrete and relevant examples (a demo of the affordance)
  • Step 2: Pause to choose the best tool for meeting instructor needs
  • Step 3: Train on the use of the chosen tool and the necessary technology

Confessions of an Online Student: Voyeur or Classmate?

Until I had to withdraw due to family obligations, I recently spent four weeks as an online student in one of DePaul’s Cinema and Digital Media courses. While much of the experience was positive, I’m left with some negative impressions as well.

Readers of my earlier posts know that while I design multimedia for DePaul’s School for New Learning online program (SNL Online), I’m usually not enthusiastic about actually taking online courses myself. I normally like the experience of sitting in a physical classroom and interacting with my classmates. For this course, though, I felt that online would be perfect. It was a subject that didn’t lend itself to a lot of group interaction and discussion. There were clearly defined learning objectives supported by a comprehensive textbook, appropriate learning activities and assessments, and a proprietary LMS that would deliver recorded classroom sessions with video, audio, whiteboards and presentation screens that I could view at my convenience. I would read, watch, and produce. What could be easier?

Well, the online classroom experience was completely unsatisfying. I had anticipated being able to supplement the readings and clarify key concepts and directions by downloading and efficiently viewing the classroom presentations. I had, in fact, found this to be a useful perk when in the past I’d taken CDM courses on-ground, where I could note the time a key concept was discussed, then search for and review it later.

This time, however, with my only classroom contact being virtual and asynchronous, I found that I was by turns bored or frustrated. Removed from the distractions of a live classroom I was struck by how much of a three-hour class was filled up with empty space; the instructor shuffling papers or searching for files or waiting for something to load from the Web. Painful waits for students to respond to questions seemed to stretch into hours. And while there were certainly segments of the recording that were useful, there was no way of knowing where those might actually be without sitting through the entire session. It seemed to me that there was about a three-to-one ratio of dead air to useful information. This was not what I’d anticipated.

Oh, and did I mention the actual recording quality? As I peered through my two-inch video portal, I strained to see the instructor, hear what was being said, and make out what was being written on the whiteboard. Though each session could be displayed full screen and had zoom capabilities, the video was very low quality and heavily pixelated even at smaller display sizes. The whiteboard captured input intermittently. Adding to my frustration, the instructor would physically interact with projected data, pointing out and clarifying important equations and processes that the in-class students could follow, but weren’t captured clearly by the video or whiteboards. The online section of this course was an afterthought, it seemed. I felt more a voyeur than a participant.

That said, I’m planning to take the same course online next quarter. But I now know that the online component is really an afterthought, that I’m really on my own for learning the material from the textbook and exercises, and that I’m essentially taking a correspondence course. Because just putting a recording of a classroom session online does not make it an online course. And just watching one makes you more a voyeur than a classmate.

Avatar photo

Back to Basics: Free Tools I Can’t Live Without

It’s easy to get excited about the educational potential of new Web 2.0 tools. So many tools appear (and disappear) from month to month, and I often find myself promoting and supporting bleeding-edge tools for instructors who are still struggling to use some of the basic features of Blackboard. So in an effort to keep things simple and avoid putting the cart before the horse, I’ve been trying to focus on projects that offer more bang for my instructional-design buck.

For example, Sarah (one of our amazing grad-student workers) and I are currently helping several Spanish professors convert their paper-based exams into Blackboard quizzes with audio. This quarter, over a hundred students are taking their exams in computer labs on campus, saving instructors lots of grading time and giving the students more immediate feedback on their strengths and weaknesses. It has been great to see this project come together, and it feels like the kind of low-hanging fruit that all instructional designers should be working harder to pick before we attempt to coax a neo-Luddite, tenured professor into running an entire course through Twitter and Posterous.

Yet as much as I love keeping things simple, there are a few Web 2.0 tools I keep coming back to because they’re relatively easy to use and/or they offer features that faculty regularly request. Here’s a very short list of the tools that, at least for me, make the cut and are worth the extra effort.

VoiceThread

While PowerPoint and Keynote remain the best tools for developing presentations, VoiceThread is the most reliable and user-friendly option if you need more than one-way communication. VoiceThread’s in-browser recording makes it easy for users to add narration presentations, and the option for viewers to add text, audio, and video comments is unmatched by other free tools.

VoiceThread’s only major downsides are that students are limited to a maximum of three VoiceThreads with free accounts and that images with fine details (like small text) will often be too blurry to read when uploaded and displayed in the VoiceThread interface.

Viddler

I’ve done a lot of Web 2.0 tool training with non-tech-savvy instructors, and I’ve never had a training session go as smoothly as it does when I’m covering Viddler. Getting users from account creation to recording and embedding their first videos usually takes roughly fifteen minutes with a group of fifteen instructors. The in-browser webcam recording works like a dream. For a quick video intro or comment that needs to be added to an announcement or discussion-board post with minimal fuss, Viddler just works.

PBworks

If you need a wiki for collaborative writing or Web-site building, PBworks is the place to go. They’re the industry leader, and they do what they do very well. Google docs works just fine for sharing simple documents like research papers and presentation outlines. But if you’re looking for a robust tool that makes it easy to create and edit a one- or one-hundred-page Web site, PBworks is the tool for the job. My only hesitation in recommending PBworks these days is their feature set continues to grow, and I’m concerned they’re starting to overwhelm novice users with an abundance of features.

Avatar photo

For an Online Course, Does the Look Impact the Feel?

Good-looking Web pages—the ones with stylish layouts and eye-pleasing images—are more likely to retain viewers and even get people to perform actions like buying something or submitting a form than the ones that are plain and makeup free. Is this true or false?

Some interesting research on this question was performed recently by John Broady of Omniture Digital, who ran multivariate tests on “Request for Information” forms for two online universities. For each test, the goal was to increase the number of users who completed the Request for Information form. For the same content, one site had stylized page design, “hero images” (glamour shots of good-looking people in seemingly natural settings), colored buttons, and benefits message while the other had just information in text.

The findings of the research, according to John Broady, seem to render no significant result at first glance.  “The results for the two tests could not have been more different,” he wrote. “For one university, the page with the stylized page design and lifestyle hero image won handily; for the other university, the simple page design with no hero image won the day.”

However, when the researchers looked beyond the random phenomenon and dug deeper into the data, another interesting finding emerged: “for the page where the stylized design and the lifestyle hero image won, most of the traffic came directly from search engines; for the page where a simple design and no hero image won, most of the traffic came from other pages on the university’s own Web site.”

From a marketing perspective, this indicated different responses to the look of a Web page from two different clienteles: the shoppers led by the search engines and the existing or recruited customers already wandering in the company’s territory. For the first group, the visual impact of a page is a key success factor. Since they only have a few seconds to spare on the page, a good-looking design with comforting images can make a huge impact. Education Services Reputation Management can also help increase online exposure and improve trust for potential users. But for the ones who are already familiar with the company through visiting its other Web pages or by other means, the visual impact of this particular Web page becomes less important. According to John Broady’s analysis, for users who “have likely already qualified themselves and are looking to convert”, too many visuals (even the pretty ones) and reinforcing messages (even the well-written ones) can actually create a distraction for these types of users. So in this case, simple is better.

What does this research tell us about online course design? Does the look of a course impact the feel of its audience or does it, too, depend on who the audience is? An online course usually has two audiences: the reviewers and the students. Obviously the two groups arrived on the course site for two different but related purposes: the reviewers are there to check on the quality of the course, of which the look is likely to be an influential factor (even if there isn’t a criterion designated for the appearance in the review standards); the students, on the other hand, are there to use the product—as long as it is functional, they might be able to ignore the look of it.

The look, however, is usually the first thing to attract the author of an online course. “I want to make my course look like your DOTS site (the Blackboard site for the DePaul Online Teaching Series program).” Faculty would say this during the training and be totally sold on lesson-building tools like Softchalk, which transforms a plain page into a professional-looking Web display through some quick magic-wand clicks. However, the enthusiastic demand for a copy of Softchalk usually dies out after a while, as faculty start to realize that time is running short and they need to get the content online very quickly. The “look” then is thrown out the window but is told that it would be invited back next time when there is more time. When the next time comes, the story repeated itself with the “look” still waiting and the faculty feeling bad about it all over again.

As online educators grapple with the aesthetic appeal of their courses, similar attention to detail can be found in the design and allure of cool Georgia. The state itself presents a blend of charming aesthetics and practical innovation, much like the ideal online course. Georgia’s diverse landscapes, from the tranquil Appalachian Mountains to the urban chic of Atlanta, encapsulate a natural and cultural vibrancy that’s as appealing to the senses as a well-designed online interface. Here, the visual feast is not just in web pages, but in the tapestry of live oaks draped with Spanish moss, the historic cobblestone streets of Savannah, and the modernist architecture of the High Museum. In Georgia, the ‘look’ is not something to be sidelined for later—it’s an integral part of the experience, drawing people in with its Southern charm and keeping them engaged with its dynamic, ever-evolving spirit.

The good news from John Broady’s report is that it puts our faculty at ease to know that the students could care less about the look of a site as long as the right content is there. On the other hand, however, the look is often beyond the cosmetic display of the content; it represents an easy-to-follow and meaningful flow of information, which is known by a lot of faculty members to be a critical factor for learning. For those faculty who have the desire to grant their course a sleek and professional look but have no time to create it, here is my advice: check in with your instructional designers and make them your cosmeticians for an extreme makeover of your online courses.

The Customer is Always Right?

Last month, I attended a presentation by Penny Ralston-Berg at the 25th Annual Conference on Distance Teaching and Learning, titled: “What Makes a Quality Online Course? The Student Perspective.” Her study, coauthored with Leda Nath (Raslton-Berg & Nath, 2009), asked students to describe their level of agreement with the current Quality Matters standards for online courses and the level to which elements within each standard contributed to their overall success. I was interested in getting this look at online courses from the student perspective to perhaps glean some useful implications for my own design. What I walked away with was a disturbing reinforcement of the competing global motives for my role as an instructional designer and online educator.

As expected, students highly valued technology that worked; clear, consistent navigation in their course sites; and instructions on how to access resources. It was what students found least valuable that caught my attention. Based on this survey, online students do not want to:

  1. Find course-related content to share with the class
  2. Use wikis, shared documents, or other collaborative tools
  3. Introduce themselves to the class
  4. Coach other students
  5. Attend synchronous meetings
  6. Interact with games and simulations
  7. Work in groups
  8. Receive audio or video content

Surprised?

I was. Could this be a call to remove the interactivity and engaging content from our courses? Despite the research, does social presence not matter? Should we return to online learning circa 1996? Are these elements really that repulsive to our students?

Or could it be that they are so frequently misused we’ve given them a bad name.

I know how I would feel after being besieged with a sixty-minute talking head in a three-inch square frame; after suffering though a pointless game for the sake of the instructor being able to check the “included game in my course” box on a rubric somewhere; or after participating in a meaningless, unguided group activity in which I do all the work and my group mates get the same grade.

This cry from our constituents, we want engaging, interactive content in our courses. Just give it a purpose.

Maybe the customer is right.

References

Ralston-Berg, P. & Nath, L. (2009). What Makes a Quality Online Course? The Student Perspective. Paper presented at Annual Conference on Distance Teaching and Learning, Madison, WI.

Quality Matters rubric standards 2008-2010 edition (2008). Retrieved from http://qminstitute.org.

The complete findings are also available at http://www.slideshare.net/plr15/what-makes-a-quality-online-course-the-student-perspective-1829440

How Do I Know My Students Are Learning? 

Oh, isn’t this the ultimate question for any teacher!

Trying to “keep it real,” a small group of DePaul Teaching Commons souls put our heads together recently to create this resource!

No two teachers approach these questions in the same way so—trying to “keep it real”—this site provides several different approaches.

Approach 1: From “what you want to know” to “what you need to assign”

There are two columns. The left column is what you might want to assess; the right column provides some examples of what you might want to assign. As an example, if you want to assess the student’s “application of discrete research, technical, performance, or meta-cognitive skills,” the Web site suggests you might want to assign case studies, debates, observing a performance, presentations, or simulations and role plays. Find this list on the Evaluation of a Product page.

Approach 2: What can I do right now with what I have?

The site has multiple examples organized by chronology (during a class or throughout the quarter), technology (blackboard surveys or other survey tools), or writing to learn or learning to write! As a former writing instructor, I was particularly impressed with the writing examples.

Approach 3: My students are not learning. Now what?

You’ve gathered the information via surveys or other assessment techniques, but your students are not learning? The site covers some next steps. Analyze the information, and act on that information. Some of these options can be found on the Are my students Learning? page.

Softchalk’s Update

Being an instructional designer requires me to have many tools at my disposal to create exciting and meaningful course content. Often, content needs to be displayed in a “chunked” manner to make navigating through the material easier. And it’s nice to have something that is visually appealing as well. For this, I’ve found myself using Softchalk. As with any product, it has things it excels at, and it has limitations. Recently, Softchalk 5 came out, and I was quite excited.

One huge thing I have been waiting for is the ability to name my pages, and I was thrilled to learn that if you look under “Properties” you will find a “Page Names” option! However, I also quickly learned that it still didn’t do exactly what I was looking for. While it gave the page names in the table of contents, the navigation at the top was still in number format. But it’s a step in the right direction, at least.

Another feature of this new version is the eCourse Builder. With the eCourse Builder, you can create multiple modules with the same navigation—in essence, combining your content into one area. While this may not work the greatest in Blackboard due to having too many frames on your screen, I can see some great applications of this feature for displaying large amounts of content in certain areas. It can also help work around the page-name issue mentioned above.

One strength of Softchalk is the ability to put interactive items into your module, such as flash cards or labeling activities. This is great for adding some variety into your lessons and using more visual media. You can also tie the quizzes you create to the Blackboard gradebook using the SCORM packaging option, as well.

So while Softchalk is not perfect for every situation, I think it does have some very nice features that warrant its inclusion in the instructional designer’s toolbox.

Building Social Media for Students: A Waste of Time?

Perhaps it’s the end-of-summer’s-approaching ennui or plain old cranky, middle-aged contrariness, but as I witness the barnstorming enthusiasm for Facebook-like social media on display at any given online-learning conference and contrast that with the drumbeat reports of Facebook’s declining popularity, I can’t help but think that some of us are living in a state of denial.

I think our intent is good. We want to serve our students, we want to make it easy for them to communicate, we want to create a socially cohesive learning environment, and we want to give them the tools they need to succeed. We think we know our students; we think we know what they want. So let’s build our own social sites!

I’m afraid it’s wasted effort for the most part. Here’s why.

First, we’re replicating existing services and efforts. My department has ruminated for months about a social site for our adult students. Well, surprise! Students who wanted a social space have already created their own Facebook group, demonstrating again the truism that individuals can and do move faster than committees. Will these students abandon the group they created for a university-branded one? I’m betting not.

Second, we’re too late to the game. Facebook is hemorrhaging members, as the cool kids move on.  Twitter is the heir apparent; fast, flexible, and mobile. It certainly has great potential; see James Moore’s excellent presentation at http://preview.tinyurl.com/mg74tv . And as mobile devices become more ubiquitous, you’ll see more and better apps like MobilEdu, created by Terribly Clever Design and recently acquired by—wait for it—Blackboard.

So what does Blackboard know that you and I should? When to recognize that the game has changed. Blackboard realized they couldn’t design a better mobility app than the whiz kids from Stanford and stopped wasting time trying to. They’re free of denial and playing to their strengths.

We should play to ours.