Are Rubrics Generalizable?

As we have learned over the past several weeks, rubrics or other scoring plans are essential tools in the online learning environment. According to Boettcher and Conrad, rubrics lay out the criteria to be used for grading assignments, and are also useful for students as a self-check (2010). For this discussion please review and evaluate the rubric posted here. A mock assignment given to graduate-level students of an Educational Technology course asks the students to prepare a 3-5 page essay detailing the benefits and features of various online learning tools.

As you review this rubric carefully consider the following:

• Is the rubric an effective scoring tool for the mock assignment? Why or why not?
• Is there evidence that the rubric is generalizable?

Focusing on the overall design and utility of the rubric, discuss whether you believe the rubric is generalizable such that different instructors would be able to score the assignment consistently. In other words would multiple instructors scoring the mock assignment reach similar conclusions and scores when using the identified rubric as a scoring tool? Or could instructors score the assignment inconsistently because of bias, or the language used in the rubric?

By Friday, June 17:

Post your comments on the generalizability of rubrics overall. Then, based upon your review and evaluation of this week’s assigned rubric, post your thoughts on whether multiple instructors might score the mock assignment consistently or not. Support your thoughts by citing information from this week’s or prior week’s resources.


View Discussion Scoring Rubric Here

Reference

Boettcher, J., & Conrad, R. (2010) The Online Teaching Survival Guide. Simple and Practical Pedagogical Tips. Jossey-Bass. San Francisco:CA

Online Cheating and Plagiarism

As an instructional designer for a non-profit trade association, how concerned about online cheating or plagiarism should I be? Citing from The Council of Writing Program Administrators, Jocoy and DiBiase reported that “Plagiarism occurs when a writer deliberately uses someone else’s language, ideas, or other original (not common-knowledge) materials without acknowledging its source” (2006, p. 2). On line cheating behavior, on the other hand, usually refers to students “cheating on examinations followed by estimates of total cheating, [or] cheating on homework” (2006, p.3). As I design online courses to be taken by professionals in my field, should I take into account that some learners may not practice the highest levels of academic or professional integrity in their studies?

Well, although plagiarism detection software is currently available on the market, it probably will not be of much use to me. Turnitin, for example, is such an application. Turnitin evaluates written work submitted by students and identifies sections where there is potentially matching text. Turnitin compares the student's submission to an online web-based repository of research/term papers and other published or unpublished resources. Learners who take online courses I've designed are not asked to submit research or other academic papers. Instead, they are frequently required to review case studies and provide answers to specific questions using critical cognitive skills such as evaluation and synthesis. Still, the potential to plagiarize exists, even under those circumstances. However, in the real world of business and commerce, perhaps we are less concerned with plagiarism and more concerned with a learner’s ability to construct and integrate knowledge, and apply that knowledge directly to his or her work.

What about cheating? Well, again, there is that possibility. The assessments that I design are not objective assessment instruments such as you might find in academia with multiple choice, true/false or yes/no options. The assessments that I prefer are designed to evaluate learning through instruments aimed at evaluating core competencies. So, I might use activities such as the construction of tables or checklists for learner evaluation.

In a video presentation titled “Plagiarism and Cheating” Palloff and Pratt discuss strategies they’ve used to prevent academic dishonesty. They discussed their practice of encouraging learners to conduct extensive and robust web searches in order to thoroughly explore course topics and content. Both reported concerns about, and occurrences of cheating or plagiarism, but maintained that the best prevention strategy is to design open book or collaborative assignments and assessments. They also stated that assignments should be designed with application-type questions . . . pointing out that these types of questions require students to apply information learned directly into the assessment or assignment.

Online facilitators certainly need to be aware that students can and do cheat and plagiarize. Facilitators should be on the look-out for such behaviors. I believe that the strategy of designing collaborative activities for learners can be highly effective in preventing cheating and plagiarism. Of course we know that learner collaboration is a best practice in the online learning environment. So, this solution – online collaborative projects – appears to be a win-win.

References

Jocoy, C., & DiBiase, D. (2006). Plagiarism by adult learners online: A case study in detection and remediation. International Review of Research in Open & Distance Learning, 7(1), 1–15.

Palloff, R., & Pratt, K. (2010) Plagiarism and Cheating. Laureate Education, Inc. Retrieved April 7, 2011 from: http://sylvan.live.ecollege.com/ec/crs/default.learn?CourseID=4894963&Survey=1&47=6223382&ClientNodeID=984650&coursenav=1&bhcp=1

The Impact of Multimedia

I really like the use of video and audio in online learning and hope to incorporate both in future courses I develop. Multimedia lessons have the potential to quickly engage learners and stimulate their interest. Technology, and in particular, Web 2.0 technology, easily facilitates the use of multimedia components into online learning environments. Boettcher and Conrad maintain that “Using audio and video tools creates a richer, more interesting, and more satisfying course experience” (2010, p.140) for learners.

According to an online source, “Mayer’s cognitive theory of multimedia learning presents the idea that the brain does not interpret a multimedia presentation of words, pictures, and auditory information in a mutually exclusive fashion; rather, these elements are selected and organized dynamically to produce logical mental constructs” (www.learning-theories.com/cognitive-theory-of-multimedia-learning-mayer.html). Stated elsewhere, Mayer’s theory simply asserts that individuals learn more deeply when they process both words and pictures simultaneously, versus processing words alone. Mayer’s theory goes on to discuss dual channel processing and the notion that we receive information through our eyes and our ears (visual and auditory channels). Engaging both channels simultaneously leads to deeper learning, hence the power and appeal of multimedia in online learning.

These days it is very easy to add multimedia elements to online courses. Take YouTube for example. YouTube is such a powerhouse – a fantastic repository of almost every type of video imaginable. Currently, I am putting together the educational programming for my organization’s annual conference. Part of my work involves selecting a keynote speaker. Over the past two or three weeks I have poured over dozens of videos in search of the perfect speaker, many of which were found very easily on YouTube. (As a quick aside . . . here is the link to a YouTube video I highly recommend for you. I found this one during my search; it’s about 18 minutes long and discusses leadership in a very interesting, poignant and engaging manner. Enjoy, when you get the chance … http://www.youtube.com/watch?v=qp0HIF3SfI4). We have a basic online supervisory course that I am updating; I can’t use this speaker or the video for the conference, but because of the ease of using YouTube videos, I can quickly embed this video into our supervisory course. Right now the course has no audio or video elements, so this will be a welcome addition, I am hoping.

As excited as I am about incorporating multimedia into my online courses, I do not advocate the use of media without a well-thought through plan about its purpose. In fact one of the most important considerations an online instructor should make before implementing multimedia is whether the media support the learning objectives. Most learners enjoy the interactivity and sense of authenticity afforded by multimedia. But as online instructors, we must ensure that the media add to, enhance, or support the content and the objectives of the lesson before adding the bells and whistles of multimedia with no consideration of its purpose.


References:

Learning-Theories.com. Cognitive Theory of Multimedia Learning . Retrieved March 30, 2011 from: http://www.learning-theories.com/cognitive-theory-of-multimedia-learning-mayer.html

Mayer, R. E.; R. Moreno (1998). “A Cognitive Theory of Multimedia Learning: Implications for Design Principles”. http://www.unm.edu/~moreno/PDFS/chi.pdf.

Moreno, R., & Mayer, R. (1999). “Cognitive principles of multimedia learning: The role of modality and contiguity”. Journal of Educational Psychology 91: 358–368.
Mayer, R. E. (2001). Multimedia learning. New York: Cambridge University Press.

A Few Considerations for Setting Up an Online Learning Experience

Setting up an engaging and interactive online learning experience requires planning and careful attention to detail. One of the first things to attend to is creating presence. In fact, Boettcher and Conrad argue that “One of the best ways to get an online course off on the right foot is to ensure the social presence of the instructor and all the learners” (2010, p. 54). Social presence gives online learners the sense that they are interacting with “real, live, breathing” people. It is the "ability of learners to project their personal characteristics into the community of inquiry, thereby presenting themselves as real people" (2010, p.76). Further, social presence is an essential component of the online community trust-building process (Boettcher & Conrad, 2010).

The second thing to attend to concerns technology. Find out what technology tools are available to you, the facilitator, and other learners. It would be an excellent and worthwhile venture for instructors and facilitators to take an inventory of the tools available to them. For example, can the course be built using a course management system (CMS)? This is important since using a CMS could prove very beneficial to instructors and students because CMS are generally feature-rich and powerful in terms of course delivery. Another question to ask is, “Can multi-media vignettes be built into the course?” Multimedia is generally highly compelling, and has the potential to quickly engage students in the learning experience. Other technology tools, especially those made available by Web 2.0 technology such as wikis, blogs, discussion groups, streaming video, and social media should be explored, and assessments should be conducted to determine how and when these tools can best be incorporated into the online course.

Of course simply taking an inventory of the technology tools currently available is not enough, although it is a good first step. After you’ve determined what is available, you must select the tools that are optimal for your learners – tools which support your learning objectives. For instance, if you’ve assigned students the task of exploring a topic through research and interviews, and have asked them to write up their findings, as well as comment on the work of their peers, a good technology tool to use would be a wiki. The optimal tool might be a blog, although both tools may support the learning objectives.

Another equally important consideration here, though, is learners’ access and comfort with the tools being used. Instructors and facilitators must determine:

1. whether students know how to access technology tools (typically accessible through the internet)

2. whether students can use the tools

As an example, we might believe that YouTube is now ubiquitous, and that everyone knows how to access videos on that site. The reality is that not everyone knows how to access videos on YouTube or through other applications even. Or it could be possible that a learner knows how to access a video, but does not have a set of speakers built into or attached to her computer. Or perhaps speakers are attached, but the learner does not know how to change the settings to get them to work properly. All of these are likely scenarios and must be taken into account when using technology tools.

Although this discussion is not a comprehensive one by any stretch of the imagination, for purposes served here the final thing to attend to when arranging an online experience involves setting up clear expectations of learners. “Clear and unambiguous guidelines about what is expected of learners and what they should expect from an instructor make a significant contribution to ensuring understanding and satisfaction in an online course” (Boettcher & Conrad, 2010, p. 55).


Reference

Boettcher, J. V. Conrad, R. (2010). The Online Teaching Survival Guide: Simple and Practical Pedagogical Tips. San Francisco, CA: Jossey-Bass.

Online Learning Communities

Active participation in an online learning community can impact student learning, and greatly increase satisfaction with online learning programs. An online learning community can be defined as a shared space on the Internet where students and instructors can connect with other, explore content, build knowledge and work together to addresses the learning needs of its members through collaborative partnerships (Pratt & Palloff, Wikipedia). One of the more significant ways that online communities help impact learning is by providing an open forum through which members provide professional support, correction and feedback “in a way that enhances learning and development” (Palloff & Pratt, 2010). Another way, according to Shrivastava, is that individual and group members can “voluntarily accept mutual responsibility for participating and sharing in the learning process.”

Satisfaction with online programs increases when learners actively participate in an online community. This is likely attributed to the belief that students feel part of something larger than themselves when they participate in an online learning community (Palloff & Pratt, 2010). Also contributing to learner satisfaction is the idea that online communities provide viable support systems for students. According to Wellman & Guilia), “ Students who take an interest in each other’s well being, both academic and social, will have more of a support system of peers than those who do not” (1999).

Which strategies should be used to create a successful online learning community? There are two essential strategies to employ. The first would be to foster individual social presence, which is the sense of knowing who the other persons are in the community, virtually, but in a real way. Learners who are able to get a good sense of their online partners, perhaps through a photo, or by learning interesting personal or professional tidbits about them, are more empowered to contribute freely, expressing themselves in authentic and powerful ways, thus contributing to an active and successful community. The second is fostering a sense of shared responsibility for the community. Community members must come to the knowledge and understanding that it is everyone’s responsibility to create a successful online learning community.

Building an online learning community supports effective online instruction. The online community gives learners an opportunity to contribute to one another’s learning, the community’s learning overall, and also to contribute to a specific body of knowledge. Research shows that stronger outcomes are the consistently observed results of active online community participation (Palloff & Pratt). Conrad & Donaldson (2004), report that “each learner’s actions contribute not only to individual knowledge but to overall community knowledge development as well” (p. 5). According to Vesely, Bloom & Sherlock, “The power of establishing a sense of community in online classes has been demonstrated by LaRose & Whitten (2000) who found a statistically significant relationship between students’ sense of community and the positive achievement of learning outcomes in the online setting” (2007). Could it be that a successful, fully active online learning community, in which members are highly engaged, involved and contributing regularly, can serve as an effective proxy for face-to-face interaction? The answer appears to be a resounding “yes.”

Hi everyone,

This is my blog. Feel free to comment as you like.

Scope Creep?

In one of my earlier blog entries I described a project I was involved with while volunteering with a former employer—a very small, charitable, non-profit organization that seeks to end domestic violence in the State of Illinois. The project was the organization’s annual fundraising event, a glitzy, evening gala featuring hors d’oeuvres, an open bar, live music, a raffle and a silent auction.

A gala committee, comprised of board members and others, was put together during the initial phases of the project. During one of the first meetings of the gala committee—still the concept phase, a board member suggested that the group seek out local celebrity chefs for a “friendly competition among chefs,” an idea that was met with much enthusiasm and excitement. The board member making the suggestion happily volunteered to recruit two or more chefs to participate in the event, whom she felt confident in securing. Well…six months into the project plan and we still had no celebrity chefs contracted. What we had was one chef who expressed a small bit of interest in the event – no real commitment at all.

As mentioned, the organization is very small and had a very small event budget. The amount allocated to food and drinks was as close to zero as you can get, since the budget was developed with the expectation that all wine and alcoholic beverages would be donated (as they had been in the past). Our expectation that the celebrity chefs would donate the food items they prepared at the event was also well-founded, given the fact that such chefs had done so in past years for other non-profit organizations.

Seventy-two hours prior to the event and we had only one chef confirmed. We decided to quickly contact the venue hosting our event; we scrambled to develop a menu to supplement what our sole celebrity chef was donating. This dilemma may not qualify exactly as being an example of scope creep since scope creep is generally thought to be the “natural tendency of the client, as well as project team members to try to improve the project’s output, as the project progresses” (Portny et al, 2008. P.346). However, according to Lynch & Roecker, scope creep “is a major cause of cost and time overrun” (2007, p.96 ). And so it was for this project…except that we might want to label this dilemma “scope retreat” instead of scope creep, since we had to scale way back in terms of our hopes of having a competition. And we had to add a very significant amount to our budget’s food allocation in order to make up for the losses in donated hors d’oeuvres and other edibles.

Looking back on how we could have handled this differently…. Everyone was very well-intentioned and tried their best. Perhaps a well-thought through contingency plan would have been appropriate. But so sure was our board member in her ability to secure local chefs that we never truly considered a “Plan B.” Mistake on our part? Definitely. A costly one? You bet! Did we learn from this? Absolutely. We should always allocate a contingency amount for an event—you never know if an expected donation will actually come through.


References

Lynch, M. M., & Roecker, J. (2007). Project managing e-learning: A handbook for successful design, delivery, and management. London: Routledge.

Portny, S., Mantel, S., Meredith, J., Shafer, S., & Sutton, M. (2008). Project Management in Practice: planning, scheduling and controlling projects. Third edition. Hoboken, NJ: John Wiley & Sons, Inc.

Communicating Effectively

When working as a manager of an instructional design project, it is extremely important to establish and implement (and continuously improve) communication channels with all stakeholders. “A project fraught with communication problems leads to delays, misunderstandings, frustration, [and] workplace conflicts…”(Sharma, 2010).

Effective communication is vital to the success of any project. Project managers can communicate with stakeholders through written communications such as email, instant messaging, memoranda and reports. Verbal communication methods include face-to-face dialogues and recorded messages. Within the realm of verbal communication is interactive communication, where individuals can interact with each other in real time.

You may be wondering which method of communication is best for a given situation or purpose. Well, generally speaking and for most situations, interactive communication (face-to-face) is best. For example, this week in my instructional design course I was asked to review a message in three different modalities: as written text in an email, as audio in a telephone message, and as a video (simulated) face-to-face message. As you might imagine, the video delivery method was most effective for me for a number of reasons including:

1. I was able to observe the other person’s facial and body expressions making the communication more authentic, sincere and earnest.

2. Observing the person while simultaneously listening to the message made the entire experience seem “real” and I wanted to provide the information requested because of the “reality” of the message.

3. Because of the other person’s physical presence (albeit virtual) I felt more of a ‘sense of urgency’ than with the other communication delivery methods. (In the message, the other person was asking for some needed data in order to complete a report which was quickly approaching its deadline.)

Many types of written communication are considered “push” delivery methods because they are presented to (or pushed to) stakeholders. The push method is effective as well, and often, is the most efficient way to reach a larger audience. For example, sending a written progress report to all the stakeholders of a large, complex project may take considerably less time to deliver than setting up and conducting a formal meeting and discussing the project’s progress. Back to my assignment…when I read the assigned message in email form, although I understood it, the tone seemed almost apologetic. I don’t believe the intent of the message was to be apologetic. Right away then, you can see how one communication method differs from another in how its message is received and interpreted.

Lastly, I would like to mention the telephone message, which is effective as well as it is a form of verbal communication. But, even if effective, recorded voice messages are not the most effective as we’ve already learned. With recorded voice, or audio, there is more of an element of authenticity than in written communication because the voice on the other end is human and real. However, recorded messages are not heard in real-time, and thus cannot offer the benefits of interactivity. For my assignment, when I listened to the recorded message, I believe that I did understand the message as it was communicated, by that I mean I understood the urgency of the message. Because there was no interactivity, I could not clarify with the other person the full intent of the message—I was forced to use my own discretion as to what the message actually meant. A disadvantage? Perhaps…but not in this assignment.


References:

Sharma, R. (2010) Effective Communication Methods in Project Communications Management. Retrieved November 18, 2010 from: http://www.brighthub.com/office/project-management/articles/85785.aspx