During the 2012 California eLearning Census, we were not surprised to find that the predominant blended learning model was the Self Blend (now renamed A la Carte). After all, eLearning was just reaching the tipping point and a significant number of eLearners were disruptors, those students whose needs weren’t being met by traditional schools.
In 2013, though, we’re finding that districts and direct-funded charters are becoming more comfortable with online and blended learning. Numbers of involved students have grown within each district and 34% were utilizing more than one blended model. When we disaggregated the data, we found significant differences between how elementary (k-8) and unified (K-12 and 9-12) districts were blending. Some of this data is listed below. The complete California eLearning Census Report may be found here.
We encourage you to join us at the 2013 eLearning Strategies Symposium that will be held December 6th and 7th at the Hilton Costa Mesa. Governor Bob Wise and Dr. Eric Mazur will keynote on Friday, December 6th. Early-bird registration is just $159 and groups of two or more may register for $149.
While 46% of all districts and charters are implementing online and blended learning, we asked those that aren’t if they were currently discussing or planning to eLearn. While overall 26% said they were, the disaggregated data shows that 44% of unified (K-12) and high school (9-12) districts were actively engaged.
Today, CLRN published the 2013 California eLearning Census report, and you’ll see out executive summary below this post. Download the full report here. Below is the first of five, themed infographics that detail census data that was collected from 516 school districts and direct-funded charters.
Online and blended learning, growing 20% to 30% yearly, have reached a tipping point. CLRN’s California eLearning Census, conducted between March and May this year, received responses from 30% of California’s school districts and direct-funded charters. While we found that 45% of all districts and charters are utilizing some form of online or blended learning, we were surprised that of those districts not currently eLearning, one-third were in the planning stages to create online or blended programs. Yes, online and blended learning are trending and many teachers and districts are jumping on the eLearning bandwagon.
Flipped classrooms and the Khan Academy have received national attention as teachers place classroom lectures online and change classroom pedagogy to include project-based work. You may even be thinking of creating an online or blended course yourself. After all, you’ve taught for many years and you’re a master of your curriculum and teaching craft, so those skills should benefit you in creating an online course.
Remember how, during your first year of teaching, you spent countless nights creating lesson plans and units only to throw most of them away the following summer? Remember how difficult that first year was? Multiply that times 10 and you’ll have your first year of online or blended learning. I’m not saying, “Don’t do it.” I’m saying that you should go in with both eyes wide open, following the advice I’ve shared below.
We speak from experience. We know a lousy course when we see one. The California Learning Resource Network (CLRN), a state-funded technology service, reviews online and blended courses for their alignments to the Common Core standards, California’s original standards and to iNACOL’s Standards for Quality Online Courses, which we helped write. Under a new partnership with the University of California, online courses must be CLRN-Certified before U.C. will consider them for A-G approval. However, to date, only 25% of the courses we’ve reviewed have received our certification. So, if commercial publishers have so much difficulty creating a high-quality, engaging, and interactive course, what makes you so special?
By all means, dream large. Online and blended learning provide opportunities often unavailable in small and medium sized districts. Opportunities to take an AP course or any world language should be available to all students, not just those in large or affluent districts. Begin with the end in mind, though.
Most successful projects begin with a thorough planning process that engages stakeholders, reviews research, and carves out a plan that solves a specific problem. Planning for online or blended learning is no different.
This year’s Keeping Pace , an annual census report and analysis of the online and blended landscape, includes a proposed 18th month planning process with specific tasks for each phase. In the Systemic Planning stage, you bring together stakeholders and perform a needs analysis that asks: 1) What’s the problem you hope to solve? 2) What is your educational goal? 3) Who are the intended student groups? and 4) What are your district’s capabilities and desires?
That’s just the beginning, though, because before creating a solution, you also need to assess your technology infrastructure, your students’ and teachers’ technology skills, the availability of quality, standards-aligned resources, and teacher professional development.
But, assuming you’ve completed a planning process and targeted specific student groups or courses to affect, what’s next?
Normally, I’d suggest at this point a discussion about whether to build or to buy. Should you build courses from scratch (and do you have the capabilities to do so) or should you shop for quality courseware that you can pilot for a year or more?
But, you’ve already made the decision to create a lousy course, so let’s proceed.
Get Thee a Learning Management System
Whether you rent an existing course management system like BlackBoard or install an open source solution like Moodle or Course Builder, an LMS is a framework that contains your content, activities, and assessments, allowing you to track student progress and conduct online asynchronous and synchronous meetings. Whichever direction you choose, spend time mastering all the LMS’ components: installing curriculum, creating class rosters, embedding outside activities, and setting up discussion groups. Don’t start without an LMS, though.
Standards-aligned, engaging content can be purchased from a publisher, found in open source repositories, or created in-house. With iNACOL standard A2 stating, “The course content and assignments are aligned with the state’s content standards..”, you want to make sure that the content you provide students not only teaches (demonstrates) a skill, but also provides students opportunities to practice and assess each skill or standard. CLRN’s reviews include these three components of each standard identified for a course.
Your textbook is not a course though. While textbooks are aligned with the standards and may include practice activities and assessments, placing your book online, be it commercial or open source, is amateurish, at best.
Quality courses will include text though, but not entire chapters printed screen after screen. The better courses CLRN have reviewed include portions of text mixed with video lecture clips, streaming video, simulations, games, and short formative assessments. Creating quality online lessons is a much more time-consuming task than creating face-to-face lessons. Provide ample lead-time to create online lessons.
Online course standard B3 states that course instruction and activities must engage students in active learning, including authentic projects and activities that challenge students beyond knowledge and comprehension. Rather than focus primarily on multiple-choice tests for assessments, it’s best to provide students knowledge work where they create, evaluate, and analyze. Students should regularly participate in online discussion groups, be they synchronous or asynchronous.
Just What Part of the “Accessible” Memo Didn’t You Get?
All teaching and learning materials must be accessible to all students. Period. If you’re creating video lectures, streaming video clips, or providing narrated presentations, each must either have closed-captions or a transcript. Online standard D10, and the Department of Justice, expects it and your students deserve it. Sites like Universal Subtitles are easy to use and allow your captioned videos to play from their site, or you may download the time codes to upload to YouTube.
While you may feel like you’ve mastered your craft when teaching face-to-face, teaching an online or blended course requires a different skill set and mastery of different tools. In an online poll we conducted, online teachers recommended that newly converted online teachers master the following tools before beginning to create an online or blended course: 1) SlideShare or a similar online presentation tool; 2) Collaborative meeting tools and related skills to set-up and conduct online discussions; 3) Portfolio creation tools for students to assemble examples of their authentic work; 4) Synchronous presentation skills because teaching “live” to online students offers completely different challenges and requires new solutions; and 5) Universal Subtitles for creating closed-captions.
One avenue of professional development is the Leading Edge Certification (LEC) for online teachers. The 45-hour LEC course includes units in online pedagogy, building an online community, accessibility, assessment, and preparation. Based on iNACOL’s Quality Standards for Online Teachers, LEC provides an opportunity to become a highly-qualified online educator.
Online and blended learning are growing quickly for a reason. These courses can help personalize learning, allowing schools to vastly expand their course catalogs, and providing students the opportunity to learn any time, any place, any path, or at any pace. We understand your eagerness to provide an online or blended option to your students. Before jumping into the water, though, we just ask that you learn to swim. Anyone can create a lousy course. It takes time, talent, and perseverance to create a great one.
As we shared yesterday, CLRN’s review process strives to be fair, unbiased, and non punitive. The CLRN Certification process is one way to shine a light on those courses that meet most of the course and online standards. With CLRN being a state-funded service, we do not charge for our review services. Still, course reviews are time and resource intensive, particularly when you’re striving to be thorough and consistent. While we hope all publishers will continue to improve their courses to be more interactive, engaging, and responsive, we’re unable to conduct full reviews of the same course when publishers slightly improve a course to better meet the standards. Because we want to continue to offer our services at no cost, we’ve modified our policies to provide publishers a narrow window when they’ve made specific improvements to meet the CLRN Certified criteria. Our new policy is listed below.
Secondary Review: New evidence for content or online course standards.
When CLRN’s review teams examine a course for its alignment to content standards and to iNACOL’s Standards for Quality Online Courses, they utilize publisher-provided correlation documents, also known as standards maps. These detail where reviewers should find evidence to confirm a standard’s presence. When a standard is not fully met, CLRN review teams provide a comment about components that are missing.
Once CLRN’s review is complete, we inform publishers that the course review will be published in seven calendar days. At this point, publishers may notify CLRN to publish the review immediately or they may allow the seven days to proceed with CLRN publishing the review afterwards.
However, if a publisher believes that there is new evidence, not previously submitted, that would clarify our review, they should notify CLRN during the seven-day window. CLRN will place a “hold” on the review and will schedule a secondary review for the specific standards for which the publisher provides new evidence. Once this secondary review is completed, CLRN will again inform publishers that the review will be published in seven days. Each course is allowed one secondary review.
Supplemental Update: New content and/or course standards.
Once a course review is published online, CLRN generally does not allow the resource to be re-reviewed for three years unless there is a major update to the course. However, in the case of courses that have not met the qualifications to be CLRN-Certified, CLRN may re-review courses provided the following:
Publishers have augmented the course to strengthen its alignment to the content standards and/or the course standards. Sufficient changes have been made to bring the course into compliance with the CLRN-Certified specifications.
Publishers provide specific evidence and locations about the standards that have been added or strengthened.
CLRN’s first priority during review sessions is to examine new course reviews. CLRN will conduct supplemental updates on a space available basis. Courses will remain in their published state during this time and publishers will be notified of the review results. No secondary review is available for supplemental updates. One supplemental update is permitted per course per year.
Major Course Update.
Courses that have experienced a major update contain substantially new content to meet the Common Core State Standards or California’s original content standards. The course’s structure, media integration, degree of interactivity and engagement, and learning management system have been significantly improved to better meet iNACOL’s course standards, particularly the 11 Instructional Design standards.
Major Course Updates are considered new courses and are reviewed in the order they are received.
With yesterday’s announcement of CLRN’s partnership with the University of California, online courses must now be CLRN-Certified before U.C. will review them for their A-G requirements. That we’ve set the bar at 80% of both the content standards and the online course standards may be seen as too generous. However, we believe it’s a fair starting point, given that not all standards are equal. The addition of our online course Power Standards requirement ensures that critical standards are met in order to earn CLRN-Certified status. We believe our partnership will help drive course quality and provide stronger courses to our customers.
So, how have CLRN’s currently reviewed courses fared?
Over the past year, CLRN has reviewed 55 online courses from seven publishers. 15 courses from six publishers, 27% of the total, have earned the CLRN-Certified badge. You can easily find certified courses through our search page . Simply select the “CLRN Certified” check-box and narrow your search by subject or grade level. In addition, we’ve added a CLRN-Certified badge to all certified course reviews.
11 additional courses from four publishers met the 80%/80% requirements for content and course standards, but failed to earn certification because of a single Power Standard: online course standard D10, which requires that course materials are accessible to all students. In each of the 11 courses, instructional lectures and/or narrated presentations did not include closed captions or transcripts, which we believe is an easy fix. If each of these courses are modified to meet D10, and we hope they are, 47% of our reviewed courses will be CLRN-Certified. For more information about why D10 is important, read “What Part of the Accessible memo didn’t you get?” from my “Advice for Course Creators and Buyers” post.
The most common problem with the majority of courses that are not certified is with their alignment with the content or Common Core State Standards. While there are exactly 52 online course standards, the number of content standards per course varies between 19 for AP Probability and Statistics to 106 for ninth grade English-language arts courses aligned to CA’s original standards. Regardless of the number of content standards, the 80% requirement applies. 20 courses (36%) failed to teach 80% of the content standards for their course. These range from a low of 34% to a high of 78%. The median (half the courses had more, half had less) percentage of content standards met was 57%.
As we mentioned in the beginning, we hope our partnership with the University of California helps drive course quality and improvement. While the CLRN’s reviews are effective for three years (and certification ends three years after a review is published), CLRN typically will not re-review a course unless there is a major update. However, because we don’t want the certification process to be punitive, we’ve created new policies to encourage course improvement. We’ll share these in tomorrow’s post.
It’s been quite a year for the California Learning Resource Network. After investing a year to create the standards and review process for online courses, CLRN began reviewing English-language arts and mathematics courses last summer, spending much of last year training our reviewers and norming the process to ensure consistency. Last spring, we added history-social science, science, and visual and performing arts courses and plans are under way to add world language reviews this winter.
During 2011/12, CLRN reviewed 55 online courses from seven publishers: Accelerate, Aventa, Education2020, K12, Inc, Pearson Digital Learning, Plato Learning, and Thesys International. Our 2012/13 review queue currently holds 88 course submissions, including many from Advanced Academics, Apex Learning, Cambium Education, National University High School, and Odysseyware. Still, many of our customers have asked us about course approval for the University of California’s A-G requirements, since A-G approval is required for courses students take for admission to the University of California. We now have an answer.
To achieve CLRN-Certified® status, online courses must address at least 80% of the course’s content standards (Common Core or California State Standards) and 80% of iNACOL’s Standards for Quality Online Programs. Fifteen online “Power Standards” must be among those verified by CLRN. These Power Standards include: Content: A3, A9 & A13; Instructional Design: B3, B4, B5 & B10; Student Assessment: C2, C3 & C4; Technology: D4, D10 & D11; and Course Evaluation and Support: E3 & E10. Course publishers may utilize the CLRN-Certified® term and logo in association with their certified products.
By request of the University of California, CLRN reviews will now include the “Not Met” content standards within posted reviews. Also displayed on CLRN’s browse page and within each course review, will be the percentage of content and online standards met. CLRN-Certified® courses will include both a reference to its CLRN-Certified® status and the new CLRN-Certified® badge.
With online and blended learning growing at double-digit rates and sliding past the tipping point, publishers and schools are both getting into the course creation business. It’s a little bit of the Wild West out there and we expect new publishers will appear on a regular basis. Remember when there were dozens of textbook companies? Eventually, the market consolidated to the three primary textbook publishers we see today, but until that happens, we’ll be tracking eLearning’s evolution. Still, like early digital cameras just before their tipping point, some eLearning courses are primitive, two-dimensional, and, well, boring.
There. I said it. Some emperors have pretty ugly clothes.
However, as this disruptive innovation continues to mature, we strongly believe that courses will increase in complexity and sophistication. Capitalism and competition will drive publishers to increasingly raise the bar, offering more engagement, media, and assessment choices.
For now, though, through our experiences with a wide variety of online and blended courses, I’d like to share some thoughts about what a great online course should (and should not) be, whether you’re creating your own or if you’re about to go shopping for one.
1. A Textbook is not a course
What if a teacher threw a book at you and said, “Read this and take the tests at the end of each chapter. When you’re finished, you’ve passed the course”? Would you choose that teacher or curriculum for your child? Would you recommend that course to others? What are the odds that an average or struggling student would complete the course, or better, be proficient in its skills? What do you think the AYP of this class would be compared with an engaging face-to-face course? A great course should include textbook content, but if teaching and learning doesn’t extend beyond the page, and the course is primarily limited to reading, you’re better off throwing a kid a book and locking her in a room with it.
So, if a textbook isn’t a course, why do so many courses look and feel line one?
2. Your course should be better than the worst teacher in my school.
The Ferris Bueller clip below is my generation’s idea of a boring teacher.
Aim higher than this economics teacher. Much higher. Please. Because in some cases, I’d rather send my child to this f2f class.
3. There can’t be quality outputs without quality inputs.
I understand the outcomes-based performance indicators movement promoted by iNACOL, Michael Horn, and others and I look forward to an agreed-upon set of evaluation criteria that can be applied to both virtual and brick-and-mortar schools. Still, until these metrics are created and standardized, we’ll all be subject to our current criteria: course completions, graduation rates, drop-out rates, and Annual Yearly Progress (AYP).
But for me, it’s Garbage In, Garbage Out. You can’t have quality outputs unless you begin with quality inputs. As Charles Babbage was asked, “Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?”
Students learn more from active discussions than from traditional lectures, and they need instructors who can engage them in the material, notes Diane Johnson, assistant director of faculty services at the Center for Online Learning at Florida’s St. Leo University. “Teacher quality is still a very important part of success in an online course, but so, too, is the course design,” Johnson says.
Teacher quality & course design. Quality In. Quality Out.
4. Just what part of the “Accessible” memo didn’t you get?
Your school built wheel chair ramps even though you may not have had any students with that need because some day a parent, student, or teacher WILL have that need. You also did it because the Federal government made you, but it was the right thing to do, yes? Of course. We want all students to have equal access to a quality education.
Should that be any different with online courses? eLearning’s equivalent of a wheel chair ramp is accessible media, particularly narrated presentations and video lectures.
Accessibility requirements for electronic media was confirmed by a US Department Of Justice letter to college presidents. In it, the DOJ reviewed its lawsuit against universities that had piloted the Kindle DX, which the DOJ won because the Kindle DX did not have a text to speech function. The letter states, “It is unacceptable for universities to use emerging technology without insisting that this technology be accessible to all students.”
Accessibility criteria for online courses was pioneered by the Texas Virtual School Network (TxVSN) which has created a detailed site, TxVSN Accessibility, with advice, criteria, and check lists. Here are their Accessibility Guidelines.
One of CLRN’s requirements when reviewing courses is that all narrated presentations and videos include transcripts or Closed Captions.
So, yes, when you’re constructing an online course, your customers, and the Office of Civil Rights, expects that course materials are accessible. There are free resources to assist you, including Universal Subtitles, which can be used to embed captions within YouTube videos.
Make use of them.
5. Consistency between courses is key
We’ve noticed that often two online courses taught in sequence, like English I and English II or Algebra I and Geometry, have very different levels of interactivity and engagement, as if they were created by two very different teams who weren’t talking to each other. While one course may have a variety of media, narrated presentations, and video lectures, the companion course would not. The basic pedagogy of each course would be the same, as if course developers were working from a template, but one course would be significantly weaker than the other.
My advice? Hire great educators to develop your courses and make sure a separate person is in charge of quality control. Include textbook snippets, but make sure lessons are embedded with short video lectures, discussions, collaboration opportunities, and media.
6. Formative Assessments Should Inform Instruction
Too often, as in most cases, they don’t.
If a student is struggling, a formative assessment is where we first discover his difficulty in grasping a concept or idea. If, after failing a formative assessment, your course just passes the student on to the next unit without 1) reteaching or 2) informing the teacher there may be a problem, you’re competing with my worst teacher for last place.
There’s a reason teachers love “clickers”. They enable teachers to instantly assess whether students comprehend concepts and they offer “just-in-time” teachable moments if students don’t.
Formative assessments, generally short multiple-choice quizzes, don’t have to be graded. They DO need to be an opportunity to determine whether a student is grasping a concept. When students have difficulty, the course should attempt to reteach using new examples or new methods until the student is ready to move on.
Otherwise, your course isn’t much better than a book.
7. Vary Your Assessments
You do know there are other assessment types besides multiple-choice, yes? Then why do so many courses use multiple chose tests exclusively? It’s fairly common for a course to have frequent formative assessments, consisting of a few multiple-choice questions that gauge a student’s progress. That’s great, and often appropriate. However, when multiple-choice tests are used exclusively in a course, you’ll never know whether a student has synthesized course concepts.
There’s a reason principals frown when teachers park themselves at the school Scantron machine scoring quizzes. Sometimes a multiple-choice test IS the best way to test knowledge. However, it’s never appropriate to use it exclusively. A great online course should have a variety of assessments, some which provide feedback about knowledge and comprehension, while others challenge students to apply, create, analyze and evaluate. The less a test is “Googleable”, the better chance you’ll have of actually evaluating student progress.