Beta-testing Designs with End-Users

Today, learners demand more customization, voice, and practicality from their learning environments (Kalaitzidis, Litts, & Halverson, 2017). Hence, instructional designers will have to upgrade learning environments in order to meet the demand of today’s learners. As discussed in an earlier post, content creation and calibration cannot be done in a silo. Content that is customized, incorporates the students’ voice, and is practical for students, has to be co-designed with students. Hence, content that is co-designed with learners is the ultimate form of personalized learning.  

Flow theory

Why should instructional designers include learners in the content creation process? First, by including learners in creating content, the learners themselves intrinsically set learning goals for attainment. In other words, when instructional designers introduce learners to the instructional objectives and learning outcomes for the units and lessons, the learners then can determine their own learning because they have been empowered by the instructional designer to customize and practicalize the content and they have been allowed to add their voices to the content creation and learning process.

Second, by including learners in the creation process, a learning flow that produces deep engagement and learner motivation can be established. Csikszentmihalyi (1990) argued that “clear goals, individual control, tasks that the individual is capable of successfully completing, and skills that must be learned” is what establishes a flow for deep learning and engagement. When learners co-create content, tasks are designed that are not too challenging or too easy. Students co-design tasks with teachers that align with their personal interests, thus placing them in a flow channel of learning.

Third, learner voice, choice, and agency are all embedded in co-designed instructional design models, as these types of models highly value empowering learners to make decisions about ends, priorities, and means (Reigeluth, Myers, & Lee, 2017). When students are empowered, then they are more engaged and thereby more capable of attaining their learning goals and the instructor’s teaching objective. 

Alpha vs. Beta testing

In many cases, after instructional designers have created their content without student input, they typically test the content in the alpha stage through the student view. For instance, instructional designers might make sure that the links work, that the dates of content release are correct, and that the aesthetics of the content is appealing. If the content passes the instructional designer’s alpha test, then it is delivered to the student without any trial run. Some would argue that this is a travesty, as students are being held accountable for content that was not given a trial run by the learners. Cars are test-driven, wine is taste-tested, and movies have trailers, all for the sake of testing the quality or operation of the product. Why then are students not given an opportunity to give their content a trail run?

Why content should be beta-tested with students

When instructional designers allow students to co-design and beta test the content, students are able to find bugs and fix them, improve content features, and optimize the distribution of learning, teaching, and assessing (Kalaitzidis, Litts, & Halverson, 2017). “In software development, the beta phase is an accepted, normal, predictable stage of product development” (Gonzalez, 2018). This is not the case in traditional instructional design. Gonzalez (2014) mentioned that “beta is a lifelong commitment to continuous …growth” (para. 4). Hence, shouldn’t instructional designers adopt beta-testing as a form of continuous professional growth? 

After doing some research on this topic, I created an instrument that not only supports mega-batching content creation but beta testing content with learners. For the instrument, click here. I also created a content rubric checklist for students that can be used for beta-testing content. This checklist is based on UC Berkley’s checklist. In sum, if instructional designers truly want to personalize learning for students, then they will not only have to incorporate the learners’ voice, choice, and agency, they will also have to incorporate co-designs that are beta-tested with end-users.

Reference:
Csikszentmihalyi, M. (2009). Flow: The psychology of optimal experience. New York: Harper [and] Row.

Gonzalez, J. (2014). Teaching in Beta: What We Can Learn from Software Developers Retrieved from https://www.cultofpedagogy.com/beta-teaching/ on October 14th, 2018

Reigeluth, C.M., Myers, R. D., Lee, D. (2017). The Learner-Centered Paradigm of Education in Reigeluth, C. M., In Beatty, B. J., & In Myers, R. D. Instructional-design theories and models: Volume IV.

Kalaitzidis, T.J., Litts, B., and Rosenfeld Halverson, E. (2017).  Designing Collaborative Production of Digital Media in Reigeluth, C. M., In Beatty, B. J., & In Myers, R. D. Instructional-design theories and models: Volume IV.

 

Scaling Content Creation

“Learners now demand more customization, voice, and practicality from learning arrangements, and can find it almost exclusively outside of formal, designed education” (Kalaitzidis, Litts, & Halverson, 2017, p. 180). Hence, our current content creation and delivery will have to change in order to meet these new learner demands. As I mentioned in an earlier post, much of what is currently being used is mass produced by publishers. Hence, the current learning arrangements that teachers have with students are strained because the specific learning needs of the students are not being met. 

Current state standards make it hard for teachers to implement learner-centered designs; instructional designs that give learners more customization, voice, and practicality. As Kalaitzidis, Litts, and Halverson (2017) mentioned, students can access much of their sanctioned Grade level content outside of formal education. Therefore, what is inherently valued within standards-based teaching and learning is not inherently valued within learner-centered designs, as learner-centered designs value “a complex system of authentic and legitimate learning activities” (Kalaitzidis, Litts, & Halverson, 2017, p. 183).

Content creation for learner-centered designs
What constitutes authentic and legitimate learning activities? Kalaitzidis, Litts, & Halverson (2017), argued that authentic and legitimate learning activities have learning tasks that constitute the following:

  • tasks are personally meaningful
  • tasks honor disciplinary and/or professional practices
  •  tasks are assessable within the context of the production and learning process
  • tasks are linked to real world practices and communities of practice

For teachers to create such content like the tasks listed above, there will have to be a major overhaul of their current teaching practices. Kalaitzidis, Litts, & Halverson (2017), argue that classrooms need to be converted into workshops,  since this format engages learners in a “collaborative production process through which they may pursue their own individual projects, yet work together toward the same ‘umbrella goal'” (p. 185). Hence, the standards-based teaching format will have to convert to a performance-based learner format where teaching is framed as a mentorship and “the roles and responsibilities of the ‘teacher’ and ‘student’… transform in ways that reflect distributed learning relationships in digital culture” (Kalaitzidis, Litts, & Halverson, 2017, p. 194).

By converting the classroom into a workshop, the leaners’ demand for more customization, voice, and practicality within learning arrangements can be met with a work-shop-style format. This particular format will enable teachers to assume the role of a mentor and distribute learning, teaching, and assessment within the workshop.

  • Distributing learning spreads the onus of learning across the entire class community.
  • Distributing teaching acknowledges and leverages the variations of learner interests as pedagogical opportunities.
  • Distributing assessment expands the objects of assessment to include peer review, audience reactions, mentor notes, and learner feedback about the instructional task (Kalaitzidis, Litts, & Halverson, 2017, p. 197). 

Scaling customized content
Scaling learner-centered designs that promote authentic and legitimate learning activities perhaps can be done with the assistance of the learners. In other words, more customization, voice, and practicality doesn’t have to come solely from the teacher. Simply allowing students to co-design content with the teacher will increase customization and voice in the classroom workshop. On the contrary, more practicality for students may not always be feasible if learning is centered around content and concepts that students deems worthless. Nonetheless, learner-centered designs and the learners themselves can help teachers scale content creation.

Reference:
Kalaitzidis, T.J., Litts, B., and Rosenfeld Halverson, E. Designing Collaborative Production of Digital Media in Reigeluth, C. M., In Beatty, B. J., & In Myers, R. D. (2017). Instructional-design theories and models: Volume IV.

The Perilous Task of Creating Marvelous Online Content

When building courses or media for online consumption, content creation and calibration takes the lions share of development. Depending upon the purpose of the content, its creation and calibration is both perilous and perpetual. Whether it’s a general blog post, video, or podcast, content creation and calibration tends to flow through the Alpha testing stage of development, given that its creator is satisfied with the outcome. The designer of the content most likely has a special affinity for it because he or she has lived with the content for several months; has worked with stakeholders and subject matter experts on the content for several days; and has bore the labor pains during the birthing and delivery of the content for several hours. Given this process of content creation and calibration, it is no surprise that the Beta testing stage of content development rarely happens in academic settings, perhaps because the content creator simply wants the consumer to consume.

Content is the lifeblood of a course, hence, “you want to create something you’re proud of” (Branson, 2018). This means that content worth pride needs to be created in the same place that babies are made. A healthy baby develops in 9 months, as a result of the romance of two. However, in most cases, content creators don’t have the luxury of nine months to develop healthy content. One might ask, what goes into content creation and how often should content be calibrated?

Content creation

Monthly, I contribute content to ulimionline.com and thewordconsciousclassroom.com.  As the old cliche goes, it’s both challenging and rewarding. To help me create content consistently, I used advise from Amy Porterfield. Amy suggested Mega-Batching content by doing the following:

  1. Brainstorm content ideas
  2. Break the content ideas down into six topics
  3. Based on the topic, what question is being answered for the audience
  4. List resources needed to create content for that topic
  5. Decide how the content will be shared (i.e., blog post, video, podcast)

Click here for a link to a planning tool that I created, based on Porterfield’s advice.

Content calibration

Once the content is published, that is not the end of the creation process. It is actually the beginning. Like the development and maturation of a child into an adult, content is constantly in need of calibration and recalibration as contexts and audiences change. It is almost always necessary to reuse, revise, remix, or repurpose content, thus the calibration and recalibration of content is constant. Borrowing from the principles of creative commons, below I define the four Rs for content calibration and recalibration:

  1. Reuse – content might have to be reused across different platforms, hence having an archiving system for content is important.
  2. Revise – content might have to be revised based on updated information or the needs of the audience. Hence, keeping raw content is important for future revisions.
  3. Remix – content may be combined with other material to create something new (e.g., incorporate the content into a mashup) (Wiley, 2018).
  4. Repurpose – content might have to be adapted for use in a different purpose. Hence, blog posts might be repurposed for a white paper or a book chapter.

In sum, content creation and calibration is a constant, and it doesn’t stop once it is published. In other words, in order to fulfill its purpose, content incessantly evolves. Like anxious parents wanting the world to see the beauty of their baby, content designers want users to see the beauty of their content and find purpose with it. To increase the chances of the world seeing their baby as beautiful, some parents might opt for acquiring a designer baby. While content designers may not have that option yet, to increase the chances of beautiful content, some designers might opt to create and calibrate content along with a their content audience, thus extending the romance of content making to its users.

References:

Branson, R. (2018). Greatest Quotes. Business Blogs.Retrieved October 06, 2018, from https://www.businessblogshub.com/2012/09/richard-branson-greatest-quotes/

Wiley, D. (2018). Defining the “Open” in Open Content and Open Educational Resources. This material is based on original writing by David Wiley, which was published freely under a Creative Commons Attribution 4.0 license at http://opencontent.org/definition/.

Digital Media Design and Blended Learning

Blended learning can be defined as a formal education program in which a student learns partly online and partly face-to-face. Blended learning is also an exciting way to personalize learning for students however, creating and calibrating assignments for different blended learning models can be daunting.  Blended learning has many models (i.e., station rotation, lab rotation, individual rotation, flipped classroom, flex, a la carte, or enriched virtual) and assignment/task creation for each model depends upon the purpose of the learning task and the learning path that the students are on. 

Specific blended learning models can be used to fit the differentiated need of learners based upon the instructional learning goal. Using Coil’s (2010) Horizontal and Vertical Differentiation Model, learning experiences can be tailored for specific students according to their specific learning needs. The table below shows how blended learning can be meshed with Coil’s (2010) Horizontal and Vertical Differentiation Model in order to consider how to design and calibrate assignments and tasks.

Much of the current content for assignments and tasks being used for the various blended learning models are pre-packaged by publishers, hence, transferring those materials to a district’s online learning management system as well as converting materials into digital media can be quite time consuming and overwhelming. Hence, when creating assignments for the different blended learning models, the standards of learning, the instructional procedures and the students’ learning stages and learning processes are essential to the creation and calibration of content design and digital media design.

Creation and calibration of assignments and tasks considers whether one should use analog or digital tools within the design while examining best practices for streamlining online assignments with other activities. If digital tools are to be used, then students can help co-design the digital media that will be used to enhance their learning. The following is a list of questions that teachers can use when creating digital media content:

  1. In order for this assignment/task to work, what gaps need to be filled with other tools/strategies?
  2. How will this assignment/task evolve?
  3. Does the assignment/task as intrinsic value for students?
  4. How can learning be distributed to students by students using digital tools?
  5. How will the learning be assessed?
  6. Can assessments be organically built into the assignments or tasks?
  7. How will the assessment be used to enhance student learning?

In sum, Kalaitzidis, Litts, and Rosenfeld Halverson, stated that “Instruction and the design of it should not be rigid, top-down activities. As learners grow, create, and demand new resources, the design of instruction must adapt to meet their needs. Instructors should elicit ideas for these refinements from learners themselves” (p.177). This suggests that instructors should keep assignments and tasks flexible and dynamic.

References:

Coil, C. (2010). Teaching tools for the 21st century. United States: Pieces of Learning.

Kalaitzidis, T.J., Litts, B., and Rosenfeld Halverson, E. Designing Collaborative Production of Digital Media in Reigeluth, C. M., In Beatty, B. J., & In Myers, R. D. (2017). Instructional-design theories and models: Volume IV.

Content Creation and Calibration

Let’s be honest, it is hard to truly personalize learning when there are established standards for learning. Please don’t get me wrong. I am not arguing against standards for learning, I’m simply trying to reconcile the ideas behind the two approaches to distributing and measuring learning. I call this ideological dissonance, the instructional dichotomy. The table below illustrates my ideas on the difference between standardized learning and personalized learning:

Table: The Instructional Dichotomy

Learners vs. Students

Why am I interested in this topic? To answer this question, I must first start with a quote.

“If you consider anyone who is learning at any age and anywhere a “learner,” then you give the responsibility for the learning to the learner. When the institution or anyone who is teaching students are accountable for the learning — not the learners — the responsibility falls on the teachers for what “students” learn. Doesn’t this seem backwards? Where is the incentive and motivation to learn if all the responsibility is on the teacher? If you change the thinking behind the terms, then using the term “learners” makes more sense” (Bray & McClaskey, 2014). 

Bray and McClaskey’s (2014) quote caused me to examine the notion of personalized learning in a standardized classroom.  How can teachers possibly shift the onus of learning from the institution to the learner?  And how can teachers employ individualized instructional procedures that are in sync with each students’ learning stage and learning process? There is no straight answer for this. Nonetheless, Kalaitzidis, Litts, and Rosenfeld Halverson (2017), asserted that today’s communities of learners “thrive upon innovation in tools, meanings, and ways-of-knowing, and favor distributed, emergent, and egalitarian methods for achieving goals” (p. 179). Hence, to authentically personalize learning in a standardized classroom, teachers will have to adopt “design identities” and learn to distribute learning, teaching, and assessment across the entire class community (pp.195-197).  This will require ontological and paradigmatic shifts towards learner-centered instructional designs (Kalaitzidis, Litts, and Rosenfeld Halverson, 2017, p. 182).

Action steps for personalizing learning in a standardized classroom

Developing “design identities” will take time to cultivate and implement in our schools. Nonetheless, I have created some action steps that I believe teachers can use to help address the instructional dichotomy of personalizing learning in a standardized educational field. Let’s begin by stating, true personalized learning starts with the student and his or her learning goal. However, to remain in compliance with the State’s required standards, I suggest starting with the standard and help students contextualize their learning goals within the required State’s standards. The student’s learning goals can be converted into competencies that align with the State’s standards. Below lists the remaining steps in my action plan:

  1. Identify Learning Objectives and Learning Outcomes – aligned with the standards and converted into “I Can” statements
  2. Identify Student Learning Goals and Student Learning Goal Attainment – aligned with the learners
  3. Identify Instructional Methods and Procedures: How will the learning objectives be achieved? (e.g., direct teaching, games, simulations, lab, multimedia, discussions, reading, field trip, drills, demonstration, brainstorming, etc.) – aligned with the standards
  4. Identify Learning Stage: What learning stage is the student in? (e.g. acquisition, fluency (reinforcement), generalization (maintenance), adaptation (experiential)) – aligned with the learners 
  5. Identify Learning Process – Doing something (Concrete Experimentation), Thinking about it (Reflection), Doing some research, Talking with others and applying what we already know to the situation (Abstract Conceptualization), Doing something new or doing the same thing in a more sophisticated way based on our learning (Active Experimentation) David Kolb – aligned with learners
  6. Teacher provides feedback to student – Feed-up, Feedback, Feed-forward; (Feed-up – clarify the learning objectives; Feedback – Respond to Student Work; Feed-forward – Modify instruction for future lessons). – aligned with the standards
  7. Student provides feedback to teacher – Feedback – aligned with the learners

In sum, I still believe that it is hard to truly personalize learning for students when there are established standards for learning. Nonetheless, to help address this instructional dichotomy, I suggest that we continue to start with state standards and help students contextualize their learning goals within those standards.

References:

Bray, B., & McClaskey, K. (2014, March ). Building Personalized Learning Environments. Retrieved July 10, 2018, from http://www.advanc-ed.org/source/building-personalized-learning-environments

Fisher, D; Frey, N. (2009). Feed Up, Back, Forward. Educational Leadership. Retrieved September 12, 2018, from http://www.ascd.org/publications/educational-leadership/nov09/vol67/num03/Feed-Up,-Back,-Forward.aspx

Kalaitzidis, T.J., Litts, B., and Rosenfeld Halverson, E. Designing Collaborative Production of Digital Media in Reigeluth, C. M., In Beatty, B. J., & In Myers, R. D. (2017). Instructional-design theories and models: Volume IV.

Y Design-Oriented Models?

What do design-oriented models look like, sound like, and feel like? I’ve been contemplating the answers to those questions for about two weeks now. Richard Schank’s book, Teaching Minds: How Cognitive Science can Save our Schools, as well as Katie Muhtaris and Kristen Ziemky’s book, Amplify: Digital Teaching and Learning in the K-6 Classroom have helped me to generate a few answers to those questions.

y-1019702_1920.jpg

Looks like:

  • models that support digital communities
    • user-designers, (e.g. content users help to design their learning experiences)
    • online learning and social media
    • incorporation of new literacies
    • develop digital citizens
  • models that support the cognitive process
    • making predictions
    • building models of a process
    • experimenting with information based on failure or success
    • evaluating information on many different dimensions
  • models that support the analytic process
    • making a diagnosis of a complex situation
    • constructing explanations
    • learning to plan
    • conducting needs analysis
    • goal setting
    • detecting causes of events
    • making objective judgments
  • models that support the social processes
    • creating influence within a group
    • working as a productive team member
    • handling conflict
    • practicing negotiation
    • describing problems precisely

Sounds like:

prolific language used to

  • compliment
  • question
  • coach

Feels like:

  • an emphasis on student ownership and creativity
  • student empowerment
  • interdisciplinary learning
  • personalized assessment
  • authentic assignments and projects
  • collaboration
  • abundant access to resources
  • continuous reflection
  • divergent and convergent thinking
  • envisioning, understanding, and communicating meaning
  • inquiry and problem-solving
  • content area experts

Schank reminds us that, “lifetime learning does not mean the continual acquisition of knowledge so much as it means the improvement in one’s ability to [employ cognitive] processes by means of the acquisition and analysis of experiences to draw on. Design-oriented models will help teachers to craft those experiences for their learners in a web-enhanced classroom.

Reference:

Muhtaris, Katie, and Kristin Ziemke. 2015. Amplify: digital teaching and learning in the K-6 classroom.

Schank, R. C. (2011). Teaching minds: How cognitive science can save our schools. New York: Teachers College Press.

close

Enjoy this blog? Please spread the word :)

Get new posts by email: