Pascale Swanson

Tag: AfL

  • Assessment for Learning (AfL) in Instructional Design and Learning Experience Design

    Assessment for Learning (AfL) in Instructional Design and Learning Experience Design

    Businessman hand attending video conference with laptop computer and tablet comouter at desk in office

    A couple weeks ago I wrote a post about three types of assessment common in the Ontario school system: Assessment as Learning (AaL), Assessment for Learning (AfL), and Assessment of Learning (AoL). That post described all three and explained their importance. If you haven’t read it yet, I suggest you start there.

    Over three weeks, I will break down each of the different types of assessment, as it pertains to instructional design and learning experience design. Last week I began with AoL. Now I’ll move on to AfL. Before I delve into AfL as it is used in instructional design and learning experience design, let’s review where the term came from as it relates to teaching in the primary and secondary school system.

    Description of AfL from the school system

    AfL is an informal assessment of what a learner has learned from an individual topic or task. This type of assessment is either unscored or a mark is given simply for the learner’s own knowledge (but it does not count toward a final grade score). In either case, the emphasis is on descriptive feedback.

    The focus of AfL is on the process of learning. The AfL step helps learners identify their strengths and weaknesses, and it ultimately helps the learners focus on what they need to know to succeed.

    AfL in instructional design

    When it comes to the instructional design environment, quizzes, in-class activities, and interactive eLearning activities are commonly used for AfL. Let’s look at some considerations for AfL in instructional design.

    1. Assessment must be based on the learning outcomes

    It is essential to create assessments that are strongly based on the learning outcomes. At the start of the course design process, you should have written some learning outcomes for the course. These outcomes, the instruction, and the assessment should be closely aligned. For more information on this topic, check out the AoL post from last week.

    2. Feedback

    AfL is all about feedback. In some cases, the assessment may be marked, but whether it is or not, the most informative feedback comes in the form of comments. In an instructor-led course, this may mean debriefing after each activity to try to understand why the learners made certain decisions. In an eLearning course, this may mean providing written feedback – ideally feedback that is both immediate and personalized.

    Consider the two examples below. Try getting the answer right and wrong for each question. Which example made you feel more satisfied with your learning? [note that the try again button is there so you can try both a correct and an incorrect response, but it would be removed in an actual assessment, particularly in example 2]

    As you can see from the second example, the feedback is in fact part of the learning process. It is particularly helpful when the feedback is specific to the choice made. In this example, there were 3 different ‘incorrect’ feedback layers, each one was based on a different incorrect answer.

    You may wonder about the learners who decide not to read the feedback. They see the checkmark or X and move on to the next question. Although those learners do indeed exist, not all learners will skip reading the feedback. Therefore, by providing feedback to everyone, we are allowing learners to make their own choice.

    3. Gamification

    In the post of AoL, I skipped gamification because it really works best in a formative assessment capacity. Any time that there are countdown timers, score cards, leaderboards, etc. the stress levels of the learners can increase. Additionally, the learners may become distracted with these extra features and spend less time pondering each question. This is not a conducive environment for summative assessments. However, when learners are still in the process of learning and the assessment is AfL, games can be a nice, light, fun way to perform assessment.

    I recently completed a workshop by Clint Clarkson, who is a master of game thinking in learning experiences. One of his takeaways was that almost everything can be turned into some form of a game (although as an ID or LXD, some discretion is necessary depending on the subject matter).

    My personal preference for gamification is all-encompassing games with a storyline and, most importantly, a purpose. Let me explain my preference with an example of what doesn’t work well, and then compare that to what I would do to fix it.

    When workshopping ideas for a learning-based virtual escape room, someone suggested that learners should answer multiple choice questions about the subject matter each time they tried escaping the room.

    Now, in my opinion, this makes the learning and the game very disjointed. I don’t see the value of the game itself in this instance. The decisions that the learners make related to how to escape are not at all related to the decisions they need to make regarding the subject matter. You may as well just give them the multiple-choice questions and skip the game.

    My idea for learning-based virtual escape rooms is that the learners must use their content-based knowledge to actually escape the room. For example, in an escape room based on ladder safety, the learners would have to select appropriate ladders, inspect them, safely carry them and use them properly in order to get out of the escape room (for example, through a high window). The learning content is 100% related to the escape room game. In these types of instances, the escape room is essentially a massive scenario-based assessment.

    Regardless of the format of the game, without a direct relationship to the content, gamification can seem superfluous or inauthentic. Remember that the point of the game is to assess learning – which means that the content should be upfront and centre.

    4. Scenarios

    In my post on AoL, I discussed the value of scenarios for summative assessment. Almost every point there would apply to formative (AfL) as well. So, I will avoid repeating myself here (although check out this post to see what I had to say about scenarios in AoL) and simply bring in one additional point: consider how you will provide the feedback.

    Scenarios can be built in a variety of ways. One way involves setting up a decision point, which allows learners to branch out to make their choice, but then after giving feedback, the scenario contracts back and continues in the same way, regardless of the choice made by the learners.

    This strategy is simpler and faster to create, and also provides excellent opportunities for immediate and personalized feedback.

    However, in some cases you may want the scenario to be more immersive. In which case each decision leads the learners on a different path, with dozens (or more) possible pathways through the scenario.

    While I generally prefer these types of scenarios, they make it much more difficult to provide immediate feedback unless each node of the branching scenario starts with a very obvious indication of whether the last choice was correct. In these instances, it’s important to remember that feedback is one of the essential components of AfL. Therefore, a thorough debrief should be presented at the conclusion of the scenario, based on the actual choices made by the learner.

    5. Tracking results

    One final consideration for instructional designers is whether or not the AfL will be tracked. Obviously AoL would need to be tracked to determine if learners have achieved the learning outcomes. However, AfL does not have this same requirement.

    While tracking AfL results does add in a layer of complexity in both the in-class or eLearning environment, it can be beneficial to know if learners are struggling long before they reach the AoL. This could provide opportunities for support and coaching. Additionally, when it comes to evaluating the success of the program, it can be useful to know which outcomes the learners struggled with throughout the course (not just on the AoL).

    AfL in learning experience design

    Learning experience design takes instructional design one step further by considering the learner experience. Since AfL occurs within the learning experience, let’s look at a few ways that can we ensure the learner is having a positive experience on the AfL.

    1. Clear purpose and plan

    Ensure that your learners know why you are doing the AfL. Let them know what will be assessed and how the assessment will work – particularly for gamified or scenario-based assessments. Additionally, let them know how they will be receiving feedback. Also, and importantly, ensure that they know if and how they are being marked.

    2. Clear instructions

    There are almost an infinite number of ways to do AfL. Your only limit is your imagination. However, what seems clear and straightforward to you, might not seem so clear to your learner. Let them know how to complete the AfL and what your expectations are. This is true for both instructor-led and eLearning courses. Again, this is particularly important for gamified and scenario-based assessments.

    3. Next steps

    Ensure that your learners understand what their options are after completing the AfL. Considering that AfL is not a final assessment, but rather than internal assessment to verify that the learners are on track, it is worth having a variety of options for your learners.

    For example, learners that successfully complete the AfL may continue on with the course. Learners who just barely miss the passing mark might be able to return to certain sections of the course for review. Learners who are far from passing might need to review everything before moving on.

    Ensure that the learners know what the options are and how to access the different options. Additionally, for learners who decide to complete some review before moving on, make a clear statement about whether and how the AfL will be redone.

    4. Accessibility

    The final point related to learning experience design is to ensure that the course is accessible (particularly with eLearning courses). Since the sky is the limit in terms of AfL choice (gamification, scenario, drag and drop, multiple choice, etc.), you might want to use accessibility standards as a helpful tool for deciding which methods to use.

    Of particular concern are any type of drag and drop, image matching, and any style of AfL that incorporates audio or video.

    Consider your learner analysis and/or learner personas, as well as any company- or government-mandated accessibility standards. All of these can help you pick the best AfL for your learners from the start (and reduce time spent redesigning non-accessible courses).

    Conclusion

    Well, like last week, this post also turned into a long one! If there is one thing to take away from this novel, it’s that AfL is all about the feedback – make sure you know how and when you will provide learners with feedback, and do you best to make it personalized.

    Stay tuned next week for the upcoming post on Assessment as Learning (AaL).

  • Assessment ‘As’, ‘For’, and ‘Of’ Learning

    Assessment ‘As’, ‘For’, and ‘Of’ Learning

    Student filling out answers to a test with a pencil.

    Before beginning my journey as an learning experience designer, I was a teacher for many years. Shortly before I left the profession, the big, exciting idea surrounding assessment and evaluation was the triple threat of Assessment as Learning (AaL), Assessment for Learning (AfL), and Assessment of Learning (AoL).

    Most teachers, especially those who had been working with the former formative/summative assessment model didn’t really like the AaL/AfL/AoL model. I, on the other hand, loved the model. Not only does it include the concepts of formative and summative assessment, but it also brings in the idea of metacognition. Research has shown that learners with strong metacognition skills (i.e., they very aware of their own personal methods and preferences of learning) are more successful in school.

    My love for this model is so strong that even as a learning experience designer, I continue to use it to craft my assessments. So, how does it work? Well, it’s based on three different types of assessment, each with it’s own purpose.

    Let’s start with the one that is the most common, and therefore the easiest to understand.

    Assessment of Learning (AoL)

    AoL is a formal assessment of what a learner has learned by the conclusion of the chapter, module, or course, etc. This is often in the form of a test or exam. Behavioural-based courses may also use a marked branching scenario for the AoL. At the primary or secondary school level, it could also be some sort of project or presentation.

    In the teaching world, AoL is also known as evaluation (note this is different than the term evaluation in the instructional design world). It is also known as summative assessment in the former formative/summative assessment model.

    Assessment for Learning (AfL)

    AfL is a less formal (or entirely informal) assessment of what a learner has learned from an individual topic or task. This type of assessment is either unmarked (contains only feedback), or a mark is given simply for the learner’s own knowledge – but it does not count toward a final grade score. This is often in the form of a quiz or game.

    As you might have noticed from the name, the focus of AfL is on the process of learning. Assessment is (or at least should be) strongly tied in with the learning process. The AfL step helps learners identify their strengths and weaknesses, and it ultimately helps the learners focus on what they need to know to succeed.

    AfL is also known as formative assessment in the former formative/summative assessment model. In the instructional design world, this is sometimes known as a knowledge check.

    Assessment as Learning (AaL)

    AaL is where the metacognition piece comes in. Just like the AfL, AaL has a strong focus on the process of learning. These types of assessments are not marked, but instead provide the learner with information about how they are progressing. This allows the learner to make their own personal evaluation about their knowledge and whether they are ready to move on to new content, or if they need to revisit what they have just learned.

    In this case, feedback is crucial – in particular, immediate and personalized feedback. This is because the learner is basing their personal evaluation of their knowledge on the feedback provided to them. Having to wait to finish an entire quiz before finding out if they were correct on question number one is not effective. Similarly, receiving general feedback that doesn’t address their own issues is not effective.

    In addition to immediate and personalized feedback, an important aspect of AaL is the frequency of use. AaL is most effective when it is used regularly. For example, following each topic. This helps the learner build competency and confidence when they are progressing well. Additionally, it helps learners target in quickly on their weaknesses when they are getting stuck.

    A final important aspect of AaL is the ‘how’. Without asking learners how they know what they know, you are missing the valuable metacognition piece. Ideally, AaL will engage learners in the learning process by helping them understand where their thoughts and assumptions come from. When done well, it can also help learners understand how they learn best (and no, I’m not talking about the debunked theory of learning styles, I’m referring to things such as how they favour piecing together information, what type of context building works best for them, how they effectively retrieve information from their long term memory, etc.)

    Conclusion

    In my opinion, compared to other assessment models, such as the formative/summative model, the AaL/AfL/AoL is the complete package. When used properly, both the learner and the teacher, facilitator, or content developer are well aware of how the learner is progressing. They are also aware of what steps need to be taken for each specific learner to be successful because they have already zeroed in on the exact stumbling block of the learner.

    Keep your eyes peeled over the next few weeks as I delve into each of these assessment types more thoroughly, including how to incorporate them in an instructional design environment and how to adapt them to fulfill effective learning experience design.