A distinctive feature of the Question Pool is its ability to 'shuffle' a pre-determined set of questions. This allows the Question Pool to offer a different question set to each learner so that behaviours like answer sharing can be discouraged.
It's important to know how a Question Pool shuffles and under what conditions in order to fully understand how to use them best.
Contents
1 How do Question Pools shuffle questions?
2 When does a Question Pool shuffle?
3 How a learner's score is calculated between shuffles
How do Question Pools shuffle questions?
Question Pool shuffling works in a distinctive way.
Rather than shuffling questions and presenting them in a completely random order like shuffling music in a playlist, its useful to think of Question Pool shuffling in terms of inclusion and omission.
When a Question Pool shuffles, a selection of pages will be chosen to be included in that instance of the Question Pool. The remaining pages will be omitted and not presented to the learner. The number of included pages is determined by the Question Pool's percentage.
For example, a 30% Question Pool will include 30% of the pages in the pool and omit 70%.
This selection of pages is randomly chosen each time the Question Pool shuffles, however the order of the pages will not change.
A 50% Question Pool of 10 questions may present the 5 included question pages like this:
Q1 Q3 Q5 Q8 Q9
But it will never present them out of order like this:
Q8 Q3 Q1 Q9 Q7
When does a Question Pool shuffle?
A Question Pool will shuffle its pages when either of the below two conditions are met. These conditions can occur more than once and independently of each other.
Condition one: accessing the course
The first is when the learner accesses the course. This includes:
- when the learner arrives to the course for the first time
- when the learner returns after leaving the course at any point.
This condition will always be active and is not able to be modified.
Condition two: completing the course
The second condition is when the learner completes the course. This could occur either:
- when the learner reaches the final page of the course
- when the learner reaches a page which has Send final score on this page enabled
This behaviour can be enabled or disabled using the Automatically shuffle question pools as soon as the learner completes the course control found in a Project's Configure page:
Here is a diagram showing how this logic works:
How a learner's score is calculated between shuffles
Since a Question Pool will always shuffle when a course is accessed regardless of what point the learner is at, it's possible for a learner's running score to be changed if they return to a course where they have already submitted answers to questions.
This is because as the Question Pool selects a new set of pages to be included and omitted, it is likely that some questions that have been answered previously are omitted. When a page is omitted, the score that page holds is taken out of the learner's running score.
This can result in the learner's score dropping as it adjusts to the new set of pages that have been included and omitted.
Example
I have a 50% Question Pool with 10 questions. This means 5 questions will be selected for the learner to answer. In this instance, it is:
Q1 Q2 Q3 Q4 and Q5.
My learner has answered all questions correctly. They receive 20% for each question, awarding them with a total score of 100%.
My learner leaves the course and then returns at some point in the future. The Question Pool is reshuffled when they return. The new Question Pool has included the following questions:
Q4 Q5 Q6 Q7 Q8.
There are two common questions between old and new instances of the Question Pool: Q4 and Q5. The rest are new questions for which no score has been achieved as they were omitted in the first instance of the Question Pool.
When the Question Pool reshuffles, the learner's score will drop from 100% to 40% to account for the new set of pages. Q4 and Q5 still contribute 20% each to the total score, but the other pages have no score to contribute yet.
Here is a diagram to illustrate how this works:
For this reason, we strongly encourage you to design your Question Pools to be taken in one sitting and discourage learners from returning after they have completed a course which uses them.