How to Test Whether Saudi eLearning Moves Employees from Viewing to Practice

Imagine your learning platform report looks healthy. Completion rates are up, logged hours are rising, and every branch finished the new eLearning module on schedule. Then a supervisor in Riyadh mentions that the team still calls for help before processing a customer return, and a branch manager in Jeddah notices the same errors appearing in customer files week after week.

Nothing in the report is wrong. It simply answers a different question. Hours and completions tell you that employees opened and viewed the eLearning content. They do not tell you whether the eLearning moved anyone from watching a demonstration to performing the task correctly on their own.

This article sets out a practical way for HR and digital learning teams in Saudi organizations to test that shift. The method is deliberately small: take one work task, build two versions of the same eLearning activity, and compare the quality of employees’ attempts and the feedback they receive, rather than total time on the platform.

Why Platform Hours Cannot Show Whether eLearning Works

Many learning management system (LMS) reports are built around access: who enrolled, who opened a module, how long each session lasted, and whether the final screen was reached. These records are useful for administration and compliance reporting, but they describe exposure to content, not capability.

Established learning evaluation models make the same distinction. The Kirkpatrick Model separates learning from behavior and defines Level 3 as the degree to which people perform the critical behaviors in their work environment. Will Thalheimer’s Learning-Transfer Evaluation Model (LTEM) has eight tiers, differentiates between knowledge, decision-making, and task performance, and states that its lower tiers are inadequate for validating learning success.

For a corporate eLearning team, the implication is direct. If the only evidence you collect is time spent and modules completed, you cannot tell an eLearning module that changes performance from one that employees leave running in the background while they do other work.

What Moving from Viewing to Practice Actually Looks Like

Viewing means the employee receives information: a video, a slide sequence, or a policy summary followed by a few recall questions. Practice means the employee has to do something with that information: make a decision in a realistic situation, complete steps in the right order, explain why an option is correct, or correct a mistake after feedback.

Practice-based eLearning usually has three features:

• A realistic trigger: a customer request, a form, a system screen, or a situation the employee will actually meet at work.

• A required action: the employee chooses, sequences, completes, or explains, rather than simply clicking ‘Next’.

• Specific feedback and another attempt: the employee sees what was right or wrong in their own response and can try again.

These features separate interactive learning that builds skill from content that is merely clickable. Feedback deserves particular attention. In a review published in Review of Educational Research, Hattie and Timperley describe feedback as one of the most powerful influences on learning and achievement, while noting that its impact can be positive or negative depending on its type and how it is given. A generic ‘Incorrect, try again’ message is not the same as feedback that points to the step the employee missed.

Why This Matters for eLearning in Saudi Arabia

Saudi organizations have a national reference point for this discussion. The National eLearning Center’s eLearning standards address government, private, and non-profit institutions that provide, operate, or support eLearning and training, including training centers and academies, eLearning platform providers, and program operators. The standards include engagement and interaction, feedback, a full assessment domain, and a continuous evaluation and improvement domain that covers measuring performance.

Even if your organization is not directly within that scope, the structure of the standards sends a useful signal: eLearning quality is a matter of design, interaction, and evaluation, not the volume of content published. A two-version test gives an internal eLearning team a structured way to show which design choices actually help employees perform, and to make that case to leadership with evidence rather than activity counts.

The Two-Version Test: One Task, Two eLearning Designs

The test compares two versions of the same eLearning activity built around one task. Everything stays the same except the element you want to evaluate: whether employees practice and receive feedback, or mainly watch and recall.

Step 1: Choose One Task That Matters

Pick a task that is frequent, observable, and costly to get wrong. Good candidates include processing a return, completing a service request, checking a document before approval, or responding to a common customer objection. Avoid tasks that are rare, highly specialized, or impossible to observe on the job. If you are unsure where to start, begin with a recurring performance problem your supervisors already report, rather than with the eLearning content you already have.

Then confirm that the problem is really a skill gap. If employees make mistakes because the system is slow, the form is unclear or they lack the authority to act, no eLearning design will fix it.

Step 2: Build Version A and Version B

Version A is the viewing design. It might be a short video or slide sequence that explains the task, followed by a few recall questions. Version B is the practice design, built on scenario-based practice. It covers the same content and objective, but employees work through realistic situations, make decisions, and receive feedback on each attempt.

Keep everything else equal: the same learning objective, a similar length, the same language, the same device access, and the same deadline. If version B is also shorter, better produced, and launched by a more enthusiastic manager, you will not know which change made the difference.

Step 3: Define Attempt Quality Before Launch

Decide in advance what a good attempt looks like in each eLearning scenario. A simple rubric with three or four levels is enough. For each scenario, describe what correct, partly correct, and incorrect mean in observable terms, for example:

• The right decision for the situation

• Steps completed in the required order

• Key information checked or recorded

• A brief, accurate reason for the choice

Use the same rubric for both versions and for any later observation on the job. This is what allows a fair comparison.

Step 4: Decide Which Feedback You Will Measure

In version B, the eLearning feedback should respond to the employee’s actual answer and link back to the rubric: what was correct, what was missed, and what to check next time. Record whether employees try again after feedback and whether their second attempt improves. That pattern of attempt, feedback, and better attempt is the clearest sign inside the platform that viewing has turned into practice.

Do not assume feedback is helpful simply because it exists. Review messages that are vague, that comment on the person instead of the task, or that reveal the answer without explaining it.

Step 5: Assign Comparable Groups and Run the Test

Split employees who do the same job into two groups that are as similar as possible in role, experience, and branch or shift. Random assignment is best; where it is not practical, alternate by team or location and record the differences. Run both eLearning versions during the same period so that seasonal workload or a policy change affects both groups equally.

A small eLearning pilot can teach you a great deal. You are not running an academic study; you are making a better design decision for one task.

Step 6: Collect Evidence Beyond the Platform

The strongest evidence appears after the eLearning activity ends. Two to four weeks later, ask supervisors to observe a sample of real or simulated attempts using the same rubric, or to review a sample of completed work. Where possible, supervisors should not know which version each employee took.

Inside the eLearning platform, check what your system can capture beyond completion. The xAPI specification published by ADL is designed for communication about learner activity and experiences between technologies, so platforms that support it can record individual decisions and attempts, not only a completion status. If your current LMS reports only completions and scores, keep a simple record of rubric results instead. For more on turning LMS dashboard data into manager action, see our related guide.

A Hypothetical Example: Handling Returns in a Saudi Retail Chain

The following example is hypothetical and is included only to show how the test works.

A retail chain with branches in Riyadh, Jeddah and Dammam wants store staff to handle returns correctly. Its current eLearning module is a nine-minute video explaining the returns policy, followed by five multiple-choice questions.

Version A keeps that design. Version B uses the same policy content but turns the eLearning into four short situations: a customer with a receipt, a customer without one, a damaged item, and a request outside the return period. In each case, the employee chooses what to do, sees feedback that refers to the specific policy condition, and can try again.

The learning team defines attempt quality as the correct decision, the correct system step, and a clear explanation to the customer. Three weeks after launch, area supervisors review a sample of real return transactions from both groups using the same rubric, without knowing which version each employee completed.

The team is not looking for a single score. It is looking for a pattern: whether version B employees improve between their first and second eLearning attempts, whether the same errors still appear in real transactions, and whether supervisors receive fewer calls for help on routine cases.

How to Read Your eLearning Test Results

Interpret the comparison carefully, and only for the task you tested:

• Version B performs better in practice attempts and on the job: extend the practice design to similar tasks and review which scenarios caused the most errors.

• Version B performs better inside the platform but not on the job: check whether the scenarios reflect real conditions, whether supervisors reinforce the new steps, and whether non-training barriers remain.

• No meaningful difference: the task may be simple enough for a viewing-only eLearning design to work, or the feedback in version B may be too generic to help.

• Version A performs better: check whether version B was too long, unclear, or difficult to use on the devices employees actually have.

Report results to managers in plain terms: what you tested, what changed, what you will do next, and what you still do not know. Avoid presenting a small pilot as proof that one eLearning format is always superior.

Practical Considerations for Saudi Workplaces

Language and Terminology

Write eLearning scenarios in the language employees actually use at work. Where instructions are given in Arabic, but system labels, product names, or technical terms appear in English, show both exactly as employees see them on the job. Test the wording with a few employees from the target role before launch; a scenario that confuses them measures language, not skill.

Personal Data in Attempt Records

Attempt records, rubric scores and supervisor observations are linked to named employees. The Implementing Regulation of Saudi Arabia’s Personal Data Protection Law requires controllers to collect only the minimum personal data necessary to achieve the purpose of processing. In practice, decide which data the test genuinely needs, tell employees why it is being collected, and use the results to improve eLearning design rather than to make decisions about individuals, unless your policies and legal advisers confirm otherwise.

Shifts and Access

For teams that work in shifts or spend most of their day with customers, keep each eLearning practice scenario short enough to finish in a few minutes. Check that it works on the devices and connections employees actually use, not only on office computers.

Common Mistakes to Avoid

• Changing several elements at once, so you cannot tell what made the difference

• Measuring only quiz scores and treating recall as performance

• Comparing groups with different roles, experience, or workloads

• Declaring eLearning success because completion rates rose

• Testing a task that employees rarely perform

• Treating every feedback message as useful without reviewing its quality

Summary

Platform hours and completion rates are a starting point, not evidence that eLearning has changed how people work. To find out whether your eLearning moves employees from viewing to practice, test one important task in two versions, define attempt quality before launch, look closely at the feedback, and follow up with evidence from the job.

The test is small enough to run within a quarter and specific enough to guide real design decisions for employee learning. Once you know which approach works for one task, you can apply the lesson to the next one and build role-based eLearning on evidence rather than assumption.

If you want Skillup MENA to help you, click here.