Online assessment validation tools and VET

Allison Miller, eWorks Accredited ConsultantAllison Miller is a regular contributor to eWorks’ blog. Passionate about providing learners with the knowledge and skills that they need in order to succeed in the world of work, today Allison discusses the role of validating online assessment in ensuring the ongoing quality of the Australian VET sector.

Issuing a vocational education and training (VET) qualification or a statement of attainment for a unit of competency (UoC) or skills set requires the careful assessment of a person’s skills and knowledge. This means the assessment process plays an important role in ensuring the ongoing quality of the Australian VET sector.

To support this, the Standards for Registered Training Organisations (RTOs) 2015 (Standards) (Clauses 1.9-1.11) outlines what a training organisation must do to undertake ongoing, systemic validation of their assessment practices and judgements, also known as assessment validation.

Assessment validation is a two staged process:

  1. Validation of the assessment tools and practices before any assessments are undertaken
  2. Validation of assessment judgements after assessments have taken place and the decision about of a learner’s competency has been provided by the assessor

Stage 1: the validation of the assessment tools and practices

  1. The beginning of a quality assessment process requires the validation of the assessment tools and practices of a unit or cluster of units by ensuring that they meet: Training package requirements – this is done by ensuring that the unit’s assessment tools and practices map to the UoC, and that assessment practices are conducted in accordance with the principles of assessment (Clause 1.8-1).
  2. Industry requirements – this is done through regular industry engagement to ensure that assessment tools and practices align with the current methods, technology, products and performance expected in the workplace.

Online tools and processes to support stage one

Once this has been done, the perfect place to store and manage a unit’s assessment tools and practices is via the learning management system’s (LMS) Grade Book/Centre. The LMS Grade Book/Centre allows students to upload their assessments into one location. This provides an easy and cost-effective way to store and retrieve students’ assessment evidence and grading decisions ready for stage two of the assessment validation process.

Stage 2: the validation of assessment judgements

Following a period of assessment, RTOs need to schedule the validation of their trainer/assessor assessment judgements to ensure they are complying with the rules of evidence (Clause 1.8-2). Clause 1.10 states that each training product must be “validated at least once every five years, with at least 50% of products validated within the first three years of each five year cycle”.

Stage 2 involves:

  • Determining the random sample of assessment evidence and judgements which has been undertaken in the six months prior to the validation process occurring;
  • Having one or more assessors/validators who were not involved in the training or assessment for those units where student assessment evidence and trainer/assessor judgements is being reviewed;
  • Assessors/validators using a checklist to determine whether they ‘agree’ or ‘do not agree’ that:
    • the assessment activity adequately meets the unit of competency requirements; and
    • with the original assessment judgement.
  • Recording reasons why an assessor/validator does not agree with an assessment activity’s adequacy/assessment judgement;
  • Recording and actioning any recommendations for continuous improvement; and
  • Examining the RTO’s whole assessment system to ensure it continues to meet training package and industry requirements (Clause 1.4) ie Stage 1.

Online tools and processes to support stage two

Some great ways to support stage two of the assessment validation process are to:

  • Set up an LMS course which outlines the RTO’s assessment validation polices and processes that validators can refer to before undertaking any assessment validation processes. This LMS course should also have discussion forum so that validators can participate in an online Q&A about assessment validation.
  • Use a webinar room or virtual meeting tool to conduct validation workshops, especially when involving assessors from other RTOs/locations.
  • Create an assessment validation discussion forum in each unit’s LMS course, placed in a hidden/orphaned section so students cannot see it, where assessment validation discussions can be record, such as:
    • how random samples of student assessments were selected
    • which student assessments were selected
    • validation process outcomes
    • recommendations for improvements
    • when/how improvements are made

Posts to the forum are date/time stamped and clearly show the validators involved for future reference and the ongoing continuous improvement process that an RTO has in place.

Assessment validation can be an emotional process

Evaluating someone’s assessment practices and judgements can be a very emotional process, invoking fears of incompetence among trainers and assessors. Being sensitive to the way information is communicated about the quality of people’s assessment judgements is very important.

This process should be viewed as an opportunity to have professional discussions about the continuous improvement of an RTO’s assessment processes, and not used as a finger pointing or brow beating exercise. Recommendations for improvements should be clearly articulated, and all trainers and assessors should be encouraged to be active participants in how these improvements will be implemented.

If you’re not sure whether your assessment practices are adequate, eWorks is happy to help.

Pigs in Space…

Cover of an Atari game called 'Pigs in Space'

Sometimes you have to say something different to get a bit of attention! The latest set of E-learning for Participation and Skills projects for the National Vet E-learning Strategy have just been announced. There are some terrific projects with one having the best project title we have seen in years: ‘Pigs in Space‘ from GOTAFE.

The project’s description can be found on the E-learning for Participation and Skills wiki.

Photo: Pigs in Space, by axeldeviaje on flickr.

VELG 2012

The eWorks conference stand

eWorks’ booth at VELG 2011

Joanne Norbury and I are looking forward to the upcoming National VET E-learning Conference (VELG), 21-22 September. The conference has attracted nearly 600 attendees last year which must have been a vocational conference record.

We will be hosting a booth to have conversations about the TrainingVC and our other products. We are also giving away an hour of free training on ‘Getting started in e-learning’ and ‘Adding Web 2.0 to your e-learning’.

I will be presenting ‘From Trainer Development to Client Engagement: Twitter for Educators and Case Studies from RTOs using Facebook.‘ on the Thursday afternoon. This session looks at what you need to get started and running with Twitter, understanding its contexts and grammar, how to leverage this to enhance your Personal Learning Network and what the social network’s role is in terms of training and education. The presentation will also address some of the issues with introducing Facebook into your organisation.

The hashtag for the conference is #2012nvc.

Take the time to come and say hi to us! We’ll be in booth no39.