Quality e-Assessment Workflows

Look at literature on e-Assessment workflows
Policy impolications
Issues around workflows
Demonstrations with Blackboard and Moodle with ReMarks

Processes must be simple and easy to use as people look for simplicity or give up on it!

• Design of assessment must be planned as integrated part of course design (Kendle & Northcote 2000)
• Assessment tasks influence the direction and quality of student learning (Maclellan 2004)
• Students need to be given more responsibity for assessment process

Formative assessment - marks with written comments not much better than just marks. Marks can get in the way of learning Butler 1988
Multiple chances to be re-assessed - much more likely to pay attention to feedback.
Large tasks - break into smaller stages with feedback at each stage and grade for work in total
Develop a grading rubric - very useful when multiple markers. Make students aware of this and what it means.
Calibration moderation - multiple staff mark same work using same rubric to test rubric

Royce Sadler (1989)

• Help students to recognise and clearly understand the desired goal to appreciate what high quality work looks like.
• Help them develop evaluative skills
• Explaining ways to close the gap between outcomes and expectations

Students more interested in how they have performed more than comparison to other students!

Advantages of e-submission

• copy for student appeals
• management for large courses
• saving on printing costs
• plagiarism detection
• security
• confirmation for students
• back up at organisation level

Challenges

• toggling with data & feedback sheet (not with ReMarks)
• Laborious annotation of work
• Better planning required for managing paperless submissions

Issues

• Integration with LMS - issue for who has handed in, multiple markers, keeping track of who has submitted
• Data-matching software - do you need to be looking for plagiarism? Does it slow things down?
• Group submissions - pragmatism and group work, how to fairly assign marks and motivate participation
• Flow onto to policies, like late submission, data storage etc
• Return of marks and assessments - easier?

Talk on Quality Assurance & moderation process

Recent review placed emphasis on quality and standards - TESQA is now national quality regulator. Accreditation linked to standards.
Link between learning objectives, assessment methods and grading of attainment - need to understand relationship to understand and monitor standards.
Academic standards depend upon individual academic expert judgement. Based upon experience of students, their education, etc.
Nulty D D (2011) - paper on consensus moderation

Consensus moderation - about quality assessment and outcomes 'look like'
Consistency in time, place & marker
Process like peer review but consensus is the aim - like herding cats
Other ways of reaching consensus - experience of students, across multiple institutions, annotated exemplars, etc. Most academics do some form of this -having assignments vetted by colleagues, consistency with international textbooks, etc

Framework for consensus moderation

1. Course level assessment planning - ensure appropriate assessment items and patterns
2. Marking student work - ensure appropriate and consistent standards are applied to marking individual pieces
3. grading student work
4. standard across courses
5. standards over time

Griffith Uni has a website now on assessment - Assessment Matters!

TESQA - show us what you do and the difference it has made, on assessment, consensus moderation, etc.

Getting experts together to justify assessment as useful exercise not only for them but also new lecturers, tutors, etc...

Double marking from different institutions as a tool for consensus
Worth being able to look at mark distributions for multiple years - to see consistency of marking
Having assignments and spread of assessments (high, middle, low)
Map down to assessment from overall course goals