Greener Journal of Education and Training Studies
Research Article (DOI: http://doi.org/10.15580/GJETS.2015.1.0523014243)
Traditional or Conveyor belt marking: Exploring the way forward at Great Zimbabwe University, Zimbabwe
Lecturer, Department of Curriculum Studies, Great Zimbabwe University, Masvingo, Zimbabwe.
Email: jrisiro @gmail. com
The purpose of the study was to examine the strengths and weaknesses of the current traditional marking and explore possibilities and challenges of introducing conveyor belt marking at Great Zimbabwe University. Individual interviews were carried out for students enrolled in the Bachelor of Education Programmes. Lecturers who have been involved in conveyor belt marking were purposefully sampled to answer questionnaires. There are possibilities of introducing belt marking at the Institution. Students were more supportive of using belt marking unlike the Lecturers who pointed out too many challenges before implementing it. The challenges experienced in belt marking were organisational and management of the marking process. The solutions to the challenges involved identifying subject specialists across the University departments, staff recruitment and in service training for Staff. Departmental Chairpersons and Subject Coordinators across Faculties need to liaise with each other for the smooth running of the marking.
Keywords: Traditional marking; Conveyor belt marking; Marking reliability; Possibilities; Challenges.
Assessment is important in evaluating modules, methods of teaching and finally grading of the learners. Palomba and Banta  defined assessment as the systematic collection, review and use information about educational programmes undertaken for the purpose of improving. It is vital that assessment is carried out in all fairness and integrity it deserves. More than often candidates cry foul especially when they did not perform well at the end of the semester or programme. Marking is one of the processes that ensure that the assessment is reliable and valid. It is against this background that the research seeks to explore the strengths and weaknesses of the traditional marking system currently practised by Great Zimbabwe University with the possibility of introducing conveyor belt marking. The research explores possibilities and challenges of introducing belt marking at Great Zimbabwe University.
A mark is a score awarded to a candidate by an Examiner based on his /her judgement [Ofqual, 2011]. In traditional marking, the system involves one Marker marking the whole script or centre. Each Marker is assigned scripts to mark by the Subject Manager or Marking Supervisors [Bukenya, 2006]. The process starts with the Chief Examiner and Senior examiners developing question papers and marking guides. The senior examiners train their markers how to apply the marking scheme [Ofqual, 2011; Ofqual, 2013]. Consistency in the application of marking scheme ensures reliability in the marking process [Chamberlain and Taylor, 2010]. Markers are placed into teams by the Chief examiner under the supervision of the Team Leader. The Team leader selects some scripts for moderation to check adherence to the marking scheme. A Marker exceeding agreed deviation depending on the subject could be asked to do a remarking. After marking transcript checkers, check for errors on the marked scripts as well as transcription of marks onto the mark sheets. After the Marker has completed marking, the marks are entered on the mark sheets and went through various boards for approval, grading, and publication of results.
New developments in marking include automated marking which uses optical mark recognition software to mark multiple choice exams. There is also on-screen marking whereby candidates scripts are scanned onto a computer for marking by Examiners [Ofqual,2013].On-screen marking has the advantage in that Examiners can be monitored and corrected by Senior examiners immediately. Senior examiners mark certain scripts on each examiners batch. As the Examiner marks the script the mark is compared with the already marked script by the Senior Examiner, thereby checking the accuracy of the Marker [Ofqual, 2013].
Conveyor belt marking involve organising Markers into groups in which each Marker is assigned a question(s) to mark by the Team Leader [Bukenya, 2006]. This type of marking is also referred to as item level marking [Ofqual, 2013]. Item level marking reduces bias (halo effect) and remove the influence of one Marker on the script [Pinot de Moira, 2011; Spear, 1996]. The process of Belt marking starts with allocation of marking scripts to the Belt Marking Supervisors [BMS]. The Supervisor and his/her team open the scripts and start the process of tallying with his/her team. This involves counting and recording the number of Students who answered each of the questions for each and every centre that wrote the examination. Each Marker is assigned a question(s) to mark. A script control movement form is filled in to account for the movement of scripts. The Supervisor can adjust the allocation of the question if there is a need. Difficult questions can be allocated to seasoned markers while the less experienced markers are assigned easy questions [Suto et al., 2011].
In Rwanda the Government has introduced belt marking where each belt marking team consists of five (5) members. This has eliminated remarking of scripts. The National Examination Council of Rwanda, indicated that in the past traditional marking in which each script was marked by one Marker led to hasty and sloppy marking as the Markers were paid according to the number of scripts they would have marked [Rwanda Focus, 2009].The Kenyan National Examination Council introduced the conveyor belt and to ensure marking reliability the Markers were trained and sat for the examination they were going to mark in order to judge their content level [Manyumba and Mutwiri, 2009].Conveyor belt marking in Tanzania, among other reasons was adopted to reduce the time for marking, queries and biases by Markers [East African Community Meeting of Secondary Education Examination Report, 2010].
Wheadon and Pinot de Moira  defined marking reliability as measure of agreement between the mark awarded to a piece of work and the mark that should have been awarded. Pinot de Moira  suggested involving many markers per script as one way of achieving marking reliability while Suto et al.  were of the opinion that by giving complex questions to be marked by seasoned examiners ensures reliability in the marking process. Research by Tisi et al.  and Pinot de Moira, 2011] have revealed several ways in which Item level marking lead into attainment of marking reliability. Factors such as test items, marking scheme, Markers subject knowledge and teaching experience were found out to be influential in determining marking reliability[Meadows and Billington,2007;Chamberlain and Taylor, 2010].Personality issues like gender bias and poor handwriting were regarded important in deciding a mark to award to a piece of work [Baird, 1989].The discussion has shown that one important factor in influencing the awarding of marks is human related such as level of education and teaching experience. It is the thrust of this paper to explore ways in which the human element in marking can be reduced by exploring the possibilities of introducing belt marking, which involve more than one marker per script, at Great Zimbabwe University.
There are challenges in the traditional marking of examination in the University set up, where the Lecturer is familiar with the Candidates. The Students often complain of foul play, favouritism and unfairness on the part of the Lecturers especially where the Candidate has failed or obtained lower grades. The Paper examines the two forms of marking in order to find out the one most appropriate for use by Great Zimbabwe University and other related Institutions.
The research was guided by the following questions:
1. What are the strengths and weaknesses of the current traditional marking at the Institution?
2. To what extent can conveyor belt marking be implemented at Great Zimbabwe University?
3. What strategies can be adopted by the University to improve the effectiveness of belt marking?
Purpose of the Study
The purpose of the study is to explore processes, possibilities and challenges of introducing belt marking at Great Zimbabwe University. The Institution is currently using traditional marking in its assessment of Students. The paper examines the strengths and weaknesses of using traditional and belt marking. The findings of the study can be adopted, modified or improved for use by Great Zimbabwe University and other interested parties in the field of education.
MATERIALS AND METHODS
A case study approach using qualitative methods was used in the research study. This involved the use of interviews and open ended questionnaires to collect information from Lecturers and Students on belt and traditional marking. The case study was suitable to the study as noted by McMillan and Schumacher (1993), that a case study is useful when the scope of the study is broad, including strengths and weaknesses. Various aspects of both traditional and conveyor belt marking were studied in order to weigh the strengths and weaknesses of each form of marking.
Purposeful sampling was done to select Lecturers in the Faculty of Education who have been involved in traditional and belt marking in public examinations as these had an in depth knowledge and experience. Most Lecturers in the Faculty of Education have previous experience in marking at various levels in the education system. Open ended questionnaires were distributed to Lecturers and Students in the Languages, Home Economics, Environmental Science, Music and Mathematics. This allowed Lecturers and Students to answer the questions freely at their own time, pace and without interference from the Researcher. Respondents managed to express their views and feelings on the introduction of belt marking at Great Zimbabwe University. Group interviews for Students in the selected Subject areas were done in order to get a wide range of views from the Students in the shortest time possible and at a minimal cost. The group interviews from the selected subject areas were limited to 6 – 12 students, selected randomly from the class list. Group interviews allowed the participants to openly express their feelings, perceptions and beliefs on various forms of marking as supported by Gall et al. (1986).Group interviews were necessary as they verified data collected from individual interviews and from questionnaires.
RESULTS AND DISCUSSION
1. What are the strengths and weaknesses of the current traditional marking at the Institution?
The Respondents were asked to give their views on the current traditional method used by Great Zimbabwe University in the marking of assignments and examinations. Twenty (20) Lecturers and twenty (20) students were given open ended questionnaires and their responses analysed in the tables below:
The research established that all the Lecturing Staff were in favour of using traditional method of marking as it enabled the Marker to control and manage his or her time of marking independently. The Lecturers were of the view that when they are tired of marking they can engage in other productive business including research and start marking after getting refreshed. For the Lecturers it was monotonous to mark the whole day confined in a room. Lecturers had the feeling that traditional marking allows them to be more accountable for the marks and scripts of the candidates. In case of remarking of scripts one Lecturer can do so which is impossible with belt marking. The Marker can mark all the questions. Traditional marking can therefore make use of few Markers in a section or department. The method becomes cheaper in terms of human resources. The management of traditional marking was seen as less taxing as it involves less coordination as is the case with belt marking. The Lecturer marks his or her own scripts, record them and process them for presentation and grading. Ofqual  observed that uniformity or fairness in marking can be achieved in whole script marking as one marker can be generous in one question and strict on the other.
The students viewed favouritism and bias by Lecturers as the greatest weakness of traditional marking. The students had the feeling that at University level the Lecturers are familiar with the Students and issues such as gender and social background of the Students can influence the marker’s decision on a script. These findings are anchored by studies carried out by Mashoko et al ; Baird . The mastering and application of the marking scheme was regarded as the major disadvantage in using traditional methods of marking by Lecturers. The research agrees with findings by Tisi et al, , Meadows and Billing,  on challenges facing a marking system. The Authors observed that marking credibility is determined much by the nature of the marking scheme and its application by the marker.
2. To what extent can conveyor belt marking be implemented at Great Zimbabwe University
The Students (20) and Lecturers (20) were asked on their views on why it would be recommended to implement belt marking at the Institution. The findings were tabulated in table 3.
It was observed that belt marking allows the Examiner to specialise in one question or questions assigned to him/her, thereby increasing the pace of marking. There is deep understanding of the marking scheme as the Marker concentrates on few questions thereby improving on reliability of marking. This contributes to efficiency and fairness in marking. The Students felt that belt marking minimise malpractice resulting in high degree of fairness in the marking process. The Students were in support of belt marking as they felt it would protect them from those Lecturers who may be biased and practice favouritism. Belt marking was regarded to have internal moderation imbedded in it by the Lecturers since one script is marked by more than one Lecturer. The spirit of team work is enhanced as the Markers work in a group. The Lecturers would tend to understand each other as well as discovering one’s strengths and weaknesses. Coordination between department and across Faculties is promoted as members from different departments work together. The slow Markers are cushioned by fast Markers thereby allowing presentation of marks to various boards and publication to be done in time. There is also maximum use of expertise in the University as Staff is sourced from other faculties and Departments to form marking groups. In a study carried out on Conveyor Belt Marking in Chikomba district on Ordinary level markers, Ngara and Ngara  noted that CBS had the strengths of increasing marking reliability , team work and efficiency among other advantages. Research carried by Spear , Pinot de Moira , Wheadon and Pinot de Moira  concurred that conveyor belt marking improves reliability of marking and reduces examiner’s bias.
Challenges of introducing belt marking at Great Zimbabwe University
There is shortage of Lecturers specialising in certain areas such that creating a belt becomes difficult. Some sections consisted of two or three Lecturers which makes belt marking impossible as the Lecturers would end up marking too many questions which defeats the purpose of belt marking. Fast Markers may end up marking many questions in an effort of assisting the slow Markers. Bukenya  noted that conveyor belt marking does not cater for individual differences of examiners. The diverse character of the Markers therefore poses challenges to belt marking. Belt marking confines Markers at one marking venue. Some Lecturers indicated that they prefer to mark either in their office or in the comfort of their homes. Some Markers are very fast while some are slow. The fast Markers would feel delayed and disadvantaged, thereby creating friction and misunderstanding within a marking group. Ofqual  observed that conveyor belt marking result in boredom, inattention and complacency. In a research done by Ngara and Ngara  conveyor belt marking can cause conflicts among team members resulting from slow markers who delay fast markers. Bukenya  argued that conveyor belt marking was time consuming and tiresome resulting in Markers breaking down more than in traditional marking system.
Lecturers expressed displeasure on the time it would take in tallying the questions answered by the Candidates to allow allocation of questions to respective Markers before marking begins. The process of tallying questions was regarded as time consuming thereby bringing frustration and fatigue in Lecturers before they start marking.
The other challenge mentioned was the unbalanced number of questions answered by the Students. Some questions are very popular which can be answered by many Candidates. This may result in some Lecturers marking many questions while others would be marking few questions. Markers mark at different pace and accuracy. It was felt that Markers allocated questions that are difficult to mark take that too long to finish yet those given easy questions to mark take less time. This causes resentment within the marking groups. It was observed that other Markers get involved in other businesses during marking thereby delaying other colleagues in the belt marking group. The Lectures expressed the fact that if one member is affected by illness, family commitment or economic problems, the whole group is delayed. It was also noted that belt marking is monotonous, as the Examiner marks the same question(s) throughout the session. Script control and monitoring script movement was a task highlighted by the majority of the Lecturers. There is the danger that some scripts may disappear with poor monitoring. Accountability of scripts may be a challenge if script movement control is not put in place. It was found that some pages within a script may be left unmarked in situations where the candidate had not put his/or her papers in order. Markers may not be bothered to check through all the pages other than the question he/she is tasked to mark.
3 What strategies can be adopted by the University to improve the effectiveness of belt marking?
The Lecturers were asked on the best strategies they had used previously in belt marking of National Examinations which can be adopted at Great Zimbabwe University to improve belt marking process. The strategies suggested can be summarised as follows:
1).Use of ‘Gunners’ to mark challenging questions: A Gunner was regarded as a soldier who can shoot with accuracy and precision. Markers who are fast and accurate could be used to mark more challenging questions in a group. The Team leader therefore has to identify Markers strengths and weaknesses and see where the Marker can perform efficiently. These Markers can also be allocated questions which were answered by many Candidates. Research carried by Pinot de Moira ; Meadows and Billington ; Suto et al  has shown that mark reliability in item level marking can be achieved by allocating complex questions to more experienced markers and simple questions to junior markers.
2).Roving Markers: These were regarded as fast and accurate Markers who can mark their allocated question(s) and go on to assist in marking questions assigned to them by the Belt Marking Supervisor. This group of Markers are important in that they can as well assist slow Markers. They are versatile and knowledgeable in a number of topics.
3).Volunteering Markers: The Belt Supervisor can allow Markers to select questions to mark which they are comfortable with and then decide what to do with the remaining questions if any. This is important in that Markers would mark those questions they have knowledge, mastery of content and the marking guide.
4).Allocating Demanding questions to each marker: The Belt Marking Supervisor can identify demanding and most challenging questions first and allocate each of the demanding question to each of the Markers. The remainder are then distributed accordingly as determined by number of questions each Marker is going to mark.
5).Sectional Marking: This involves placing the Markers into sections such that Markers would mark questions in a section they are comfortable in terms of content mastery and understanding of the marking guide.
The respondents suggested staff rationalisation in which Lecturers from different faculties lecturing similar modules would come together and form marking groups. The Lecturers would identify modules they would mark efficiently. Specialists develop detailed marking schemes which can be used by all the Lecturers marking the module. Staff development on belt marking was regarded as a way that would allow the full implementation of belt marking at the institution. The expertise of Lecturers involved in belt marking of public examinations could be consulted for effective implementation of belt marking.
The research found out that although Lecturers were aware of some of the advantages of conveyor belt marking over traditional marking they were not willing to adopt and implement it at the University. The Lecturers preferred to continue with traditional marking. The Lecturers were in favour of traditional marking as they felt that it gave them autonomy of when and where to mark the scripts. They did not want to be confined in the marking venues where they would mark as a group. The Lecturers preferred to do other productive work like research when they felt that they are tired of marking. They expressed that belt marking would not give them the freedom to attend to other business during the belt marking session. The Lecturers wanted to manage, control and organise their marking rather than being controlled by the needs of the marking group members. Lecturers were also not comfortable with Lecturers from other Departments marking scripts for students they did not teach. In contrast the Students’ were of the opinion that the University should replace traditional marking with conveyor belt marking. The Students were of the view that belt marking would protect them from those Lecturers who are biased and unfair in their marking.
There are several challenges indicated by the respondents that can threaten the introduction and implementation of belt marking at Great Zimbabwe University. Among the challenges are few lecturers within the same area of specialisation such that forming a belt marking group is difficult. This would mean one Lecturer marking too many questions per script thereby defeating the concept of belt marking. The multi-campus model by the University meant that the Lecturers had to come from different points to converge at the marking venue thus adding logistical problems. The idea of getting confined at one venue during the marking period posed some problems as other Lecturers would want to do other business when they feel they are tired. Belt marking was regarded as cumbersome as it involved tallying of scripts controlling the movement of the scripts. Despite the challenges it was observed that with Institutional support in form of training workshops on belt marking, staff rationalisation and recruitment in areas of shortages, belt marking can be introduced in the institution.
Although the Lecturing Staff did not recommend conveyor belt marking at Great Zimbabwe University, they suggested ways in which belt marking could be introduced at the Institution.
Lecturers from different departments can work together to create a belt for example Geography Lecturers from Curriculum Studies, Teacher Development and Natural Sciences can come together to form viable belt marking teams. This calls for staff rationalisation in the University.
The Lecturing Staff recommended recruitment of more Lecturers in the different specialist areas to allow formation of belt marking groups easy.
Lecturers could work together to develop a detailed marking guide to allow those with less knowledge in the area to mark the questions allocated to them.
Workshops and seminars on belt marking could be held to allow members to appreciate and adopt the new marking technique. Local Lecturers who have experience in belt marking were suggested to facilitate such seminars.
Lecturers who teach similar modules could come together to align their modules such that the content for a certain module taught across Faculties be the same for example a Climatology module, if taught in more than one department should have similar content and be taught by anyone from these departments. This would allow setting similar examination to be marked by all the members.
The experienced and those with mastery in the belt marking group can be allocated more difficult questions where as those who are not well versed in the area can be allocated easy questions to mark.
Baird J . what is a name? Experiments with blind marking in a level Examinations. Educational Research, 40:191-202.
Bukenya M . Comparing the Reliability of the Conveyor Belt Marking System with the Traditional Marking System: A paper Presented at the 32nd IAEA Annual Conference,Singapore,22nd -26 may,2006.
Chamberlain S, Taylor R . Online or Face- to- Face? An Experimental Study of Examiner Training. British Journal of Educational technology, vol.412, issue 4, pp.665-675, July 2011. East African Community Meeting of Secondary Education Examination Report, 2010.
Gall MD, Borg, Gall. Educational research: An Introduction. New York: Longman Publishers.
Manyumba D, Mutwiri JG . The Kenyan National Examination Council: Challenges Associated with Implementation of Control Mechanisms in Public Examinations and How the Kenya National Examinations Council [KNEC] has handled some of these challenges: A Paper presented at the 27th Annual Conference of the Association for Educational Assessment in Africa [AEAA], Yaonde, Camerron, and August 24 – 28, 2009.
Mashoko,D,. Mateveke,P,. Kufakunesu,M,. Mashoko,E..Public Examinations: A Fair Deal or a Burden?The Zimbabwean Experience: International Journal of English and Education, Vol.2, Issue 3,July, 2013.
McMillan JH, Schumacher S . Research in Education: A Conceptual Introduction, 3rd Ed. New York: Harper Collins.
Meadows, Billington L . NAA Enhancing the Quality of Marking Project: Final Report for Research on Marker Selection. London, QCA. Available at: http://archive.teachfind.com/qcda/orderline.qcda.gov.uk/gempdf/184962531X/QCDA104980 marker selection pdf (accessed 10 may2014).
Ngara, R, Ngara, R. . Conveyor Belt Marking: Opinions of ZIMSEC Markers in Chikomba District. Hope Journal Research (House of Pakistani Educationists), Vol,1, Issue 2, June, 2013.
Office of Qualifications and Examinations Regulations (Ofqual) Interim Report .Review of Quality Marking in Exams in A levels, GCSEs and Other Academic Qualifications.
Office of Qualifications and Examinations Regulations (Ofqual) Interim Report . Quality of Marking: Review of Literature on Item Level Marking.
Ofqual . Inquiry into Examination Errors Summer 2011 Final report. Available at: http://ofqual.gov.uk/files/2011-12-20-inquiry-into-examinationerrors-summer-2011-finalreport.pdf. [Accessed 3 May, 2014].
Palomba CA, Banta TW . Assessment Essentials: Planning, Implementing and Improving Assessment in higher Education. San Francisco: Jossey-Bass. 4pp.
Pinot de Moira . Why Item Mark? The Advantages and Disadvantages of E-Marking. Machester, AQA, Centre for Education Research and Policy. Accessed at: http://cerp.aga.org.uk/research-library/why-item-mark-advantages-anddisadvantages-emarking (retrieved 10 May, 2014).
Rwanda Focus, Thursday, 28 March, 2009.
Spear M .The Influence of Halo Effects upon Teachers’ Assessment of Written Work. Research in Education, p85.cited in Meadows M and Billington L . A Review of the Literature on marking reliability. Manchester, AQA. Available at: http://orderline.education.gov.uk/gempdf/1849625344/QCDA104983-review-of-literature-on-marking-reliability.pdf (accessed 10 May, 2014).
Suto I, Nadas R, Bell J . who should mark what? A study of factors affecting marking accuracy in Biology Examination. Research paper in education, Taylor and France Online. Available at: www.tandfonline.com/doi/abs/10.1080/0267152090272183. (Retrieved 10 May, 2014).
Tisi J, Whitehouse G, Manghan S, Burdett . A Review of Literature on Marking Reliability Research (Report for Ofqual).Slough National Foundation for Educational Research. Available at: www.Ofqual.gov.uk/files/2013-06-07-nfera-a-review-of-literature-on-marking-reliability.pdf. (retrieved 10 may, 2014).
Wheadon C, Pinot de Moira A . Gains in marking reliability from item level marking: Is the sum of the parts better than the whole? Educational Research and Evaluation: An International Journal on Theory and Practice,19:8,665-679,Dol:10.1080/13803611.2013.828629.
Cite this Article: Risiro J, 2015. Traditional or Conveyor belt marking: Exploring the way forward at Great Zimbabwe University, Zimbabwe. Greener Journal of Education and Training Studies, 5(1):024-031, http://doi.org/10.15580/GJETS.2015.1.0523014243.