Factors influential to the scoring reliability of either Speaking or Writing test

(Mr Tich)

Language proficiency test score is absolutely important to judging examinees’ language competence and generalizing their language proficiency through contexts. The score also provides basis for the recognition of language qualifications as well as helps stakeholders to situate and coordinate their efforts. Therefore, the more the scoring is, the more reliable the test score is. However, the scoring reliability is sometimes not ensured due to a number of factors. In this writing, I will analyze some factors that influence scoring reliability of Speaking test at my university. These factors include constraints, examiners’ characteristics and scoring procedure factors.

At my university, there are three or four English proficiency tests a year. The speaking tests are conducted in classrooms with two examiners and about 18-20 test takers in every single room. Due to the large number of candidates, the raters are occasionally under time constraint, so the test scores given to the last students in the list may be decided in hurry and can be inaccurate. As well as this, test scoring fee is quite low because of cost restraint; thus, honestly, some raters work since they are appointed to do so; they are not happy and do not enjoy doing it.  Another influential factor to scoring reliability is the examiners themselves. Some have busy schedule or personal problems; some are unwell on the test date due to which they cannot maintain their concentration and stable performance during the scoring time. Besides, inconsistence in understanding and interpreting the rubric, descriptors and criteria of the assessment guidelines result in bias in test scores. Additionally, some cannot keep themselves in good mood when external actors like weather are tough or unfavorable.

The scoring procedure at this school is problematic from time to time too. There are rubrics, criteria and guidelines documents for assessment and scoring. Nonetheless, not all the raters strictly follow these scoring guidelines or score against the criteria. There are examiners giving test takers scores based on their experience and emotion. They are affected by HALO effects such as test takers’ good appearance, behavior, politeness, courtesy, nice outfit and makeup, beautiful smile and great attitude, excellent voice or confidence. Seriously, some raters even base their scoring on comparison of speaking performance among test takers. All these influential factors and causes result in inaccuracy in test scores which in turn trigger failure in ensuring consistency and fairness, intra-scorer reliability as well as inter-scorer reliability.