The Z-score imbroglio: Towards a fair and simple solution

Image courtesy Tharunaya

I will assume in this article the general acceptance of two principles of fairness with regard to university admission. Fairness principle 1: That, apart from any affirmative action criteria used, admission should be based on merit rankings. Fairness principle 2: That affirmative action criteria and the merit ranking scheme should not be arbitrary, but must follow fixed transparent principles or past precedence.

It is possible to show that the current solutions being debated to the Z-score imbroglio fails to meet these principles. An explanation and solution is given in the twenty numbered paragraphs below.

1. The G.C.E A’level exams: in 2011 were administered under two syllabi. One set of students sat papers set according to the new syllabus; another set (presumed to be repeaters) according to the old syllabus.

2. Admission to university: is based on a national merit ranking of all students who sit the exams, and a long standing affirmative action criterion which creates quotas for each district. The Z-score is a statistical method of expressing the performance rank of an individual within a group.

3. Precedence: In the past, when two syllabus’ were used, the Z-scores were calculated separately within the syllabus group (in statistics jargon – treating each group as a separate population) and the resulting Z-score was used to compare all the students and create a single national ranking.

4. Breaking with precedence: The first results/rankings in 2012 had changed from this method to one of pooling all the students (from both sets of exams) into one group, and then calculating the Z-scores. (I will not dissect the appropriateness of this method here. Suffice to say, there are pros and cons, and appropriateness will depend inter alia on assumptions about the relative ability of students and the relative difficulty of the exams)

5. Restoring precedence: The Supreme Court in their judgement on 22 July 2012 found the pooling-method to be unjustified and ordered a reversion to the separated-method of calculating the Z-scores. The judgement was well reasoned: stating basically that no justification had been established for a deviation from the method used in the past; and, in the absence of prior notice, students had a “legitimate expectation” (a legal principle) to be evaluated according to that precedent.

6. One merit list or two? There is some confusion amongst commentators who think that calculating the Z-score separately for two groups should have led to two sets of rankings. They are mistaken. The very purpose of calculating a Z-score is to find a common denominator by which students in different groups can be ranked against each other. (In fact, even when students sit one syllabus, they sit different combinations of exams, of different difficulty, and, in that sense, can be thought of as belonging to different groups. The Z-score of each separate exam is calculated and averaged, making the Z-score the common denominator of comparing performance across different subject groups as well).

7. Winners and losers: As a result of the change, from the pooling-method to the separated-method, as directed by the Supreme Court, as would be expected, the rankings changed: some who had been previously selected for university entrance would now be denied, and others who had been previously denied would now be selected. The relative position of those who remain selected would also change.

8. Old-syllabus set dominate losers: It has been claimed by those with access to UGC data that the change would have the following connected consequences: (a) increase the number of university entrants from the new-syllabus set and decrease the number from the old-syllabus set. (b) Result in a much lower proportion of entrants being from the old-syllabus set (presumed to be repeaters), than the average proportion of repeaters amongst the entrants in past years.

9. Remedy 1: Arises from agitations and protests by affected students: they demanded that in addition to all those who are now selected under the separated-method, all those who were previously selected under the pooling-method should also be admitted.

10. Remedy 2: Arises from the options provided to Supreme Court and on 3 September, 2012. They provide for increasing the intake ONLY from the more affected old-syllabus set, so as to achieve the outcome of increasing the proportion of old-syllabus entrants to a level similar to past proportions of repeaters (the four methods that were proposed are different only in their demarcation of where to locate the reference proportion from the past – within courses, at the district quota level, at the national-merit level etc.).

11. Remedies are different: Remedy 1 and 2 will have some similar consequences, if the change in method reduced entrants predominantly from the old-syllabus set. They are also different. Remedy 1 benefits precisely all those who faced disappointment, and none other. Remedy 2 may or may not apply to all those disappointed and may benefit others as well.

12. Violating fairness principles: Table 1 describes in simple terms some possible ways in which both Remedy 1 and Remedy 2 will give rise to injustice by violating the fairness principles. In the interest of brevity, and the non-technical reader, the discussion is not exhaustive and the interpolation of affirmative action criteria and other related issue are omitted.

13. Enumeration: in simple form is provided in table 1: digits 1 and 2 are used to signify those who sat the new syllabus and old syllabus respectively. It is non-contentious to rank their performance within their syllabus group, which is shown in columns (ii) and (iii) respectively. Columns (iv) and (v) describe possible combined rankings under the pooling and separated-methods respectively. In the example used, the UGC diagnosis is reflected, the old-syllabus group is 50% in the pooling-method and 25% in the separated-method. Cut off for admissions in the example is the 4 top students by national ranking.

14. Analysis of Remedy 1: This remedy ensures that all those selected previously by the pooling-method are precisely re-included. In the example, it requires increasing the intake to additionally admit b2. This violates Fairness principle 1, because d1 who is ranked higher in merit than b2, is not admitted. How do we explain this to d1?

15. Legitimate Expectation? An argument made by some for this violation of merit order is that b2 now has a “legitimate expectation” (a legal principle) to be admitted as a result of being given that expectation previously under the pooling-method, which the Supreme Court (also referring to “legitimate expectations”) dismissed on the basis that it violated Fairness principle 2. Even if correct, there is no reason why that “legitimate expectations” should be met in a way that violates Fairness principle 1. It can, for instance, be met by admitting both d1 and b2.

16. Flawed application: Furthermore, this is a flawed understanding of the legal principle. A “legitimate expectation” case exists when an individual forms an expectation of benefit on the basis of some existing policy, norm or practice with precedence. In this case the results announced were under an unexpected new ranking method (with no precedence) – found by Court to be inappropriate. That bureaucratic mistake cannot create a “legitimate expectations” for those who received the erroneous results – especially not when the validity of that method was also being hotly contested in the media for quite some time before the Courts over-ruled it.

17. Analysis of Remedy 2: This remedy increases the intake of the old-syllabus set only, and to some reference proportion. In the example, if we take the reference level as 50% it results in additionally admitting b2 and c2. This case is severely problematic. Now, the Fairness principle 1 is violated for both d1 and e1. Further, c2 does not benefit even from the sympathy that b2 would have for having had her hopes up and then disappointed – c2 had no expectation of being included by the pooling-method. What explanation can we give e1 for excluding her, and selecting c2 who is of a lower merit ranking?

18. Unfair: The main problem for fairness with the proposed solutions is that they aim to increase the intake not uniformly according to merit, but ONLY for those who were disappointed or ONLY for the old-syllabus set. (I have not explored here several other grievances that could arise, through the interaction with the affirmative action criterion that works on top of the merit system for selection. Remedy 2, for instance, could have the additional problem of discriminating against any new-syllabus students who were also selected under the pooling-method but not under the separated-method).

19. Grievance: It should be admitted that those who were mistakenly led to hope (as a result of the using the pooling-method of ranking) that they would be selected, have a grievance – at the very least some have lost the ability to prepare and repeat the exam in 2012 (having now to wait a further year), and others may have lost job opportunities, which they would have taken had they known they would not attend university. Grievances should be addressed. However, the means of redress will be controversial if they proliferate further grievances in the process. There are non-controversial means available: for instance, monetary compensation.

20. Solution: A solution should be based on giving effect to a set of equitable and transparent principles. The Remedy 2 presently before the Courts have neither quality. What are the criteria of equity that justify the proposals? A possible solution on the basis of holding on to the two principles of fairness, described at the outset, is the following: (a) expand the intake as much as possible, but do so from among all the students, in accordance with the operative merit ranking (subsequent to the Supreme Court decision of 22 July, 2012); (b) then, offer reasonable compensation for those who were previously selected but yet remain excluded. This solution is consistent with fairness principles outlined. It has the added advantage of simplicity.

  • Sie.Kathieravealu

    First of all, the causes that led to the creation of these problems must be brought to surface and those responsible for the creation of the problem have to be severely dealt with for creating havoc among the Youth causing FRUSTRATION which might lead to another turmoil in the country.

    Secondly those concerned must explore the necessity of “producing” and increasing the ‘graduates’ population. Are these aspirants to the Universities bent on increasing their knowledge? I have my doubts.

    “Demand and Supply” principle must be taken into consideration. Do we need a “graduate” or SLAS person just to attend to normal correspondence from the public. Many “graduates” are being employed as “management assistants” in most government departments. These “posts” can be “filled-in” by the GCE (O/L) qualified persons. They need not WASTE about six years of the precious time in Universities to get these jobs.

    Over to those concerned for their consideration.

  • http://yahoo prathap

    if they are going to divide the seats based on percentage the old syllabus students will get advantage and they can enter the university with lower grades.. there is no use for that increased 4928 seats. again the ministry has confused the students. i was eligible for medicine on the basis of the rank releases on july 22. therefore i didn’t take 2nd shy exam even though i have applied. but now i don’t know whether i will get medicine or not. please take action to give justice to the new syllabus students. if you have any suggestions forward them to my mail.

  • Ranjani

    It is the first committee appointed by the UGC chairman including Prof. Kulathunga created all these havoc due to an incorrect z scores provided to all students which is the one corrected by the Supreme Courts. Look like going back to a similar or a much worse mistake for the second time.

  • Ahamed

    It is a bloody shame on the members of the commitee which decided the Z-score first, at the begining itsellf they could have adopted the correct method of calculating the Z-score without creating a havoc.

    There are some grey areas in the new and old calculation of Z-score, for example say if one distrct has a quota of 20 students for medicin/engineering, and there are 5students already eligible for admission on married basis, so there is a chance of 25 students from rank 1-25, to enter into university from the district. My question now is by adopting new additional amount of students(4900)or new Z-score system, will the students having rank from 20 to 25 get a chance to enter universit???if not it is an unjust to the students having rank from 20 to 25. Please reply

  • http://yahoo prathap

    Hi every one. please join hands to fight for us. i am a new syllabus first shy student effected by the z score issue. at first i was 66 in my district on the basis of the ranks released on dec 26th.and i was not eligible for medicine. and i decided to take 2nd shy. again in july i got my new rank it was 52 in my district. usually the district quota for my district is 30. and according to the ranks released in july 22nd 33 students from my district were eligible to enter medicine from my district under merit basis. altogether 63 (33merit + 30quota) students got medicine including me. as i got to know that i was eligible to enter medicine, so i didn’t attend my 2nd shy exam which commenced on august 6th. again the UGC released another new rankings on september 6th separately for new and old syllabi.in that rankings i was 44 in my district among the new syllabus atudents. i got to know about the percentages of university admission from a face book group named z score. accordingly the maximum percentage allocated to 1st shy students under merit was 44% and repeaters was 59%. and the maximum quota percentage for 1st years from my district was 53% and repeaters was 72%. as a result of this many new syllabus students including me in my district lost medicine. and only 38 new syllabus students from our district are now eligible for medicine where as many old syllabus students with lower grades than us are now eligible to do medicine. the percentages which UGC considered were based on the past 5 years. it is obviously a wrong method. students who studied for two years(2nd shy people) get more opportunities than the students who studied for 3 years(1st shy people). please take this into consideration. THE GROUND VIEWS MAGAZINE AUTHORITIES, I HUMBLY REQUEST YOU TO TAKE ACTION TO PROVIDE JUSTICE FOR THE STUDENTS WHO WERE EFFECTED BY THIS WRONG CALCULATION. if you want to see any statistics please visit this face book page http://www.facebook.com/?sk=welcome#!/groups/465178600168483/