Why should one take into account both the matura exam result and the EVA indicator?
The mean exam result and the EVA, that is the teaching effectiveness indicator, are complementary measures. Together, they provide more comprehensive information about a school than any of them separately. As a consequence of taking into account of both the matura exam result and the EVA indicator, the school result is presented on a two-dimensional coordinate system. Due to the statistical uncertainty, estimates of indicators of a school’s place are presented in the form of ellipses. The size of the ellipsis graphically represents the degree of that uncertainty. The more exam results are incorporated into the calculations, the lesser the uncertainty and, therefore, the smaller the ellipsis.
A description of the place of the school (ellipsis) in the coordinate system (the exam result, the EVA) may be facilitated by distinguishing of several reference points.
- Neutral schools. General or technical upper secondary schools, in which we record both the average exam results against the national figure and average effectiveness.
- Schools with high exam results and high effectiveness measured by the EVA (upper right-hand quarter).
- Schools with high exam results and low effectiveness measured by the EVA (lower right-hand quarter).
- Schools with low exam results and low effectiveness measured by the EVA (upper left-hand quarter).
- Schools with low exam results and low effectiveness measured by the EVA (lower left-hand quarter).
Besides the main types, one can observe intermediate types: schools with average results and low/high EVA and schools with average effectiveness (EVA) and low/high matura exam results. in those cases, the ellipses of the schools are located on the vertical or the horizontal axis, respectively.
The above points of reference are to help in interpretation of the school’s place in the coordinate system, rather than categorisation of schools.
How to read the graphs?
To facilitate statistical interpretation of the position of a lower secondary school, two ellipses have been plotted on the graph. The smaller, dark grey, identifies the area where 50% of schools are located. The second ellipsis, light grey, demarcates the area, where 90% of schools are located. Point (100;0) is the centre of symmetry of the ellipses.
Example 1.
LO A. Maths and Sciences indicator
The position of the green ellipsis on the graph indicates that we are dealing with a school which, in terms of the evaluation exam results (the Maths and Sciences indicator), is located close to the central point of the coordinate system, that is it is an average (neutral) institution at the national scale. Schools located close to the centre of the coordinate system are the largest category of general (technical) upper secondary schools in Poland. It is so, for both the exam result, and the EVA indicator most often take values close or not so distant from the average.
A very high number of exam data available for school A (in the three-year period, 673 new exam takers) makes the position of the school plotted with high precision, which is graphically represented by a small ellipsis. Thanks to this precision (statistically quite untypical, for school that large are a minority), we can say that the average level of skills in the Maths and Sciences area of the matura exam takers in the school is placed with a high degree of probability (95%) within the range of values about 97-100. The range may be read by projecting the ellipsis onto the horizontal axis. It was possible to identify the teaching effectiveness indicator in that area, that is Maths and Sciences EVA, with a similar degree of precision. The indicator falls within the range of around -1 through 1 (a projection of the ellipsis onto the vertical axis). Focusing on the centre of the designated range for the EVA indicator, it can be said that the average matura exam taker in that school obtain almost precisely the score at the exam on the standard scale 100/15 which resulted from the Maths and Sciences achievements with which he had come to the school.
To facilitate statistical interpretation of the position of a school in the coordinate system which forms a national frame of reference, two grey ellipsis have been plotted (contour lines). The smaller, dark grey, identifies the area where 50% of general and technical upper secondary schools from the whole Poland are located. The second ellipsis, light grey, demarcates the area, where 90% of schools of that type in Poland are located. Point (100;0) is the centre of symmetry of the ellipses. School A, analysed above, of course falls within the smaller ellipsis.
Example 2.
School B. Maths and Sciences indicator
Let’s look at the results of another upper secondary school. The confidence interval for that school against the matura exam result in the area of Maths and Sciences achievements stretches from around 83 to 88 points, that is more or less one standard deviation on the scale of students’ individual results below the national mean. This is a low result. The EVA indicator is also rather low and falls within the range (-8; -4).
The information placed on the website (below) on the number of students taking specific subjects informs that the school’s position was identified mostly on the basis of basic Maths with a small “admixture” of other science subjects, also taken mainly at the basic level.
Indicators calculated on the basis of exam data from the years 2010-2012. 2010-2012.
Number of exam takers, whose results were used in the analysis: 112.
| subject | total | advanced level |
| Maths | 112 | 1 |
| Biology | 16 | 2 |
| Chemistry | 7 | 0 |
| Physics | 2 | 0 |
| Geography | 12 | 3 |
| IT | 0 | 0 |
It should be emphasised that the fact that students take many subject at their matura exam in the school, as well as advanced level exams, does not affect directly the estimation of evaluatory exam indicators. Indeed, we can see that schools, in which exam takers are ambitious when it came to selection of exams, get better scores. However, it is so not because ambitious choice itself increases the chances for a high result, but because such choices are caused by a higher level of skills of the exam takers. However, when the choice of advanced level is not accompanied by a high level of skills, it does not lead to increasing the evaluation indicators. In other words, the applied method of analysis enables quite clear estimation of the level of matura exam takers’ skills on the basis of the tasks their solved.
The information on the matura exams taken does not only show what was the basis for calculation of indicators for a given school, but also makes it possible to reconstruct the school’s profile.
Although we used only general upper secondary schools in the examples, the results for the population of technical upper secondary schools should be interpreted analogously. It must be remembered, however, that although the mission of the former is general education, the latter must combine general with vocational education. In the second dimension, evaluation of technical upper secondary schools should be facilitated by the results of exams that confirm obtaining professional qualifications.
