Back to All News

EDUCATION, EMPLOYMENT AND WORKPLACE RELATIONS REFERENCES COMMITTEE - 29/10/2010 - National Assessment Program-Literacy and Numeracy

Estimates & Committees
Sarah Hanson-Young 29 Oct 2010

Senator Hanson Young -Could we go back. I firstly want to pick up on something that Senator Marshall said in relation to the statically similar schools. Correct me if I am wrong, but is it not that there are wrong comparisons based on what is being used to compare schools-statistically similar schools? It is not necessarily that there are wrong comparisons, it is that the comparisons do not reflect a school's performance or are in any way able to be related to the performance based on NAPLAN. It is not necessarily that there are errors on the website? Is it not that the model by which comparisons are being made is not effective for what it is meant to be showing?

Mr Gavrielatos -We were very concerned about the methodology used, because it did not incorporate a number of other factors that equally contribute to educational outcomes, such whether there is selective enrolment at the school, whether there is exclusive enrolment patterns, the student profile and the like-all of which can contribute to educational outcomes in a school in differing ways. So you have to ensure that the methodology that you use to construct an educational advantage index is one that captures and is respectful of the complexities of schooling and the demographics within the school.

Senator Hanson Young -It is not just based on a postcode.

Mr Lipscombe -The complexity of schools is so great that it is very difficult to imagine a system of comparing them in any sort of reasonable way. Any teacher with any sort of experience will tell you that in one school in one area with exactly the same group of teachers, the student performance will vary from year to year. One group of students will come through in year 9 this year and they will perform particularly well. Another group will come through-exactly the same teachers, the students have the same parents and same background and they live in the same area-and they will perform differently because a number of things impact on them. How you take that into account in these sorts of formulae that seek to compare schools on a statistical basis, I do not know.

Mr Gavrielatos -Indeed, in our submission on page 21 we list some examples of the quite significant variation from year to year, from cohort to cohort. We say here for example that Fish Creek and District Primary School in Victoria, with 105 students was ranked second last by the Herald Sun on year 5 writing, with a score of 402-82 below the national average, but the same school in 2008 scored 477 in year 5 writing, just 8 below the national average. That is a massive variation from year to year. Again, our point is that it is inaccurate and totally dishonest to suggest that this data can be reliably used as school performance data. It is snapshot diagnostic information for teachers to interpret and for systems to interpret in order to be able to better target and meet the needs of individual children and system needs.

Senator Hanson Young -What is the purpose of trying to compare schools?

Mr Gavrielatos -I do not know. There are different purposes stated. Whenever you conduct analyses of student data and other school data you certainly look at a variety of approaches to identify where improvements can be made or cannot be made, but it has to be done in a very delicate context, recognising that there are many variables et cetera. When we look at data and interrogate data we do so cognisant of the fact that there is a margin of error, we do so cognisant of the fact that things can change from day to day and we do so cognisant of the fact that there are resources at our disposal or not, and we make judgments accordingly. So our concern is that this information is presented with a precision that is not becoming. And there are consequences to the high stakes attached to it: there is potentially a negative downward effect.

Senator Hanson Young -What measures would you like to see? What measures would you suggest are appropriate to fill out the desires of those parents, teachers and government officials who want to be able to go to the My School website and look at school performance? What measures would need to be taken in order to give an accurate picture?

Mr Gavrielatos -The provision of information to parents is not new. Contrary to many media reports and statements this year, it has existed for a long time. Parents have been able to visit schools and websites for many years to get information, but it is always important to start with the context of a school and the challenges and opportunities et cetera of a school. Thereafter you can further unfold the story in terms of the achievements in particular schools-not only through a literacy and numeracy test but in other areas of academic, sporting and cultural achievement. These are important things about the life of a school, and clearly our mission is to develop the whole child by ensuring access to a broad, rigorous, rich and rewarding curriculum for every child.

Mr Lipscombe -Can I also say that NAPLAN and tests like it are useful at a systems level. They enable you to identify shortcomings at a systems level, but the more you drill down the less reliable it becomes. You get down to the point where it is not reliable at all-it is not useful at all for making the sorts of claims that are made for it. And you can get down to the school level and value-add. As I mentioned before, if you take into account a particular class in a school, the measurement error around that is such that the class could have gone up by a year, could have gone down by a year or could have stayed where it was.

You do not know from the NAPLAN results, when they are put on the My School website or wherever else they are put, precisely where that class is on that test result. So it is the issue about reliability: it has its uses at a systems level-maybe we should identify resourcing issues generally across the system and shortcomings in the system; it can enable you to do that-but the more you drill down the more you have to question the data and the use that the data is being put to.

Senator Hanson Young -So would you argue that there is a better way of assessing, particularly, the literacy and numeracy skills of students which would then form part of the context of a broader look at what a school's performance is?

Mr Gavrielatos -In terms of student assessment, we have always felt and held the view that assessment should follow curriculum. Oddly enough, we believe that you should assess kids on what you may have taught them. NAPLAN is not based on the curriculum; it is a standardised test.

CHAIR -That is radical, isn't it?

Mr Gavrielatos -I know it is radical, but we are often called dinosaurs for that reason. We have developed a national test in the absence of a national curriculum. It will be argued that it is located in our curricula as they exist across the country but, when assessing children, we assess them in a variety of ways on a daily basis informally and formally. When it comes to the assessment regimes, there are a variety of approaches you can take to look at the health of systems and schooling in general. You can do so through sampling testing, for example. In Australia we have sampling testing when it comes to science and ICT. They are not census tests-not every kid does it. So there are other ways in which you can assess, and certainly we believe that an assessment should be linked to curriculum.

Senator Hanson Young -So it is not necessarily that you have a problem with NAPLAN; it is that you do not believe that the NAPLAN results reflect what they are being presented on the My School website as reflecting-or do you also have a problem with NAPLAN itself?

Mr Gavrielatos -We have always said that NAPLAN and other such instruments-mass standardised testing-may provide some diagnostic value but that is where they should remain because it is diagnostic value to better inform teachers, parents and schooling systems about the needs of children and better targeting the needs of students. There is a whole debate about mass standardised testing that can be had. We are not having that debate. We recognise that it may offer some diagnostic value. However, that value is compromised when you start attaching high stakes to it because you start distorting the possible diagnostic value that it may have.

Senator Hanson Young -Do you think there is any way of being able to assess an individual student's performance, put that together with their fellow students and match that up with the rest of their school population to be able to determine overall school performance?

Mr Gavrielatos -Unfortunately, it is not that simple: 'Hey presto! Here's the figure: your school is 423.' Unfortunately that just does not exist in the world of education. It is a complex world which we inhabit. It is a complex world because of the social and human dynamics of teaching and learning, and you can never reduce (a) the life of a child or (b) the operation of a school to a single figure. It just does not exist.

Senator Hanson Young -Since NAPLAN has been conducted across the country and the My School website has subsequently been rolled out, have you seen in those schools which have been listed on the site as 'underperforming' an increase in programs being delivered to them, whether that be in funding or in extra support?

Mr Gavrielatos -I forget the exact number. It may have been a handful that were announced in February by the then minister that were identified courtesy of My School, and somehow we would not have known about those schools in the absence of My School. No, I am not aware of that, because the national partnership programs for low SES and literacy had already been allocated and designated prior to My School. That was the bulk of it. We are talking about hundreds and hundreds of schools.

Mr Lipscombe -I should also say that, because of the imprecision related to NAPLAN and using it for such a purpose, it may in fact have led to a misallocation of those funds. If you are seeking to identify what are supposedly the poorest performing schools in the country you are likely not to have done that at all in whatever sense you mean, because of the errors and so on around that.

Mr Gavrielatos -Because it is the poorest performance compared to what? Compared to like school groups or compared to national averages or what have you?

Mr Lipscombe -That is right.

Mr Gavrielatos -These are all issues that need to be looked at more deeply.

Senator Hanson Young -We heard from the ACT Council of Parents and Citizens Associations earlier this morning and I asked them about if they felt that parents were encouraged or if the whole idea of NAPLAN and My School, and the publicity around that, had driven any more interest in parents being engaged directly with their schools. That is one of the arguments that have been put forward. Have you seen evidence within your member organisation of parents being more engaged with their school communities? If so, has that been a positive?

Mr Gavrielatos -I know Bob will supplement with some more direct examples as well, but in the first instance there is no doubt that we saw a massive debate about NAPLAN, My School and the like generated in this country. That is a good thing, and I would like to think that we contributed in no small way to the generation of that debate. Indeed, I say that the level of debate in this country around these matters was achieved in a relatively short period of time-particularly when compared to other countries like the UK, for example. It took them 10 or 15 years to get to that level of debate which ultimately saw governments there making changes to standardised testing and the like, having had that debate. We certainly hope that this debate continues so we can achieve sound education policy which will continue the improvement of students in our schools.

CHAIR -On that note we do have to conclude. I thank both of you again for the submission and for your appearance, and for the quality of the discussion we have had.

Back to All News