Programme for International Student Assessment

Methodology

An international consortium of experts led by the Australian Council for Educational Research (ACER) is responsible for the design and implementation of PISA. A US consortium member, Westat Inc., has responsibility for defining the sample procedures for each country, quality assuring each country´s sample and producing weights for the survey data. As ONS is the national project manager for PISA in Northern Ireland, they are responsible for drawing the sample of schools and students in line with these procedures and submit information to Westat Inc.

The survey in 2000 was the first international survey of this scope to have been carried out among students.  There were a number of earlier studies which produced a basis to build on, but it was important to standardise the survey procedures and literacy tests so as to allow comparable data to be collected from all participating countries.  Therefore PISA requires countries to conduct all aspects of the survey according to an agreed design and using common instruments.

Sample

As in 2000, the target population for the 2003 survey was 15 year olds or those recently turned 16 who are enrolled in schools.  The OECD requirement is that at least 4,500 students should be sampled in the UK but as the Northern Ireland target population represents less than 4% of the total UK population, the sample contribution required from the region would be relatively small. In turn, this would not allow reliable estimates of student achievements in NI so it was agreed that a sufficiently large sample (approximately 3,000 students) was to be drawn from NI.

A two-stage sample design was used.  At the first stage a sample of schools was drawn and then at the second stage a sample of students was selected from each participating school.

The Department of Education supplied a list of schools in NI in which students are enrolled in Years 11 or 12. The list excluded special educational needs schools. The schools were then stratified according to factors that are related to educational outcomes (i.e. grant-aided or independent). Grant-aided schools were then stratified into grammar and secondary schools and were then further stratified by the proportions of their students achieving 5 or more good passes at GCSE level. The schools were also grouped into the five Education and Library Boards. A sample of 150 schools was then selected with the probability proportional to the estimated number of students aged 15 or recently turned 16, so that equal numbers of students could be selected from each participating school. There were also substitute schools selected which could be approached if a school refused to participate.

The fieldwork for this survey was conducted during the month of March when it is thought that students are at their peak intellectual performance.