Programme Evaluation
From the outset, evaluating Project K has been a priority. Our credibility and ability to develop and expand Project K depends on proven performance. It is therefore vital for all of us that we can prove that the programme is worthwhile and achieving its objectives.
Reflecting that, qualified staff at National Support Office are dedicated to evaluation and a database has been developed to help measure programme outcomes. The evaluation measures were developed by Project K and other experts, including Dr Niki Harre of The University of Auckland.
Project K assesses the effectiveness of its programme by conducting both outcome and process evaluation. Outcome evalutation looks at whether the outcomes of a programme are consistent with Project K's stated outcomes, e.g. did the programme work and if so, how well? Process evaluation assesses whether the programme is meeting required standards.
In line with our policy of continuous improvement, evaluation design and procedures have been refined and improved over the years.
The objectives of the Project K programme are that:
- Students are motivated to pursue education and employment.
- Students have the confidence to set challenging goals and are motivated to achieve them.
- Students have positive social relationships with peers and family
- Students have the ability to interact with adults to obtain the support and resouces they need.
- Students are connected to their community and are aware of the resources and activities available to them, and the opportunities to contribute.
- Students have a positive attitude towards health, and adopt behaviours that promote wellbeing.
- Young people are economically and socially contributing members of society.
To ensure that our evaluation design and practices remain credible and also to provide opportunities for external evaluation projects, we have developed an association with University of Auckland and Massy University.
We are also open to proposals for independent research projects from Masters and PhD students or other research agencies.
Student Identification
The concept of self-efficacy is a key factor in enhancing personal achievement and wellbeing. The strength of self-efficacy beliefs influences how an adolescent thinks, feels, motivates him or herself, behaves and therefore ultimately how successfully the young person will make the transition from adolescence to adulthood. Project K builds self-efficacy by providing challenges that enable students to experience success through effort and perseverance.
The Self-efficacy Questionnaire, along with teacher input, is used to identify the Project K target group. Informed consent is sought from school staff, parents and students, and selected students are then randomly allocated to a group of 12 Project K students, or a control group.
The control group are evaluated but do not participate in the programme. Comparisons between the control group and the Project K group pre- and post-programme allows us to conclude that any differences between the two groups reflect the influence of the programme, and are not due to normal developmental changes. Pre- and post-programme evaluation measures include the Self-efficacy Questionnaire, a Health and Lifestyle Behaviours Questionnaire and school data.
Evaluation Measures
Self-efficacy
Project K has designed a measure of self-efficacy, specifically for New Zealand students, to provide Project K with a robust measure at the leading edge in evaluation. The measure assesses student beliefs in their abilities in three areas, school work, interpersonal relationships and being able to get help from adults. Confidence in each of these domains can affect whether a student will attempt a task and encourage persistence, especially in the face of temporary setbacks.
Health and Lifestyles Behaviours
The health and lifestyle measure is designed to examine the impact of Project K on adolescent health behaviours. The items for this measure were based on: behaviours that Project K is designed to influence, a literature search into key health behaviours in adolescence and New Zealand youth health statistics. Also included in the questionnaire are questions on perceived levels of parental monitoring and family cohesion, which have been found to have a protective influence on adolescent risk behaviour. Although Project K may not directly impact on parental behaviour, these measures have been included to help show whether students respond differently to the Project K programme, depending on the quality of family relationships and support.
Social Competence
A measure of levels of social competence has also been included in the evaluation. Social competence, the ability to manage behaviour, thoughts and feelings in adaptive ways is considered essential for successful adjustment at home, school and in the community. The impact of Project K on social competence is examined through parents’ observations of behaviour in the home.
Objective Measures
Objective measures are also collected to provide a more complete view of the impact of Project K on student outcomes. We examine students’ goal achievements, school qualifications and enrolment status, and on activities on leaving school, such as whether students are in further education, in full-time work or inactive.