Evaluation of Raphael Services

Project Type - Mental health, Program evaluation and review


  • 2012


St John of God Health Care (SJGHC) appointed HOI to undertaken an evaluation of six (6) Raphael Services that are located in Western Australia, Victoria and New South Wales.

The Raphael Services provide community-based early intervention Perinatal and infant mental health (PIMH) services for families with children from conception to when the child reaches four years of age. The Services have been established since 2003 and services are provided in response to community needs in each state.

The evaluation had two primary aims.

  1. Conduct a comprehensive evaluation of the Raphael Centres 
  2. Develop evaluation framework.  To develop an evidence-based evaluation framework for the purposes of this review and for use by SJGHC in undertaking future evaluations of the Raphael Centre services.

HOI designed the evaluation framework based on the Donabedian Quality of Care Framework that incorporated structure, process and outcomes attributes to support quality-improvement practices.  The approach included:

  • consultations with a range of stakeholders involving a mix of face to face interviews, focus group discussions and telephone interviews. Key stakeholders consulted: clients, referrers, staff, centre management, SJGHC Executives
  • case study visits to the 6 Raphael Centres to consult with nominated stakeholders.  The objective of these consultations was to gain a comprehensive understanding of the appropriateness, effectiveness and efficiency of the service models
  • administration of an online staff survey of all Raphael Centre staff
  • analysis of demographic, outcomes (DASS21, HoNOS, Edinburgh Postnatal Depression Scale, Mental Health Questionaire ) and financial data.

HOI produced a comprehensive and evidence based evaluation report summarising the findings of the analysis undertaken and provided a suite of recommendations for improving the service. The report included an assessment of the following areas:

  • Cultural components of the service
  • Excellence in care
  • Excellence in governance and management of resources
  • Formation, education and leadership
  • Services provided to the socially disadvantaged
  • Service efficiency
  • Future evaluation strategy