OTRU conducts formative and outcome evaluations of Smoke-Free Ontario Strategy programs.

• Formative evaluations provide information about how well programs are doing in their early stages, allowing program managers and policymakers to adjust guidelines and implementation processes.
• Outcome evaluations provide policymakers and managers with evidence about the impacts of specific tobacco control programs.

We combine quantitative and qualitative approaches to measure the results of interventions and explore when and why particular outcomes occur. Agencies receiving Smoke-Free Ontario program funding are expected to evaluate their own work; OTRU’s evaluation efforts focus on programs delivered through Public Health Units, which are expected to adapt programs to local contexts. OTRU also collaborates with agencies in evaluating high priority programs with a province-wide reach.

Current program evaluation projects include:

• Anti-Contraband Measures: Evidence for Better Practice
  
  Using a mixed methods approach, this study draws policy lessons learnt from anti-contraband measures in Canada and around the world.  The results are based on a comprehensive literature search, a series of key-informant interviews, and four expert focus panels held in Canada and Europe.  Analysis suggests that the type of contraband and the means of distribution influence the efficacy of different policies and where the risk might be displaced when action is taken.  Therefore policy measures that will adequately address contraband tobacco are equally dependent on context and source. 

• Evaluation Newsletter: Smoking and Quitting Behaviours of Low SES Adult Smokers

• Total Retail Display Ban Process Evaluation
  
  This evaluation examines the total retail display ban that took effect May 31, 2008.  Through document review and interviews with key-informant stakeholders from the Ministry of Health Promotion, Tobacco Control Area Networks, Public Health Units, health advocacy groups, and vendor associations, we aim to document and analyze the successes and challenges encountered in implementing and enforcing the ban.  Results from this study could demonstrate the effectiveness of the implementation of a total retail display ban and could be used as a model for other jurisdictions.

• Youth Action Alliances
  
  Case study evaluations will be conducted with Youth Action Alliances that have recruited youth from specific neighbourhoods and communities that have been identified as underserved.  Each study will use multiple data sources and qualitative analysis to evaluate and describe the successes and challenges in establishing each group and the perceived impacts on those involved including the wider communities and neighbourhoods in which groups operate.

  Case Study One: Evaluates the process of establishing Youth Action Alliances in Toronto in neighbourhoods that were identified in the Strong Neighbourhoods Report (2005) as receiving insufficient resources compared to need. 

  Case Study Two: Evaluates the creation of two Youth Action Alliances comprised of youth who identify as Aboriginal. 

• Developing Macro and Micro Level Evaluation Plan for Alberta Cancer Board Cessation System.
  
  The Alberta Cancer Board has contracted the Ontario Tobacco Research Unit to design a Macro and Micro Evaluation Plan for the Alberta Cessation System and its programs. The hub of the Alberta Cessation System is a Cessation Center, which will be linked with existing and new tobacco control efforts in the province that influence and support smokers to quit. Currently an evaluation plan for Cessation Center and Ask Advise Refer programs is being developed, which will consist of the following components: a program logic model detailing inputs, activities, outputs and outcomes; a clear set of evaluation questions pertaining to all levels and elements of the logic model; a set of performance indicators to properly assess inputs, outputs and outcomes of the program; a program evaluation design detailing approaches toward formative and outcome assessment of program implementation and effectiveness.
  

• How Well is the Current Smoking Cessation Strategy Working for Young Male Smokers?
  

  Using a mixed method approach, this evaluation study assesses the extent to which Ontario’s smoking cessation system meets the needs of young male smokers who are 19-29 years old.   We chose this sub-population since data from the latest Canadian Community Health Survey (2005) showed that young Ontario men aged 25-29 have the highest prevalence of current smoking at 39%; almost double that of their female counterparts (21%) and of Ontario’s general smoking population (20%).  The high level results are presented in the newsletters below as well as in the final report (forthcoming).

• Evaluation Newsletter: Smoking and Quitting Behaviours of Young Men

• A Model for Assessing Gaps in Smoking Cessation Systems and Services in a Local Public Health Unit. Reducing the burden of tobacco disease requires a comprehensive smoking cessation system with multi-level interventions (policies and programs) that address environmental, institutional, and social systems. An overall vision for smoking cessation in Ontario has been developed and provides a basis for evaluating the adequacy of the existing system. Those responsible for comprehensive local tobacco control need protocols for assessing and planning local systems consistent with this vision. A pilot study, aimed at determining the methodology needed to reveal the needs for a comprehensive smoking cessation service, was conducted in a Public Health Unit in Ontario. The high level results are presented in the newsletters below as well as in the final report.
  
  
• Evaluation Newsletter: Cessation Services in Muskoka
  
  
• Formative Evaluation of the Youth Action Alliance (YAA) Program. This mixed-methods evaluation explores patterns of implementing the Youth Action Alliance in Public Health Units across the province. Program-wide surveys, together with in-depth comparative case studies, provide a wealth of information about what is, and is not, working well; for whom; under what circumstances; and why.
  
  
• Formative Evaluation of the Smoke-Free Ontario Act: This evaluation includes three province-wide compliance surveys of randomly selected tobacco vendors and public places (restaurants and bars): one survey conducted prior to implementation of the Act on May 31st 2006, and two surveys conducted post implementation. Estimates of compliance are presented for youth access to tobacco products, point of sale promotions, and smoking in public places at the provincial-, regional-, and premise type-level. Comparisons are reported between surveys, regions and premise type. This quantitative approach is bolstered by key-informant interviews, population surveys and media analysis.
  
  
  • Baseline Survey Report
  • First Post-Implementation Survey Report
  • Second Post-Implementation Survey Report