Safer neighbourhoods survey and public attitude survey - options for development

Report: 13
Date: 3 February 2005
By: Commissioner

Summary

This report updates members on the options available to the MPS Research and Survey Unit with regards to the development of the Safer Neighbourhoods Survey (SNS) and Public Attitude Survey (PAS). It outlines the history of the PAS, focussing on the relationship with the SNS, provides options, estimate costs and selects the appropriate way forward.

A. Recommendation

That Members note the report.

B. Supporting information

History of the Public Attitude Survey

1. The Public Attitude Survey (PAS) measures Londoners' perceptions of policing needs, priorities and experiences and has taken place since 1983.

2. Prior to the 2002 survey, approximately 4000 interviews were carried out annually across London, giving a representative sample of residents across the 32 boroughs. This survey method was 'snap shot' with interviewing taking place across two months of the year. This mainly occurred between the months September-November.

4. The PAS interviewed residents, face-to-face in their own homes and was based on quota sampling. The main data sample was weighted to ensure it was representative of the population of London. This took place in two stages: Weighting on age, sex, ethnicity and working status at the borough level and weighting by borough population size.

5. In 2002 the Public Attitude Survey and methodology was redesigned in light of the increasing demand for robust public attitude information and in tandem with new technological advances to survey methodologies. In previous years the main drivers for the PAS were the internal requirements for data to inform corporate and local planning cycles and self-assessment exercises. Annual qualitative and quantitative summaries were considered adequate to meet such a need. In 2002, the MPA Consultation Strategy, Local Crime and Disorder Audits and central government guidance hugely raised the profile and importance of public consultation. The service also had and continues to have business requirements for the type of data that is elicited from the PAS. Best Value Performance Indicators (BVPI's) or statutory performance indicators (SPI's) and planning objectives are focused around public reassurance, fear of crime, police visibility and other perception based performance indicators that can only be gauged through surveys such as the PAS. The data not only monitors trends over time but also assesses the impact of particular events or initiatives.

6. There was and still is a very real need for 'real-time' data, rather than snap-shot data, which quickly becomes dated or compromised by one event. To this end the 'rolling-survey' methodology was introduced with reporting occurring on a quarterly basis and annually at the end of each financial year.

7. With the introduction of the step-change and the Safer Neighbourhoods (SN) programme the PAS was redesigned in 2004 to reflect the changing priorities of the Metropolitan Police Service (MPS), Metropolitan Police Authority (MPA) and Greater London Authority (GLA) and to provide results which are strategically actionable. The revised PAS now serves as the measurement tool for Londoner's confidence in policing and as a tool for continuous improvement at ward level through the Safer Neighbourhoods Survey. This approach originated from the growing recognition that it was not always 'crime' that drives Londoners satisfaction with policing, experiences of anti-social behaviour also have an impact. The revised PAS began in April 2004 and currently excludes wards that are taking part in the Safer Neighbourhoods Survey, thus enabling ward and borough level data to be collected and compared, but this does mean that the PAS does not fully cover the MPS. Methodologically the survey has continued as usual with approximately 250 in-home interviews taking place in each borough on an annual basis and reports produced quarterly and annually.

Safer Neighbourhoods Survey

8. 96 Safer Neighbourhoods wards were rolled out in April 2004. A baseline survey was carried out in these wards, interviewing 100 residents prior to the implementation of the safer neighbourhoods policing teams. The survey asks residents to identify problems in their neighbourhood, improvements the police can make and concerns residents have regarding crime and safety.

9. The methodology matched that of the PAS and therefore used quota sampling. Quotas were set on age, ethnicity, gender and working status based on 2001 census data. Five enumeration districts were randomly selected in each ward for inclusion in the study and addresses were randomly generated from within these areas. The interviews were carried out face-to-face, in resident's homes.

10. The MPS has provided baseline data for 96 wards rolled-out during April 2004. We have also provided follow-up reports three months after the implementation of the teams for six wards and will provide another follow-up report six months after the policing teams implementation (due to report in March 2005). This information has provided Safer Neighbourhoods policing teams in the 96 wards with excellent information with which to begin their consultation process and identification of key areas which are of concern to residents. The follow-up reports have provided an insight into the early success of the introduction of SN in London.

Safer Neighbourhoods Survey: A methodology for evaluating the impact of SN policing in London

11. As the SN Programme has evolved and the interest of key stakeholders (e.g. GLA, MPA) has grown ever keener it has become clear that some revision of the Safer Neighbourhoods survey model is required. The original approach to evaluating Safer Neighbourhoods included a survey strategy that featured interviews in all wards within the MPS (target = 624) and one that relied on continued use of the MPS Public Attitude Survey (in Non-Safer Neighbourhoods areas). At current prices this approach has a projected annual cost of around £1.5 million.

12. There is the potential, with appropriate use of sampling techniques, to deliver significant cost savings from the original plan, without compromising data resilience and value. There are roughly between 16-20 wards per borough and within each ward approximately 15-61 enumeration districts. The MPS would like to survey 20 of the new Safer Neighbourhood wards rolled-out during April 2005 prior to the implementation of the policing teams and continue surveying these wards on an annual basis, for the next five years.

13. The MPS suggests that 'control wards' are used in the research design. In order for changes in people's experiences and perceptions to be monitored over time, wards without policing teams and which are demographically similar and similar in terms of crime rate to the Safer Neighbourhood wards must also be monitored. This enables us to observe whether changes in the Safer Neighbourhood wards are not simply a reflection of patterns occurring elsewhere in London.

14. The MPS recommends the use of three such 'control wards'. By focussing on 23 wards for its SN evaluation design in the MPS we will be able to increase the sample size in each ward from 100 to a panel of 400 people.

Opportunities to rethink SN evaluation in light of MPS new framework agreement

15. The Research and Survey Unit are currently in the process of creating a new framework agreement of external research companies who are able to support the SNS, PAS and CVS (Crime Victims Survey). This will provide the MPS with the opportunity to commission our surveys using new methods that were previously unavailable. With this in mind the Research and Survey Unit have looked at a variety of surveying options for the development of the PAS and SNS. The surveying options are outlined below.

16. We considered the following for the survey design:

• There is a need to ensure that PAS and SNS methodologies are compatible.
• There is a need to state whether changes in key indicators of people's reported perceptions and experiences are statistically significant and whether the introduction of SN teams contribute to the opinions and experiences of the ward residents.
• We need to take account of the fact that victimisation is highly clustered both demographically and geographically and experience of crime is an important mediating variable in how people feel about policing.
• It is also important to bear in mind that each of the following survey options have advantages and disadvantages:
  • Probability sampling involves the selection of a sample from a population, based on the principle of randomisation or chance. It allows significance testing to be carried out more robustly, there is less possibility of selection bias because the selection of individuals for interview is not at the discretion of the interviewer. However, there is the possibility that groups of interest will be excluded by chance. This is a very costly option.
  • Quota sampling is cheaper and ensures a demographically representative cross-section of the population is interviewed. However to ensure interviewers do not just include 'easy to find' respondents, quotas must be based on appropriate criteria (including housing tenure) and interlocking which can raise costs.
  • Face-to-face interviews can be longer and a rapport can be established with the respondent allowing for better recall of more sensitive experiences from the respondent. However, costs increase dramatically if face-to-face surveys are conducted using probability sampling.
  • Telephone interviews are cheaper but there are time limitations for the questionnaire. Generally questions have to be more pre-coded. There are issues concerning the use of telephone sampling with regards to sample representation of the population and changing nature over time e.g. with the introduction of mobile phones, possibility of people changing telephone numbers or being cut off.
  • Postal surveys generally have a poor response rate and people who have a greater interest in the topic are more likely to respond, biasing the sample. However, they do allow respondents to free text and are cheap.
  • Panel interviews involve the selection of citizens who are representative of the community. They are used to assess service needs, identify service issues and consult on the appropriateness of service developments. They are particularly advantageous when needing to track people's views over a period of time as they measure the change in viewpoint by contacting the same individuals repeatedly. The panel does have to be refreshed periodically and to allow for attrition.

We set out the below option for the Safer Neighbourhood impact analysis choosing the most applicable of the above methodologies.

Safer Neighbourhoods Survey: Options for the methodology for evaluating the impact of SN policing in London

17. In light of the changes with regards to the purpose of the Safer Neighbourhoods Survey the Research and Survey Unit recommend that the sampling methodology incorporate interlocking quotas in terms of age, gender and ethnicity and include housing tenure and working status within the quota framework. The creation of a panel of respondents within each ward will be required in order to measure the change and the choice of face-to-face interviews will be consistent with existing methodologies for the PAS. The use of control wards, which are wards that are not selected as Safer Neighbourhoods yet but match the selected wards in terms of demographics and crime rate, will enable changes in people's experiences and perceptions to be monitored over time. This enables us to observe whether changes in people’s experiences of policing in the Safer Neighbourhood wards are not simply a reflection of patterns occurring elsewhere in London. This methodology is the most cost effective method for producing useful and robust findings on the effectiveness of SN.

Costs of MPS Surveys of the Public: New Estimates

18. The Research and Survey Unit is currently in the process of creating a new framework agreement of external market research companies who are able to support the SNS, PAS and CVS (Crime Victims Survey). The cost estimates below are working estimates only. When all tenders for the surveys have been critically reviewed we will be in the position to award the contract for the PAS, SNS and CVS to companies based on the ability to conduct the surveys, the added value of the professional expertise and overall costs. The working estimates we have are as follows:

• Safer Neighbourhoods Survey
  £519k + VAT annually. This provides the MPS with 800 interviews within each of the 23 wards (20 SN wards, three control wards) in the first year, followed by the creation of a panel of 400 interviews in the second follow-up phase (this allows for respondents to refuse to take part in the follow-up survey the subsequent year). It is envisaged that the cost will remain constant in subsequent years as a minimum of 400 interviews will always be achieved each year, whether within the embedded panel or part of the refreshed sample.
• PAS
  The cost of the Public Attitude Survey is currently at £210k. The MPS Research and Survey Unit do not envisage the costs deviating dramatically from the annual costs at present. We will be able to provide more information when the PAS contract has been awarded.

Public Attitude Survey: options for development to monitor SN/non SN pan London

19. The PAS will have to ensure data capture from across Safer Neighbourhood and Non Safer Neighbourhood Wards, distinguishing between SN/non SN areas. We will commission our survey work so that the PAS ensures pan London coverage, taking into consideration quota sampling techniques described above.

20. Currently the PAS reports on a Borough basis quarterly, aggregating the findings for each quarter and displaying results on the MPS intranet site. We now need to more robustly measure change between the quarters and it has therefore become a necessity to separate the quarterly data. We suggest reporting on Borough results annually and Met-wide results quarterly.

Timetable of development

21. February 2005 - Select Research companies to go onto the framework agreement and allocate the corporate projects. Present the proposal to the MPA finance committee. Confirm costings with MPA consultation committee.

March 2005 - The chosen survey companies require a month to set up systems, procedures and protocols for the data collection, coding, analysis and if required reporting.

April 2005 - Data collection begins for the SNS and PAS for the new financial year.

Other Data on People’s Experiences of Policing

Understanding People's Experiences of Policing: Qualitative Methods

22. The MPS understands that its surveys may not always tap the experiences and perceptions of hard-to-reach groups. In short, our consultation work must become more inclusive and more sensitive to diversity. Deputy Commissioner’s Command (DCC2) has commissioned qualitative research focussing on the following areas:

• According to the surveys minority ethnic groups continue to fear crime more than whites. The survey does not tell us why this is and was raised as an issues in need of exploration
• More information is required on different faith groups perceptions of policing
• No reported crimes - perception is that the police will not be able to do anything. We need to investigate these views.
• Lesbian, Gay, Bi-Sexual and Transgender (LGBT) groups
• Youth
• Representative groups for i.e. homeless people
• Lower levels of satisfaction amongst those that have had interaction with the police - why?
• Businesses perceptions of policing
• Victims of domestic violence
• Racially motivated crimes - lower levels of satisfaction
• Once follow up information is available from the Safer Neighbourhood wards we would be in a position to identify where wards have a positive impact. Focus groups could be conducted to explore success in more depth. This would additionally facilitate the process evaluation of the implementation for SN.

A work plan is in progress.

Local level evaluations

23. The Safer Neighbourhoods Survey will only be evaluating the impact of the Safer Neighbourhood policing teams in the 20 wards. Roll-out of the policing teams is taking place across the entire MPS until 2008. It is therefore essential that the teams in all relevant operational wards capture data locally as part of the wider SN strategy to embrace public feedback and consultation as a core component of London’s policing. The Key Informant Network (KIN) questionnaire has been designed centrally for this purpose and will be used along with other community engagement activities, such as community meetings and increased police-resident interaction on a day-to-day basis.

C. Equality and diversity implications

The respondents for the Public Attitude Survey are selected to meet set quotas according to age, sex, ethnicity and working status. At present the Public Attitude Survey does not contact set quotas of LGBT people, refugees or asylum seekers or people without permanent homes. In order to achieve this other methods of consultation are being commissioned such as focus groups.

D. Financial implications

The MPS would welcome feedback on whether the MPA would in principle support such a move toward the strategy presented in this paper. Regardless of whether the MPS surveys at a Ward or Borough level, sufficient funds are currently allocated to the Public Attitude Survey in order to track how Londoners experience and perceive policing services. The MPS suggests that a robust approach to the evaluation of Safer Neighbourhoods is necessary as well.

E. Background papers

• Public Attitude Survey

F. Contact details

Report author: Professor Betsy Stanko

For more information contact:

MPA general: 020 7202 0202
Media enquiries: 020 7202 0217/18

Send an e-mail linking to this page

Feedback