Healthy Communities: Australians' experiences with primary health care in 2010–11 - Technical Supplement - ABS Patient Experience Survey

Healthy Communities: Australians' experiences with primary health care in 2010–11

ABS Patient Experience Survey

This section summarises methods used to calculate descriptive statistics for the performance indicator “measures of patient experience”, using data from the Australian Bureau of Statistics (ABS) Patient Experience Survey 2010–11. The Patient Experience Survey is conducted annually when ABS collects information from a representative sample of the Australian population. The Patient Experience Survey is one of several topics on the Multipurpose Household Survey, as a supplement to the monthly Labour Force Survey.

The data included in Healthy Communities: Australians' experiences with primary health care in 2010–11 relates to the survey cycle conducted from July 2010 to June 2011. At that time, the ABS collected information from individuals about their experiences with the health system in the 12 months prior to interview. Demographic information was also collected.

Scope and coverage

The Patient Experience Survey 2010–11 included persons aged 15 years and over and excluded the following:

  • Members of the Australian permanent defence forces
  • Diplomatic personnel of overseas governments
  • Overseas residents in Australia
  • Members of non-Australian defence forces (and their dependents)
  • Persons living in non-private dwellings such as hotels, university residences, boarding schools, hospitals, retirement homes, homes for people with disabilities, prisons, etc.
  • Persons living in very remote parts of Australia
  • Persons living in discrete Indigenous communities.

These survey scope exclusions have implications when interpreting results disaggregated to the Medicare Local and peer group level. Survey results for two Medicare Locals have not been presented in this report because the exclusion criteria outlined above meant a significant proportion of the population living in these Medicare Locals was not covered by the scope of the 2010–11 survey. Results for ML309 Central and North West Queensland (where at least 45% of the population were in very remote areas and not covered) and ML508 Kimberley-Pilbara (where at least 41% of the population were in very remote areas and not covered) are denoted as not available for publication (NP). Further, another five Medicare Locals and one peer group had more than 5% and less than 25% of their population in very remote areas (which were excluded from the scope of the survey):

Medicare Local or Peer group Percentage of population in very remote areas
Northern Territory (701) 23%
Goldfields – Midwest (507) 14%
Far West NSW (118) 10%
Far North Queensland (311) 8%
Country North SA (405) 7%
Rural 2 peer group 20%

Consequently, results presented for these Medicare Locals and the Rural 2 peer group may not represent a fair assessment of the circumstances for these areas. This should be considered when interpreting the ABS Patient Experience Survey 2010–11 data for these Medicare Locals and the Rural 2 peer group. In this report, each of these Medicare Locals and the Rural 2 peer group are denoted with a dagger symbol (†).

Data collection

After fully completing the Labour Force Survey at a household, a usual resident aged 15 years or over was selected at random (based on a computer algorithm) and asked the additional questions in relation to the patient experience. Data were collected using Computer Assisted Interviewing, whereby responses were recorded directly onto an electronic questionnaire in a notebook computer, usually during a telephone interview. For further information refer to the publication Patient Experiences in Australia: Summary of Findings, 2010–11 (ABS cat. no. 4839.0).

Sample Size

Of persons asked to participate in the survey, 26,423 (81.4%) fully responded to the Patient Experience Survey 2010–11. The sample was designed to produce representative results for states and territories. Data collection for the Patient Experience Survey 2010–11 predated the establishment of Medicare Locals. Accordingly, for some Medicare Locals with their population living in very remote parts of Australia or in discrete Indigenous communities, survey results are not available at a level of reliability equivalent to state and territory level estimates for the 2010–11 survey cycle reported elsewhere.

In 2012, the ABS was commissioned by the National Health Performance Authority to assign all respondent data to Medicare Local geography to produce survey results for each Medicare Local and peer group. The ABS Patient Experience Survey 2010–11 data have been weighted to meet independent population benchmarks for the civilian population aged 15 years and over living in private dwellings in each State and Territory, excluding those living in very remote areas of Australia, at 31 March 2011.

In the weighting process final weights were compiled through a generalised regression process taking into account sex, age group, State by Capital City Statistical Division/Rest of State plus ACT and NT, as defined in the ABS Australian Standard Geographical Classification (ASGC).

The ABS considered the possibility of re-weighting the Patient Experience Survey data to take into account the Medicare Local level geography. ABS advised the Authority that this was not necessary as previous investigations by ABS (both data-based and algebraic) had indicated that re-weighting of this type had little or no effect on prevalence rates as presented in this report or the associated confidence intervals. Further, given the relatively small sample size and consequent size of some of the individual cells, attempting to re-weight the Patient Experience Survey 2010–11 data might have resulted in degrading accuracy due to the need to collapse benchmarks such as age, sex and part of state that do benefit accuracy.

Data quality

The Patient Experience Survey results represent respondents’ perception of their health status and views on experiences of using the health care system. Respondents’ recall, perceptions and views are influenced by a number of factors which should be considered when interpreting the data.

The definition of urgent medical care was left to respondents to determine. However, discretionary interviewer advice was available as follows:

Include: Health issues that arose suddenly and were serious, e.g. fever, headache, vomiting, unexplained rash. Exclude: Appointments to get a sick certificate for work.

Percentage (proportions)

The Patient Experience Survey 2010–11 results are expressed in terms of percentages, that is, the number of people in the Medicare Local area with a characteristic of interest, divided by the defined eligible total population and expressed as a percentage (per one hundred population). The denominator varies by survey data item. For example, the denominator for many survey data items is all persons aged 15 years and over in Australia, whereas the denominator for survey data items about emergency departments (ED) is all persons aged 15 years and over in Australia who had been to an ED for their own health in the previous 12 months. The responses “Don’t know” or other applicable categories are included in the percentage denominator, unless otherwise stated. Further, percentages presented in this report for Medicare Locals and peer groups are weighted survey estimates. The Patient Experience Survey results for Medicare Local peer groups are calculated using the results of all survey responses within the peer group and have been described as an “average” in this report. These peer group results are not the average or arithmetic mean of the Medicare Local percentages presented in this report.

Reliability of percentages (proportions)

Two types of error are possible in estimated percentages based on a sample survey. These are non-sampling error and sampling error.

Non-sampling error may occur in any data collection and at any stage throughout the survey process. Examples include:

  • Non-response by selected persons
  • Questions being misunderstood
  • Responses being incorrectly recorded
  • Errors in coding or processing the survey data.

The ABS attempts to minimise non-sampling error through a range of procedures including cognitive testing, extensive interviewer training, detailed interviewer instructions and follow-up approaches to selected households.

Sampling error occurs because a subset of the total population is used to produce estimates that are designed to represent the whole population. Sampling error can be reliably measured, as it is calculated based on the scientific methods used to design surveys.

As the percentages reported in Australians’ experiences with primary health care in 2010–11 are based on information obtained from a sample survey, they are subject to sampling error. That is, they may differ from proportions that would have been produced if all persons in Australia had been included in the survey. Accordingly, confidence intervals are released, in addition to point estimates, to indicate the range in which the population value (as compared to the statistic derived from respondent surveys) is likely to lie.

Confidence intervals are constructed using the point estimate of the population value and its associated standard error. There is approximately a 95% chance (i.e. 19 chances in 20) that the population value is within 1.96 standard errors of the estimated proportion. The 95% confidence interval is equal to the estimated percentage plus or minus 1.96 standard errors.

All point estimate percentages included in this report have a confidence interval width less than or equal to 20 percentage points. A width of a confidence interval of 20 percentage points, for a point estimate of 40% for example, is between 30% and 50%. Medicare Locals with estimated proportions with a confidence interval width greater than 20 percentage points, or for which the results are not a fair assessment for the Medicare Local due to survey scope exclusions (see Scope and coverage) are indicated as not available for publication (NP).

Significant testing

When comparing two point estimates or percentages of a characteristic of interest within a survey, it is useful to determine the degree of certainty of differences between them or whether the observed differences relate simply to, for example, sampling variability. One way to compare two point estimates is to test whether the difference between them is statistically significant. This test assesses whether the difference between two point estimates is statistically significant at the 95% level. If the two estimates are statistically significantly different, there is a very small chance (5% or less) that differences between them relate to, for example, sampling variability. In presenting ABS Patient Experience Survey 2010–11 results by Medicare Local, a # (hash) has been added to those Medicare Locals for which there is a statistically significant difference (at the 95% confidence level) to the point estimate for their peer group.

Survey results for Aboriginal and Torres Strait Islander Peoples

The ABS Patient Experience Survey, 2010–11 did not survey people living in very remote areas of Australia or discrete Indigenous communities specified in the Indigenous Communities Frame of the ABS Population Surveys Framework (see Glossary). Approximately 400 Aboriginal and Torres Strait Islander people responded in the Patient Experience Survey 2010–11 in other parts of Australia, compared to the non-Indigenous sample size of approximately 26,000. These survey respondents who reported that they were Australian Aboriginal and/or Torres Strait Islander comprised 1.5% of all respondents, compared with 2.4% of the Estimated Resident Population who were Aboriginal and Torres Strait Islander peoples.1 This difference in survey sample outcome, together with the survey scope exclusions, was considered by the Authority when interpreting the survey results for Aboriginal and Torres Strait Islander peoples. Further, the primary health care service usage, access and experiences of Aboriginal and Torres Strait Islander peoples who live in discrete Indigenous communities or in remote areas of Australia, may differ appreciably from those Aboriginal and Torres Strait Islander peoples who responded to the ABS Patient Experience Survey 2010–11. Accordingly, while responses from Aboriginal and Torres Strait Islander people are included in the results presented in this report, they are not presented separately.

1. Using preliminary Estimated Resident Population aged 15 years and over for Australia based on the ABS 2011 Census of Population and Housing as at 30 June 2011 as published in Australian Demographic Statistics, March 2012 (cat. no. 3101.0)