Healthy Communities: Australians' experiences with access to health care in 2011–12 - Technical Supplement - ABS Patient Experience Survey

Healthy Communities: Australians' experiences with access to health care in 2011–12

ABS Patient Experience Survey

This section summarises methods used to calculate descriptive statistics for the performance indicator measures of patient experience, using data from the Australian Bureau of Statistics (ABS) Patient Experience Survey 2011–12. For the Patient Experience Survey, conducted annually, the ABS collects information from a representative sample of the Australian population. The Patient Experience Survey is one of several components of the Multi-Purpose Household Survey, as a supplement to the monthly Labour Force Survey.

The data included in Healthy Communities: Australians experiences with access to health care in 2011–12 relate to the survey cycle conducted from July 2011 to June 2012. At that time, the ABS collected information from individuals about their experiences with the health system in the 12 months prior to interview. Demographic information was also collected.

Scope and coverage

The Patient Experience Survey 2011–12 included persons aged 15 years and over and excluded the following:

  • Permanent members of the Australian Defence Force
  • Diplomatic personnel of overseas governments
  • Overseas residents in Australia
  • Members of non-Australian defence forces (and their dependents)
  • Persons living in non-private dwellings such as hotels, university residences, boarding schools, hospitals, retirement homes, homes for people with disabilities, prisons, etc.
  • Persons living in discrete Indigenous

These survey scope exclusions have implications when interpreting results disaggregated to the Medicare Local and peer group level.

In the 2011–12 survey, for the first time, the ABS included within the sample households located in very remote areas of Australia. Seven Medicare Locals and one peer group had more than 5% of their population in very remote areas:

Medicare Local or Peer group Percentage of population in very remote areas
Central and North West Queensland (309) 45%
Kimberley-Pilbara (508) 41%
Northern Territory (701) 23%
Goldfields-Midwest (507) 14%
Far West NSW (118) 10%
Far North Queensland (311) 8%
Country North SA (405) 7%
Rural 2 peer group 20%

Consequently, results presented for these Medicare Locals and the Rural 2 peer group in this report from the 2011–12 survey may not be comparable with results from the 2010–11 survey. This should be considered when interpreting the ABS Patient Experience Survey 2011–12 data for these Medicare Locals and the Rural 2 peer group. In this report, each of these Medicare Locals and the Rural 2 peer group are denoted with a dagger symbol (†). In some cases the results for these Medicare Locals have been suppressed (see Suppression of estimates section).

Data collection

Data were collected using Computer Assisted Interviewing, whereby responses were recorded directly onto an electronic questionnaire in a notebook computer, usually during a telephone interview. The survey is conducted at the conclusion of the ABS monthly Labour Force Survey and persons 15 years and over are randomly selected to answer a series of questions regarding their experiences as a patient in a variety of settings. For further information refer to the publication Patient Experiences in Australia: Summary of Findings, 2011–12 (ABS cat. no. 4839.0).

After fully completing the Labour Force Survey at a household, a usual resident aged 15 years or over was selected at random (based on a computer algorithm) and asked the additional questions in relation to their patient experience. If the randomly selected person was aged 15 to 17 years, permission was sought from a parent or guardian before conducting the interview. If permission was not given, the parent or guardian was asked the questions on behalf of the 15-17 year old. The question on waiting time for medical specialists (see Waiting times for medical specialists relative to health status section of the main report), was not put to the parent or guardian because it sought personal views.

Changes to survey questions

The Patient Experience Survey is conducted annually. However, questions in the survey can change over time, which can make comparisons between years difficult. In the 2011–12 survey, the following questions changed.

Cost barriers to GP care

In 2011–12, respondents who did not see but needed to see a GP in the preceding 12 months were not asked if they delayed seeing a GP due to cost. They were however asked if the reason they did not see a GP at all in the preceding 12 months was due to cost.

Cost barriers to seeing a dentist or hygienist

In 2011–12, respondents who did not see but needed to see a dentist, hygienist or dental specialist in the preceding 12 months were not asked if they delayed seeing one of these dental professionals due to cost. They were however asked if the reason they did not see a dental professional at all in the preceding 12 months was due to cost.

Cost barriers to seeing a medical specialist

In 2011–12, respondents who did not see but needed to see a medical specialist in the preceding 12 months were not asked if they delayed seeing a medical specialist due to cost. They were however asked if the reason they did not see a medical specialist at all in the preceding 12 months was due to cost.

Sample size

Of persons asked to participate in the survey, 26,437 (79.6%) fully responded to the Patient Experience Survey 2011–12.1 This includes 334 interviews for persons aged 15 to 17 years, where permission was not given by a parent or guardian for a personal interview. In these circumstances the parent or guardian responded to a reduced range of questions on behalf of the 15-17 year old.1 The sample was designed to produce representative results for states and territories. Data collection for the Patient Experience Survey 2011–12 predated the establishment of some Medicare Locals. Accordingly, for some Medicare Locals that have a population living in very remote parts of Australia or in discrete Indigenous communities, survey results are not available at a level of reliability equivalent to state and territory level estimates for the 2011–12 survey cycle reported elsewhere.

In 2012, the National Health Performance Authority commissioned the ABS to assign all respondent data to Medicare Local geography to produce survey results for each Medicare Local and peer group. The ABS Patient Experience Survey 2011–12 data have been weighted to meet independent population benchmarks for the civilian population aged 15 years and over living in private dwellings in each state and territory, at 31 March 2012.2

In the weighting method final weights were compiled through a generalised regression process taking into account age group, sex, State by Capital City Statistical Division/Rest of State plus ACT and NT, as defined in the ABS Australian Standard Geographical Classification (ASGC). The ABS considered the possibility of re-weighting the Patient Experience Survey data to take into account the Medicare Local level geography. ABS advised the Authority that this was not necessary as previous investigations by ABS (both data-based and algebraic) had indicated that re-weighting of this type had little or no effect on prevalence rates as presented in this report or the associated confidence intervals. Further, given the relatively small sample size and consequent size of some of the individual cells, attempting to re-weight the Patient Experience Survey 2011–12 data might have resulted in degrading accuracy due to the need to collapse benchmarks such as age, sex and part of state that do benefit accuracy.

Data quality

The Patient Experience Survey results represent respondents’ perception of their health status and views on experiences of using the health care system. As these data are self-reported, respondents’ recall, perceptions and views should be considered when interpreting the results.

Percentages (proportions)

The Patient Experience Survey 2011–12 results are expressed in terms of percentages, that is, the number of people in the Medicare Local area with a characteristic of interest, divided by the defined eligible total population and expressed as a percentage (per one hundred population). The denominator varies by survey data item. For example, the denominator for many survey data items is all persons aged 15 years and over in Australia, whereas the denominator for survey data items about emergency departments (ED) is all persons aged 15 years and over in Australia who had been to an ED for their own health in the previous 12 months. The responses ‘Don’t know’ or other applicable categories are included in the percentage denominator, unless otherwise stated. Further, percentages presented in this report for Medicare Locals and peer groups are weighted survey estimates. The Patient Experience Survey results for Medicare Local peer groups are calculated using the results of all survey responses within the peer group and have been described as an ‘average’ in this report. These peer group results are not the average or arithmetic mean of the Medicare Local percentages presented in this report.

Reliability of percentages (proportions)

Two types of error are possible in estimated percentages based on a sample survey. These are non-sampling error and sampling error.

Non-sampling error may occur in any data collection and at any stage throughout the survey process. Examples include:

  • Non-response by selected persons
  • Questions being misunderstood
  • Responses being incorrectly recorded
  • Errors in coding or processing the survey data.

The ABS attempts to minimise non-sampling error through a range of procedures including cognitive testing, extensive interviewer training, detailed interviewer instructions and follow-up approaches to selected households.

Sampling error occurs because a subset of the total population is used to produce estimates that are designed to represent the whole population. Sampling error can be reliably measured, as it is calculated based on the scientific methods used to design surveys.

As the percentages reported in Healthy Communities: Australians’ experiences with access to health care in 2011–12 are based on information obtained from a sample survey, they are subject to sampling error. That is, they may differ from proportions that would have been produced if all persons in Australia had been included in the survey. Accordingly, confidence intervals are released, in addition to point estimates, to indicate the range in which the population value (as compared to the statistic derived from respondent surveys) is likely to lie.

Confidence intervals are constructed using the point estimate of the population value and its associated standard error. There is approximately a 95% chance (i.e. 19 chances in 20) that the population value is within 1.96 standard errors of the estimated proportion. The 95% confidence interval is equal to the estimated percentage plus or minus 1.96 standard errors.

Since the Patient Experience Survey started in 2009, making this a relatively new collection, its temporal stability between 2010–11 and 2011–12 was assessed by comparing the point estimates and their variability. They were found to be good for both high prevalence items (e.g. seeing a GP) and low prevalence items (e.g. cost barriers to seeing a GP) between 2010–11 and 2011–12.

Suppression of estimates

The formation of confidence intervals is based on a normal approximation and the application of the central limit theorem. This approximation is less valid for estimated proportions that are either very small (close to 0%) or very large (close to 100%). As such, the decision was made, in consultation with the ABS, to impose stricter confidence interval constraints on estimates as they get closer to 0% or closer to 100%.

All point estimate percentages included in this report have a confidence interval width less than 20 percentage points. A confidence interval width of 20 percentage points, for a point estimate of 40% for example, means that we can be 95% confident that the true population value is between 30% and 50%. Medicare Locals with estimated proportions with a confidence interval width greater than 20 percentage points, or for which the results are not a fair assessment for the Medicare Local due to survey scope exclusions (see Scope and coverage) are indicated as not available for publication (NP).

Point estimate percentages that are between 5% and 15% or between 85% and 95% are only included in this report if their confidence interval width is less than 15 percentage points. Point estimate percentages that are either less than or equal to 5%, or greater than or equal to 95%, are only included in this report if their confidence interval width is less than 10 percentage points. Medicare Local catchments that do not satisfy these criteria are indicated as not available for publication (NP).

In addition to these constraints, five Medicare Local catchment areas were identified as having a small sample and potentially less robust estimates. These are Far West NSW (Medicare Local reference code 118), Great South Coast (Vic) (212), Lower Murray (Vic/NSW) (213), Central and North West Queensland (309) and Kimberley-Pilbara (WA) (508). Point estimate percentages are only included in this report for these Medicare Local catchments where the confidence interval width is less than 10 percentage points.

Annotation of estimates

This report includes some estimated proportions with a 95% confidence interval that includes either 0% or 100%. This means that the estimate is not statistically significantly different from either 0% or 100%. These estimates are annotated with a ‡ symbol. Inference regarding these estimates should be made with caution.

Significance testing

When comparing two point estimates or percentages of a characteristic of interest within a survey, it is useful to determine the degree of certainty of differences between them or whether the observed differences relate to other factors, such as sampling variability. One way to compare two point estimates is to test whether the difference between them is statistically significant. This test assesses whether the difference between two point estimates is statistically significant at the 95% level. If the two estimates are statistically significantly different, there is a very small chance (5% or less) that differences between them relate to sampling variability or other factors. In presenting ABS Patient Experience Survey 2011–12 results by Medicare Local, a # (hash) has been added to those Medicare Locals for which there is a statistically significant difference (at the 95% confidence level) to the point estimate for their peer group.

1. Australian Bureau of Statistics 2012, Patient Experiences in Australia: Summary of Findings, 2011–12 , Explanatory Notes, cat. no. 4839.0, ABS, Canberra, viewed 22 May 2013, http://www.abs.gov.au/AUSSTATS/abs@.nsf/Latestproducts/4839.0Explanatory%20Notes12011-12?opendocument&tabname=Notes&prodno=4839.0&issue=2011–12&num=&view=.

2. Estimated Resident Population aged 15 years and over for Australia based on the ABS 2011 Census of Population and Housing as at 30 June 2011, Australian demographic statistics, March 2012, cat. no. 3101.0, ABS, Canberra.