The term “External Quality Assurance (EQA)” has a variety of possible definitions. 1 Perhaps the most all embracing definition of quality in healthcare is one of the earliest from Donabedian 2 : “The managed process whereby the comparison of care (in the present context, laboratory results) against predetermined standards is guaranteed to lead to action to implement changes, and ensuring that these have produced the desired improvement”.
Donabedian’s definition by implication outlines the cyclical nature of quality improvement. This paper aims to describe the cycles of EQA as they affect patient results and care and to outline possible pathways to greater public accountability from the perspective of clinical biochemistry.
“Management can have measurable effects on performance—an aspect that is usually only appreciated when things go wrong”
We need to apply the broadest use of this definition to laboratory practice to ensure that its benefits are passed on to patient care in the form of more reliable results. The traditional subdivision of laboratory practice into its service elements of preanalytical, analytical, and postanalytical phases is insufficient to describe laboratory practice as influenced by EQA. The substance and style of laboratory management will have influence beyond these phases on the ability of the laboratory to deliver a reliable service to patients. Management can have measurable effects on performance—an aspect that is usually only appreciated when things go wrong.
Any interpretation of such a definition implies an audit cycle of continuous improvement. Each stage in the process of EQA can be held to have its own cycle. The cycles that are described here are principally, but not exclusively: scheme performance as overseen by steering groups (fig 1 1), ), current EQA practice where specimens are circulated and the results are reviewed within laboratories (fig 2 2), ), and the work of the National Quality Assurance Advisory Panels (NQAAP) (fig 3 3 ).
Scheme organiser quality cycle. EQA, external quality assurance; NQAAP, National Quality Assurance Advisory Panels; QA, quality assurance.
Laboratory quality cycle. EQA, external quality assurance; NQAAP, National Quality Assurance Advisory Panels.
National Quality Assurance Advisory Panels’ (NQAAP) quality cycle.
Common to all schemes is the circulation of material to enable interlaboratory comparisons. This material may be derived entirely or partially from human or other mammalian sources, and is typically in lyophilised or liquid form. The interpretation of data generated from measurements on this material may itself be treated by some schemes as EQA material. A scheme is being piloted that circulates clinical and laboratory data for interpretation by laboratories or individuals. This raises important clinical governance issues, which will be dealt with later.
The responsiveness and relevance of any scheme to clinical practice is dependent on the dynamics between those involved in the day to day running of the scheme (which for convenience we will wrap up in the title of “scheme organiser”) and, in the case of the UK, the steering group composed of independent experts who guide the scheme and help to provide professional (if not public) accountability. The scheme organiser quality cycle (fig 1 1) ) incorporates those elements of direct laboratory contact with the scheme, namely the receipt of EQA material and of the report after data processing. These points of contact are the yardstick by which the laboratory judges the value of the scheme as a tool for performance improvement and education. Most scheme organisers are only too happy to be proactive in helping their users through these processes.
The laboratory quality cycle (fig 2 2) ) is more reactive in that the user awaits prompts from the scheme organiser, awaiting either EQA material for analysis or the scheme’s report, which may require remedial action.
The NQAAP quality cycle (fig 3 3) ) is also reactive, in that the panels rely on the candour of the scheme organisers when identifying poor performers in situations where the panels need to exercise judgement as to the corrective response required. The confidential nature of these relationships, a feature of EQA schemes in the UK since their inception, has never been challenged, and is an obvious source of sensitivity within the laboratory community. With increasing moves towards transparency and visible public accountability, there will be pressure to relinquish this feature in the future.
The interlinked nature of these cycles is evident from figs 1 1 –3 . One can describe further cyclical interactions at each end of the chain. Thus, the laboratory cycle should link in with directorate of pathology and thence trust level clinical governance procedures. The NQAAP cycle reports to the Joint Working Group on Quality Assurance (JWG), a body composed principally of panel chairs, with its own chair appointed by the Royal College of Pathologists. The panels interact with accrediting bodies (principally CPA (UK) Ltd) in their approval of the schemes themselves.
Effective EQA is clearly not just a matter of accuracy and precision at the bench, nor simply one of application of interlaboratory data to improve these parameters. It requires integrity (in every sense) in all the elements of its organisation. We need to look at the audit cycles in figs 1 1 –3 more closely to see how variations in different parts of the process can affect the perception of laboratory performance and its contribution to patient care.
Many aspects of scheme design can affect the external assessment of laboratory performance. Although it is usually impracticable for the frequency of distribution of specimens to reflect clinical practice, it should be frequent enough to identify problem trends in seldom requested analytes, such as trace metals and vitamins. EQA schemes need to circulate material with sufficient frequency to identify those laboratories that perform poorly because infrequent batches are giving them insufficient opportunity to maintain the analytical skills required in these often challenging assays. A good scheme will also circulate material with concentration pitched to examine clinical decision levels, in addition to technical issues that inform clinical usefulness, such as linearity and detection limits.
The scheme should ensure that despatch arrangements are appropriate for preservation of specimens. Schemes should allow laboratories sufficient time for analysis and reporting, and have a clear policy on accepting late submissions and amendments for whatever reason.
Scheme organisers need to provide clear reports to users as to their performance and resist the temptation to overload users with information simply because the technology exists to provide enormous amounts of data. Web based solutions are proving a popular tool to shorten lines of communication and reduce the cost of reporting by conventional mail. However, they are not always as easy to use as might be anticipated, particularly where hard copy is required. Laboratories should check that they have the processing and printing technology to reproduce the reports correctly before committing to such a system.
“Scheme organisers need to provide clear reports to users as to their performance”
Problems arise as a result of failures of the “manufacturer quality cycle” for reagents or the EQA material itself, outside the control of the schemes and their users. These can be method related (the result of a reagent manufacturing defect) or matrix related (the result of differences between the EQA material and human samples). Both users and schemes need to be aware of these pitfalls, which by their nature can take some time to become evident and longer to investigate and resolve. Method related changes are clinically important when the variation crosses decision cutoff points; hence the importance of schemes addressing performance in these areas. The performance in some assays, particularly immunochemical methods, has given sufficient concern for guidelines to be drawn up whereby a poorly performing method can be drawn to the attention of The Medicines and Healthcare Products Regulatory Agency. The guidelines have been ratified by the JWG for further promulgation (R Cramb, personal communication, 2002). Problems can arise as a result of single assay manufacturing failure or problems inherent in the assay technology used. These second types of problem are particularly intractable, which is not surprising in view of the enormous sums invested in developments that manufacturers are unwilling to recognise as flawed after they reach the commercial market.
Matrix issues may not just be limited to the difference between circulated EQA and human material, but may be dependent on the interaction between the EQA sample and its container. Thus, unexpectedly low salicylate results in one scheme were found to result from interactions between the EQA material and the O ring sealing the cap of the container in which the material was circulated (D Wardley, personal communication, 2003).
Schemes should agree criteria for poor performance for ratification by the NQAAP. The application of these criteria will always be subject to judgement based on assay characteristics and clinical usefulness. For example, there is an obvious requirement for the criteria for sodium to be tighter than—for example, those for vitamin E. Automation has improved the performance of assays over the years. Wider limits are to be expected with newer assays or where multistep methodology, often not lending itself to automation, is inherently less accurate and precise.
One criterion is that of non-participation. A laboratory that fails to return results can clearly be held to be in breach of performance limits after failure to return the results of two or three successive distributions, dependent on the frequency of these distributions. Reasons for this can vary from staff illness or poor communication to equipment failure, and require attention from the scheme organiser at an early stage if the clinical service is not to be compromised. Migration between schemes is a recognised ploy for maintaining the semblance of acceptable performance (“we’ve only just joined, so that’s why our performance is substandard”). Clearly, in such cases performance will constantly be substandard, hence the existence of monitoring mechanisms at scheme organiser/NQAAP level for addressing these problems.
Finally, part of the activity of scheme organisers is to promote participation in schemes so that laboratories that offer a clinical service for any given analyte subscribe to the EQA scheme for that analyte where one exists, as outlined in the next paragraph.
Note that the traditional classification of operational aspects of this cycle into preanalytical, analytical, and postanalytical works well for the identification of technical problems, but less so for system failures.
Laboratories should review their repertoire regularly to ensure that their membership of EQA schemes continues to reflect both the breadth and depth of their repertoire. They should document that they are members of schemes relevant to all their analytes where possible, and that the schemes relate to the relevant clinical scenario. Clearly, successful performance in a “general biochemistry” bilirubin scheme based on adult concentrations and methodology cannot be extrapolated to paediatric bilirubin measurement. There are many other such examples. Point of care testing (POCT) facilities that are managed by laboratories are (and should be) subject to the same performance standards as their host laboratory when the same methodology is used. Even where POCT and laboratory methodologies differ there may be an argument for applying the same performance criteria to achieve cross method comparability where the use of the POCT relates to a crucial clinical situation—for example, with regard to ward based bilirubinometers (F Mackenzie, personal communication, 2003). Some schemes offer discounted membership for POCT where the host laboratory is already a member of the scheme in question.
Some analytes do not have their own EQA scheme. Such analytes are either under development and have not yet been widely adopted, or generate only minority interest for whatever reason. There may also be technical problems associated with the design of a scheme where it is difficult to manufacture appropriate material for distribution—for example, where faecal analysis is involved. Under these circumstances, it is particularly important to subject the assay to internal quality control, preferably using commercially marketed material or, if unavailable, pooled patient material. Laboratories should be careful to adhere to local and national guidance concerning the retention of surplus patient samples for quality control pools. 3
Look at the postmark on the packaging and note any postal delay. Delays can occur in internal hospital post, public mail, and courier services.
“Laboratories should review their repertoire regularly to ensure that their membership of EQA schemes continues to reflect both the breadth and depth of their repertoire”
The specimen needs correct processing on receipt, care being taken to comply with instructions for reconstitution (where applicable), storage, and deadlines for analysis and reporting.
Analysis should be undertaken as for a routine specimen. This is sometimes difficult, but specimens should not be treated as any more special than they need to be. It is not unknown for senior staff to become very possessive of EQA analysis to the extent that some insist on analysing the specimens themselves wherever possible. This personalises poor performance issues, making them more difficult to resolve.
When considering the possibility of a “laboratory blunder” in contributing to poor EQA performance, consider all the steps in specimen handling, recording, and reporting. Northam succinctly grouped these into “blunder prone situations, analyses, or persons”. 4 Given the myriad possibilities, it is not surprising that reasons for random errors are not always identified. Systems failures under pressure will produce blunders in the same way that they did when first studied over 30 years ago, 5 but the blunder rate (as opposed to variety) should decrease with increasing automation. 6
The contribution of identifiable analytical error to poor EQA performance is often concentration dependent, particularly where concentrations of analyte necessitate dilutions. Although some current pieces of equipment automate these steps, wherever there is operator intervention there is the possibility of error. As mentioned above, senior staff involvement is not always a guarantee of quality or relevant experience. Other common errors are transcription errors in reporting either by paper or web based methods, and delays in analysis or return of reports leading to non-inclusion of results.
One important contribution of collated EQA information to the laboratory quality cycle is the overview it gives the user of assay performance across methods, giving the user “best buy” information when new equipment or a new method is under consideration. Satisfactory performance in EQA schemes should be an essential criterion in documents prepared for tenders for new or replacement equipment.
Breach of the criterion of non-participation typically occurs in respect of specialised assays where batches are performed infrequently: the EQA specimen arrives at the wrong time and fails to make the batch. However, non-participation can hide more serious problems if it occurs because the staff member responsible has failed either to analyse or report the results on an EQA distribution. Staff illness and equipment failure can also be responsible. If communications are poor within the laboratory, the situation may only become evident when the scheme organiser draws it to the attention of the NQAAP.
Changes in methods inevitably give rise to changes in performance, easily recognisable as a “step” in graphically presented information. Although this would hopefully reflect an improvement, failure to inform the scheme organiser of a change in method, or filing incorrect method information will have the opposite effect. Comparison of EQA with internal quality control (IQC) results will often point to the solution. EQA failure where IQC is satisfactory can indicate either a method related change in performance or a “one off” occurrence in the processing of a particular EQA specimen.
Failure to progress round the cycle can be the result of all these factors, especially poor internal resolution of EQA because of failure to identify the problem using a systematic approach. 7 Inappropriate calibration, typified by the use of “fudge factors”, intercepts on standard curves introduced without sufficient evidence for doing so, and other changes without appropriate evidence are predictable sources of problems.
The principal functions of an NQAAP involve elements of both professional self regulation (in respect of scrutiny of laboratory performance) and public accountability (in respect of poor performance).
The panel relies on detailed scheme organiser reports. The quality of the information exchange between the scheme organiser and the panel in this cycle is vital to the outcome.
Assay performance has to fail to meet established performance criteria before the panel becomes involved. Where reporting arrangements are robust and these criteria are clear and explicit, the panel is able to ensure a continued quality service to patients. Sometimes this involves communication between panels, particularly where a single diagnostic procedure involves more than one discipline, such as establishing best practice for the examination of cerebrospinal fluid for haem pigments, which is performed in some microbiology and chemical pathology departments after lumbar puncture. 8, 9
“Assay performance has to fail to meet established performance criteria before the panel becomes involved”
EQA in the UK has from its outset been based on educational support from scheme organisers for users with problems, made easier by the confidentiality afforded by the relationships between the laboratories, the schemes, and the panel. Professional misconduct and negligence, particularly in pathology, has in recent years attracted unprecedented media interest and government scrutiny. As a result of this, the challenge to professional self regulation is to increase the transparency of EQA procedures while keeping laboratories involved. Although very few of the schemes are run as profit centres, they are all businesses that have an understandable interest in preserving and expanding their customer base to ensure continued viability. One way of doing this might be to shield the panel from individual problems. This runs counter to current trends and the result would be inadequate panel input into problems, with an increased risk of their being prolonged and a concomitant risk to patient care.
For their part, the panels have to respond to scheme organisers comprehensively and in an appropriate time frame. An important panel function is to spot method failure across schemes and laboratories measuring the same analyte by the same method. The panel is in a unique position to do this, and this can serve as an early warning system for method related changes in performance
Several challenges await clinical biochemists in the way they interact with external quality assurance systems. A current pilot interpretive scheme allows for collective participation in addition to individual scheme membership. The desire outside medicine for measures of clinical performance is such that the lack of compulsory individual participation in such schemes may be challenged in the future. Scheme organisers and participants will need to satisfy themselves that the systems of assessment accurately reflect laboratory practice. The reluctance of some to participate in the pilot may reflect concerns that this is not the case. The interpretation of results is only a small part of the way in which senior staff in laboratories interact with clinicians, both in primary and secondary care, in the management of individual cases. In addition to interpretative comments on reports, this includes telephone consultations, access to case notes, and ward visits, which provide a full clinical picture on which diagnosis and treatment plans are based. The challenge to interpretative schemes is to incorporate these aspects. Criteria for poor performance will need to be sufficiently robust to survive legal challenges, and may require contact with bodies other than the NQAAP. With histopathology there is a reporting relationship with the professional performance panel of the Royal College of Pathologists in respect of individual substandard performance. This is separate from any other relationship that histopathology schemes may have with the NQAAP for histopathology.
Operational relationships between scheme users, organisers, and external surveillance bodies will become more web based to improve communications and reduce reliance on paper records. The data will have to be interpreted and stored in a secure manner.
Although laboratory accreditation is now compulsory for pathology laboratories in England, 10 there is currently no licensing of laboratories, as is the case in some other countries, where licensing may be subject to satisfactory EQA performance. However, given recent adverse publicity suffered by laboratory medicine as a result of controversies over retained organs and high profile laboratory errors, laboratories should be prepared for more of a “command and control” culture in future. Disease oriented task groups such as The National Screening Committee’s “Informed Choice Programme” should incorporate the quality assurance community in their deliberations when they need to consider interlaboratory comparability.
Many within the EQA community have come from the service culture of the National Health Service, but this community is now a mixture of commercial, not for profit, and voluntary bodies. Because of this, there are vested interests in addition to declared roles. This introduces tensions in the pursuit of best practice where vested interests come into conflict with service culture.
Best practice is a combination of applied science and process management: as we observe and reflect on our own performance as a profession, we need to assure ourselves that the way we pursue this withstands external scrutiny.
1. Burnett D . Quality assurance and evaluation. In: Understanding accreditation in laboratory medicine . London: ACB Venture Publications, 1996:218–47.
2. Donabedian A . Evaluating the quality of medical care. Millbank Memorial Fund Quarterly 1966; 44 :166–206. [PubMed] [Google Scholar]
3. The use of human organs and tissue. An interim statement, 2003. Available at: http://www.doh.gov.uk/tissue/interimstatement.pdf (accessed 11 September 2003).
4. Northam BE. Whither automation? Ann Clin Biochem 1981; 18 :189–99. [PubMed] [Google Scholar]5. McSwiney RR, Woodrow DA. Types of errors within a clinical laboratory. J Med Lab Technol 1969; 26 :340–6. [PubMed] [Google Scholar]
6. Chambers AM, Elder J, O’Reilly DS. The blunder rate in a clinical biochemistry service. Ann Clin Biochem 1986; 23 :470–3. [PubMed] [Google Scholar]
7. Westgard JO, Burnett RW, Bowers GN. Quality management science in clinical chemistry: a dynamic framework for continuous improvement of quality. Clin Chem 1990; 36 :1712–16. [PubMed] [Google Scholar]
8. Cruickshank AM. ACP Best Practice No 166: CSF spectrophotometry in the diagnosis of subarachnoid haemorrhage. J Clin Pathol 2001; 54 :827–30. [PMC free article] [PubMed] [Google Scholar]
9. Beetham R, Fahie-Wilson MN, Holbrook I, et al. CSF spectrophotometry in the diagnosis of subarachnoid haemorrhage. J Clin Pathol 2002; 55 :479–80. [PMC free article] [PubMed] [Google Scholar]
