Year : 2022 | Volume
: 16 | Issue : 1 | Page : 1--3
American Board of Anesthesiology and Accountability (or lack of): What the rest of the world can learn?
Clinical Associate Professor, Department of Anesthesiology and Critical Care Medicine, Perelman School of Medicine, Hospital of the University of Pennsylvania, 680 Dulles, 3400 Spruce Street, Philadelphia, PA, USA
680 Dulles, 3400 Spruce Street, Philadelphia, PA
|How to cite this article:|
Goudra B. American Board of Anesthesiology and Accountability (or lack of): What the rest of the world can learn?.Saudi J Anaesth 2022;16:1-3
|How to cite this URL:|
Goudra B. American Board of Anesthesiology and Accountability (or lack of): What the rest of the world can learn?. Saudi J Anaesth [serial online] 2022 [cited 2022 Aug 16 ];16:1-3
Available from: https://www.saudija.org/text.asp?2022/16/1/1/334756
Recently, I conducted a nationwide questionnaire survey to understand the perspectives of anesthesiology residents in the United States with regards to the recent change in their primary certification examination. The questions were direct, explicit, and addressed an important aspect of their training. Broadly, the questions aimed to understand how the residents' felt about the recently introduced additional layer of testing called “Objective Structured Clinical Examination” (OSCE) in the final certification examination conducted by the American Board of Anesthesiology (ABA). Since its introduction, its conduct, validity, and relevance is questioned; however, this survey is the first of its kind. Before I tease out the survey and its potential learning opportunities to the larger world, for the benefit of international readers, let me lay out the actual conduct of the Structured Oral Examination (SOE), more commonly referred to as Viva Voce in the rest of the world and the OSCE.
The format of the Viva Voce is very similar across the world. Typically, two or more examiners ask similar questions. The answers are promptly analyzed and the performance is scored either independently or by mutual discussion among the examiners. The examiners are not supposed to talk to each other or get influenced by others' body language and opinions while the questioning is on. The OSCE has different formats in different countries and most countries do not have this component.
In the United States, a candidate encounters many standardized actors playing the roles of patient, surgeon, anesthesiologist, and colleague interested in performing a clinical quality improvement initiative. The tasks are “disciplining a colleague,” “consenting a patient for a given procedure,” or “appeasing a surgeon,” etc. The ABA videotapes the OSCE performance and the examiner watches the video to allocate a score; however, the candidates have no access to this video. Clearly, all these stations are subjective as evidenced by the opinion of over 700 of about 4200 residents in the questionnaire survey. Factors such as gender, race, accent, language, appearance, nationality, presumed political affiliation, and age are perceived to influence the scoring. There are two less subjective stations, one is a demonstration of a nerve trunk, nerve, or a vessel using ultrasound on a standardized patient and the other is an interpretation of electrocardiogram (ECG or EKG), transesophageal echocardiography (TEE), and changes appearing on an intraoperative multisystem monitor. There is no appeal mechanism for either OSCE or SOE, only rescoring; a simple mathematical addition of numbers for $250.00.
The United Kingdom was the first country to introduce the OSCE for their primary examination in the 1990s, conducted very differently; however, poorly validated. They have an SOE too, referred to as Viva Voce. However, they are more objective with the same root and secondary questions posed to all the candidates. In addition, United Kingdom has an appeal process. A candidate is allowed to make an appointment and sit with the Royal College of Anesthetists representative who will discuss reasons for failure in person and in detail. They do not charge any fees for that service.
India does not have a mandated national exam for specialty eligibility certification. Candidates enroll in a training program (much like the USA) and appear for a qualifying university-sponsored examination. There is little transparency and no appeal process. Bias is extremely common in viva voce. As a result, it is not surprising that OSCE was considered a better evaluation tool when compared with the conventional examination. It was also agreed that OSCE is easier to pass than the conventional method and 29 (82.5%) of the 35 candidates asked commented that the degree of emotional stress is less in OSCE than traditional methods. Additionally, the candidates do not fail the entire certification for failure in one or two sections of OSCE that carry limited weight in the assessment scale. Moreover, OSCE is also truly objective and has very little or no involvement of humans.
Importantly, there is no evidence to suggest that ABA board certification improves patient outcomes. The demerits of the only published paper are well-known, Briefly, Silber et al., compared 8,894 patients anesthetized by midcareer anesthesiologists, 11–25 years from medical school graduation, who lacked board certification with all others (much bigger cohort, about 61,000), which itself is incorrect. In this retrospective study, the outcome observed was death rate within 30 days of admission, in-hospital complication rate, and the failure-to-rescue rate (defined as the rate of death after complications), all of which may not have much to do with the anesthesia delivery. The risk factors for 30-day postoperative mortality are many and include Higher American Society of Anesthesiologists' physical status scores, extremes of age, emergencies, perioperative adverse events, and postoperative Intensive Care Unit admission. Even though, Silber et al. attempted to adjust for many such variables, the only way one might find a correlation between anesthesia and subsequent 30-day mortality is by demonstrating an anesthesia-related event that was contributory. In addition, as the authors themselves mention, nonboard certified anesthesiologists worked in less-desirable hospitals that were ill-equipped. This likely explains higher 30-day mortality than anesthesia care.
Another study performed by the current ABA directors used the license disciplinary action as a pointer of written and oral specialty certification examination effectiveness. Although negligence or incompetence are the most common causes (yet, only 34% of the total), factors such as alcohol and substance abuse, inappropriate prescribing practices, inappropriate contact with patients, and fraud are responsible for the majority of such actions and these cannot be foreseen in the OSCE or structured oral examination (SOE)., Nevertheless, anesthesia residents do not take board examinations to decrease disciplinary proceedings from licensing authorities.
In any case, the responses of over 90% of 710 anesthesiology residents who decided to take part in this national survey representing 42 states clearly indicate that ABA's OSCE is discriminatory and influenced by factors such as race, religion, language, political affiliation, gender, accent, language, etc. These factors have nothing to do with knowledge, professionalism, and communication. In addition, the ABA refuses to entertain any questions or reevaluations. There is no appeal process. In other words, there is no transparency. Wikipedia describes Institutional racism, also known as systemic racism, as a form of racism that is embedded through the laws and regulations within a society or an organization. ABA is an organization that follows a closed-system approach. It consists of directors and staff who make rules, change policies, conduct exams, and introduce more exams. Although it is presumed to operate under the umbrella of the American Board of Medical Specialties (ABMS) and has the blessing of the Accreditation Council for Graduate Medical Education (ACGME), it is not answerable to either of them. There is no ombudsperson and there is no authority to which one can make complaints about any noticeable organizational failures. The exams and their validation is a closely guarded secret. The candidates are supposed to take their word that these exams are fair. This is a kind of authoritarianism under the cover of their laws and regulations.
The results of this national questionnaire survey should be an eyeopener to not only the ABA but also other similar testing and regulatory bodies across the world. In the context of OSCE, it should be noted that, recently, the Federation of State Medical Boards and National Board of Medical Examiners (NBME), cosponsors of the United States Medical Licensing Examination, announced the discontinuation of work to relaunch a modified step 2 clinical skills examination (an OSCE equivalent). Nonetheless, the examination bodies have a responsibility to demonstrate that their certification results in anesthesia service providers who provide better care (than that provided by those without such a certification), which, in turn, leads to improved patient outcomes. The examining bodies must videotape all components of the examination and willing to discuss it with the candidates if they wish so. They should not oppose the idea of candidates making the video public. After all, the idea is to improve patient care. If a candidate benefits from watching such videos and can learn from other's mistakes and successes that should be encouraged and not secretly guarded.
In addition, any research into the value and validity of these exams must be performed by independent bodies bereft of vested interest. We should all remember that many careers were ruined by the USMLE clinical skill assessment component that was recently abolished. There was sufficient “evidence” to support its introduction and its continuation. Similarly, many anesthesiology residency aspirants were rejected solely based on their step 1 scores. I am sure there was “evidence” to support the practice of using step 1 scores for interview shortlisting.
The United States is supposed to be a beacon of hope for millions around the world and an example of fairness and transparency. In this context, the ABA could be seen as an oppressor and the candidates unfairly treated as oppressed. In this “oppressor-oppressed” notion, one side always benefits; whether it is male gender (in gender discrimination), upper-caste (in caste discrimination), dominant religion (in religious persecution), and so on. No doubt, these issues are sensitive; however, it does not mean that they should not be discussed. In the context of the ABA's OSCE, as a result of their polished behavior, having attended a better school system, raised in a nicer neighborhood, instilled a better sense of dress code, etc., certain group of candidates is bound to benefit from the existing system. However, these factors have nothing to do with anesthesia delivery or its safety. This is a unique feature of the US demography and unequal wealth distribution. As the ABA directors themselves state, they judge based on the “expectation that minimally competent candidates would, on average, 'often' demonstrate the qualities expected of an ABA Diplomate”, of course, such an approach has historical precedent, since these qualities are never explicitly stated. Sadly, medical and other health professionals were 'the staunchest supporters of the Nazi regime' and I certainly hope that the ABA does not describe its diplomates in the context of “master race” envisioned by the Third Reich. No examination can be vaguer and yet described as “objective”. Similar issues might exist in other parts of the world for different reasons. As a result, it is important for all the certifying bodies to eliminate all examination components that have subjectivity element, however small they are.sw These include OSCE and SOE. If not, they must videotape and discuss the reason for failure with the candidate. The murder of George Flyod would not have come to light without the video that was captured by the alert public bystanders., Without transparency, any testing system will lose its relevance.
World over, hospitals and health systems rely on the American Board of Anesthesiology as a mark of quality. I am sure that the ABA and all similar bodies will establish transparency and eliminate bias to reassert themselves as impartial examining and certification authorities.
|1||Goudra B, Guthal A. US residents' perspectives on the introduction, conduct, and value of American Board of Anesthesiology's objective structured clinical examination-results of the 1st nationwide questionnaire survey. Anesth Essays Res 2021;15:87-100.|
|2||Goudra B. Objective structured clinical examination-Are they truly objective? Anesth Analg 2021;133:e3-5.|
|3||McColl DJ, Mendonca C. The primary FRCA OSCE–an accurate measure of communication skills?. Anaesthesia. 2009;64:106.|
|4||Jindal P, Khurana G. The opinion of post graduate students on objective structured clinical examination in anaesthesiology: A preliminary report. Indian J Anaesth 2016;60:168-73.|
|5||Silber JH, Kennedy SK, Even-Shoshan O, Chen W, Mosher RE, Showan AM, et al. Anesthesiologist board certification and patient outcomes. Anesthesiology 2002;96:1044-52.|
|6||Fecho K, Lunney AT, Boysen PG, Rock P, Norfleet EA. Postoperative mortality after inpatient surgery: Incidence and risk factors. Ther Clin Risk Manag 2008;4:681-8.|
|7||Zhou Y, Sun H, Culley DJ, Young A, Harman AE, Warner DO. Effectiveness of written and oral specialty certification examinations to predict actions against the medical licenses of anesthesiologists. Anesthesiology 2017;126:1171-9.|
|8||Morrison J. Physicians disciplined by a State medical board. JAMA 1998;279:1889-93.|
|9||Institutional racism. In: Wikipedia. 2021. Available from: https://en.wikipedia.org/w/index. php? title=Institutional_racism and oldid=1030401560. [Last accessed on 2021 Jun 26].|
|10||United States Medical Licensing Examination,Announcements. . Available from: https://www.usmle.org/announcements/. [Last accessed on 2021 Jan 28].|
|11||Warner DO, Lien CA, Wang T, Zhou Y, Isaak RS, Peterson-Layne C, et al. In response. Anesth Analg 2021;133:e5-7.|
|12||Bhopal R. Hitler on race and health in Mein Kampf: A stimulus to anti-racism in the health professions. Diversity & Equality in Health and Care [Internet]. 2005;2. Available from: https://diversityhealthcare.imedpub.com/abstract/hitler-on-raceandhealth-in-mein-kampf-a-stimulus-to-antiracism-in-the-health-professions-2567.html;. [Last accessed onn 2021 Jul 24].|
|13||Dixon PJ, Dundes L. Exceptional injustice: Facebook as a reflection of race- and gender-based narratives following the death of george floyd. Social Sci 2020;9:231.|
|14||Barrie C. Searching Racism after George Floyd. Socius 2020;6:2378023120971507|