Determination of minimally clinically important differences for JOABPEQ measure after discectomy in patients with lumbar disc herniation

Introduction

Lumbar disc herniation (LDH) is caused by a change in the structure of a spinal disc. This can apply pressure on spinal nerve roots, causing pain in the lower back or legs, numbness, tingling or weakness in the foot (1). A lumbar herniated disc often requires care by spinal surgery and healthcare professionals, and may include physical therapy, medications, or possibly surgical procedures (2).

As far as surgery for LDH is concerned, however, the definition of treatment success rate depends on how success is evaluated (3). Outcomes were generally assessed based on surgeons’ subjective views and evaluated using imaging studies during patients’ follow-up visits. However, surgeons’ perspectives and results of imaging examinations frequently do not correlate with patient satisfaction (4). On the other hand, it was shown that the best measurement of treatment quality should be the patient’s opinion of the results by patient-reported outcomes (PRO) tools. Therefore, in recent decades, several questionnaires such as the Japanese Orthopedic Association Back Pain Evaluation Questionnaire (JOABPEQ) have been developed and validated for the clinical and functional evaluation of spinal treatment outcomes (5,6). To determine whether a medical intervention improves perceived outcomes in patients, the minimally clinically important differences (MCID), was first employed by Jaeschke et al. in 1989 (7). It is the smallest change in a treatment outcome which patients perceive as beneficial and lead the clinician to consider a change in treatment (8,9). In the other word, the MCID can be used as a threshold to detect change to indicate response to an intervention. The MCID is now a well-established concept and used as a reliable method for evaluating an intervention (8,9). The JOABPEQ has been shown to be a more accurate outcome measure, reliable, and valid scale for evaluating patients with low back pain as LDH (6). Although sensitivity to change has been demonstrated (6), however, no study has yet determined the MCID of this questionnaire in patients with LDH. In other words, the question that remains is: what is a MCID for the JOABPEQ measure to assess surgical success in patients with LDH? Therefore, the aim of this prospective study was to determine the MCID of the JOABPEQ in patients with LDH.

Methods

Patients and data collection

From March 2013 and August 2016, a sample of patients with LDH at our institution in Tehran, Iran consecutively, was included in this prospective study. Data to evaluate the outcome for all patients who were to undergo discectomy with a single-level disc herniation were collected and patients were assessed preoperatively and at least 1 year postoperatively. The diagnosis of LDH was made on the basis of clinical and radiographic evidence. All participants underwent a complete clinical examination for LDH including an evaluation of clinical symptoms and clinical examination, and imaging studies—including plain radiography, computed tomography (CT) and magnetic resonance imaging (MRI) of the lumbar spine. In all cases, more than one spine surgeon confirmed the diagnosis, and surgery was performed by experienced surgeons. Patients were asked to fill out preoperative and follow-up questionnaires and to undergo follow-up examinations at last follow-up. There were no restrictions on patient selection with regard to types of LDH, age or other characteristics. Patients who had previous back surgeries and spinal anomalies were excluded. Characteristics including age, gender and body mass index (BMI), visual analog scale (VAS) associated with leg pain (mm) and VAS associated with back pain (mm) were determined. Type of herniation, smoking histories and the duration of symptoms (in months), were evaluated. The time-point for post-operative assessment was 1 year after surgery.

Operative procedure

Standard open lumbar discectomy was carried out to manage LDH in patients who have persistent symptoms of the condition that do not improve with a conservative treatment (10). The sample size was calculated based on 20% failure in surgical success.

Measures

(I) The Japanese Orthopedic Association Back Pain Evaluation Questionnaire (JOAPEQ) score: The JOABPEQ for assessing low back pain was designed. It is a disease-specific tool and contains 25 items tapping into five subscales: social function (four items), mental health (seven items), lumbar function (six items), walking ability (five items), and low back pain (four items). The score for each subscale range from 0 to 100, with higher scores indicating better conditions (6). The JOABPEQ subscale scores were calculated at baseline and 1-year after surgery

(II) The Iranian version of Oswestry Disability Index (ODI) (version 2) was used to assess functionality. The ODI contains 10 items and its score range from 0 to 50, with higher scores indicating a worse condition. The psychometric properties of the Iranian version of questionnaire are well documented (11). The ODI score was measured at admission and at last follow-up (1 year after surgery). A MCID is a threshold used to calculate the effect of clinical treatments. In this study the MCID value that is calculated is specific to the LDH patients. Surgical success was defined as a 13-point improvement from the baseline ODI scores (12).

The clinical consensus procedure was employed as a standard method used for assessing of surgical success (13). Spine surgeons conducted clinical and radiographic examinations pre- and post-operatively. The inclusion criteria comprised: specialization in spine surgery, working in these specialties for at least 15 years, active daily practice in the treatment of LDH, having treated at least 100 patients with LDH a year, having authored five or more publications on LDH.

Spine surgeons conducted clinical and radiographic examinations pre- and post-operatively. In addition, if necessary, the final assessment of surgical success is considered using a 13-point improvement from the baseline ODI scores (12). Finally, the dichotomized assessment of satisfied versus unsatisfied was obtained.

Statistical analysis

The score difference between pre- and post-intervention was calculated for each JOABPEQ subscale measure. The change score for all subscale measure were calculated by subtracting the post-intervention subscale scores from the pre-intervention subscale scores. Therefore, a positive subscale score indicated an improvement and a negative subscale score indicated deterioration (14).

There are many ways to determine the MCID. One method is the anchor-based approach, also known as the anchor, is used to compare the change observed in a PRO measure (15). In this study, the anchor was the clinical consensus diagnosis provided by physicians on the basis of Delphi method, and the PRO measure was the JOABPEQ subscales. Two statistical methods were used to calculate MCID after surgery using the anchor-based approach: (I) The “average change” approach defines MCID as the average change score seen in the cohort defined as responders; (II) The “receiver operating characteristic (ROC) curve” approach defines MCID as the change value that provides the greatest sensitivity and/or specificity for a positive response (12).

In the ROC curve analysis, each JOABPEQ subscale outcome measure is considered as a diagnostic test and the global rating scale is considered as a reference standard (16). In this analysis, the dichotomized assessment (improved versus unimproved) was used to determine the sensitivity and specificity of change scores of each outcome measure (17). Sensitivity is the ability of an outcome measure to detect a clinical change when it actually exists (i.e., true positive results) while specificity is the ability of an outcome measure to detect the absence of a clinical change when it actually does not exist (i.e., true negative results). The ROC curve analysis was performed to identify the optimal cut-off point of JOABPEQ subscales (at which sensitivity and specificity would be maximal) for assessing surgical success. Areas under the curve (AUC) were calculated as measures of the accuracy of the tests. The AUC is a measure of discrimination; a model with a high area under the ROC curve suggests that the model is able to accurately predict the rate of an observation’s response. An AUC of 0.50 was considered no discrimination (0.7≤ AUC <0.8 as acceptable discrimination, 0.8≤ AUC <0.9 excellent discrimination, and more than 0.9 outstanding discrimination) (18). All statistical analyses were performed using the PASW Statistics 18 version 18 (SPSS, Inc., 2009, Chicago, IL, USA) and P<0.05 was considered to indicate a statistically significant difference.

Results

A total of 133 patients undergoing elective LDH were included in the study; 4 were excluded because of deficient follow-up results, and 2 patients due to recurrent disk herniations. All 127 patients were available for follow-up at 1 year. Mean age (SD) of patients in the cohort was 51.2 (9.4) years, and 68 (53.5%) were male. Table 1 summarizes the patient demographic characteristics and their scores on the VAS score, and the ODI.

Table 1 Demographic data and preoperative status of patients with lumbar disc herniation (n=127)
Full table

One year after surgery, a significant improvement was observed according to clinical consensus procedure (P<0.001), JOABPEQ subscales (P<0.002 for all of subscales) and ODI. Based on the ODI, post-surgical success was 81.9% (n=104). Mean improvement in the ODI was 19.6 (SD =9.8) and statistically was significant (P<0.001). No significant differences were observed for post-surgical success between levels of LDH.

Considering average change approach, the results showed that the MCID for the JOABPEQ subscales were: low back pain, more than 22.4; lumbar function, more than 24.2; walking ability, more than 27.9; social life function, more than 17.1; mental health, more than 14.8 points. The findings are shown in Table 2.

Table 2 Mean (SD) of pre-operative, post- operative and change score for JOABPEQ questionnaire
Full table

According to the ROC analysis, the optimal cut-off value of JOABPEQ subscales to assess surgical success was measured: a score change of 19.1 points on the JOABPEQ low back pain subscale, 21.3 points on the JOABPEQ lumbar function subscale, 24.5 points on the JOABPEQ walking ability subscale, 14.3 points on the JOABPEQ social life function subscale, and 12.8 points on the JOABPEQ mental health subscale should be considered a MCID. The cut-off points, sensitivity, specificity, respective standard error (SE), and areas under the ROC curves with confidence interval (CI) for JOABPEQ subscales are presented in Table 3. The area under the ROC curve was greater than 0.70 for all five subscales. The findings are shown in Figure 1 and Table 3.

Table 3 Cut-off point, sensitivity, specificity, respective SE, and AUC for JOABPEQ questionnaire subscales
Full table

Figure 1 ROC curve for the JOABPEQ subscales to assess surgical success in patients with LDH. The optimal cut-off for maximum sensitivity and specificity were: low back pain, 19.1; lumbar function, 21.3; walking ability, 24.5; social life function, 14.3; and mental health, 12.8. ROC, receiver operating characteristic; LDH, lumbar disc herniation; JOABPEQ, Japanese Orthopedic Association Back Pain Evaluation Questionnaire.

No discectomy case was observed to have a missed level surgery. In 2 cases (1.6%), dural laceration occurred during surgery which were repaired and no one showed CSF leakage or meningitis. No mortality rate was observed due to surgery.

Discussion

An understanding of the MCID of a scale is vital for the planning of new clinical assessments and interpretation of results. This is the first study to characterize the MCID for the JOABPEQ in patients with LDH to assess surgical success. These results are the smallest change patients perceive as a meaningful improvement. However, there is a statistical and clinical difference in use of the MCID. Hence, additional studies are needed to validate the threshold values for MCID to quantify the meaningful clinical benefit in these cases (19).

In the present study, the two values for JOABPEQ subscales based on MCID calculation methods were introduced for patients with LDH undergoing surgery to allow for the determination of clinically meaningful responses. However, it is difficult to choose the best method from the two methods, because an optimal MCID threshold value or best MCID calculation method for a specific surgery or patient population has not yet been established. Albeit, some researchers have suggested that the most efficient way to determine whether the health status of patients undergoes a true change following intervention is the ROC analysis (20). Overall, it seems that the MCID values achieved from the ROC analysis provide valuable information for the researchers and clinicians to decision-making process about the identification of a true change in health status of patients with LDH (20). On the other hand, intervention satisfaction and acceptability are multifactorial and influence on the MCID value, and it is also the definition of surgical success or failure is controversial between patients and clinicians. However, the main strength of this study is the definition of surgical success as anchor based on clinical consensus method to assess surgical success (MCID).

In this paper, a JOABPEQ cut-off value was determined to assess surgical success in patients with LDH by ROC approach and average change approach analysis; to our knowledge, this is the first attempt to determine the cut-off value in this manner. We believe that this is a logical and reasonable way to define the cut-off value for based on JOABPEQ tool. To date, some of studies have assessed MCID thresholds in patients undergoing LDH intervention (surgery). Solberg et al. (21), measured MCID thresholds for patients with disc herniation, who were undergoing surgery. They reported MCID thresholds of 20 for ODI, 2.5 for numerical pain scale (NRS) back, and 3.5 for NRS leg and 0.3 for Euroqol (EQ-5D). Glassman et al. (22) reported a cutoff for success of 19 ODI points in patients who were operated with lumbar spine arthrodesis, which is very similar to Solberg et al. (21) results. Another report by Copay et al. (23) had determined lower estimates of 13 points for the ODI, 1.2 points for NRS back pain scale, and 1.6 points for NRS leg pain scale. However, they used a mixed patient sample involving different lumbar spine surgery procedures. However, it is difficult to make comparisons with our current study because difference in PRO measures, difference in satisfaction anchor measurements and differences in study design and patient population. In addition, our study includes a homogeneous patient population (patients with LDH) undergoing the same procedure. The lack of a standard definition of an anchor such as Delphi method, improvement from the baseline PRO measures, and return to work caused a change to calculate on MCID. In other word, difference in anchor can lead to variance in the MCID thresholds calculated. Hence, the best anchor for MCID calculation is required to assess.

In user’s guide of JOABPEQ, to evaluate a therapeutic effect of a treatment, a 20-point increase is judged “effective” (24). However, the MCID value was not specific to the populations studied. In this study MCID was calculated exclusively for LDH patients. One might inquire about the main weakness of the paper that is trying to adapt a score for back pain to LHD outcome, however, according to literature, there is no difference in final outcome for patients with LHD treated with or without surgery, so the research for minimal changes is a nonsense. However, authors recall an important everlasting question in spine surgery: to determine an exact evaluation method for the success rate of LDH surgery. Moving health care to non-invasive methods, this is a sensitive and significant theme for spine surgeons. This type of studies often fails in a review because of the very heterogeneous population investigated and the subjectivity of the measurements. In this study the authors made a trial to get an exact tool to evaluate the success rate of open discectomy in LDH patients. In my opinion the methods used in this work are respective and the interpretation of the results is correct.

This study has some important limitations. First, our study suggests that the MCID for the five domains of JOABPEQ can be used to detect surgical success in patients with LDH. However, our findings do not imply that one cannot use the MCID value to evaluate improvement in these cases. Because, at present, there are a number of different methods (as many as 9 methods) to obtain an MCID, as there are a number of different factors as gender, age, detailed medical history, and general health, that can influence the MCID value, and this has led to or resulted in a number of methodological or interpretation problems (25). Second, the cases were not subdivided into subgroups based on surgical technique due to limited number of patients. Third, the MCID value is specific to the populations studied and perhaps different in other populations due to difference in characteristic of cases. Fourth, due to the lack of a true gold standard for assessing the patient surgical success, certain cases may have been incorrectly classified. Hence, a standardized method for assessment of successful outcome is needed. Henceforth, further studies with larger datasets and longer follow-up are needed to evaluate the above issues.

Conclusions

The findings showed that the JOABPEQ tool is responsive outcome measures for evaluating change over time in patients with LDH. In addition, the MCID values obtained in this study will help the clinicians and researchers to assess surgical success in these patients.

Acknowledgements

The authors thank the staff of the Neurosurgery Unit at Imam-Hossain Hospital, Tehran, Iran.

Footnote

Conflicts of Interest: The authors have no conflicts of interest to declare.

Ethical Statement: The Ethics Committee of Shahid Beheshti University of Medical Sciences, Tehran, Iran, approved the study and written informed consent was obtained from all patients.

References

1. Fardon DF, Milette PC. Nomenclature and classification of lumbar disc pathology: recommendations of the Combined Task Forces of the North American Spine Society, American Society of Spine Radiology, and American Society of Neuroradiology. Spine 2001;26:E93-113. [Crossref] [PubMed]
2. Klineberg E, Ching A, Mundis G, et al. Diagnosis, treatment, and complications of adult lumbar disk herniation: evidence-based data for the healthcare professional. Instr Course Lect. 2015;64:405-16. [PubMed]
3. Chapman JR, Norvell DC, Hermsmeyer JT, et al. Evaluating common outcomes for measuring treatment success for chronic low back pain. Spine (Phila Pa 1976) 2011;36:S54-68. [Crossref] [PubMed]
4. Haefeli M, Elfering A, Aebi M, et al. What comprises a good outcome in spinal surgery? A preliminary survey among spine surgeons of the SSE and European spine patients. Eur Spine J 2008;17:104-16. [Crossref] [PubMed]
5. Carreon LY, Glassman SD, Djurasovic M, et al. Are preoperative health-related quality of life scores predictive of clinical outcomes after lumbar fusion? Spine (Phila Pa 1976) 2009;34:725-30. [Crossref] [PubMed]
6. Azimi P, Shahzadi S, Montazeri A. The Japanese Orthopedic Association Back Pain Evaluation Questionnaire (JOABPEQ) for low back disorders: a validation study from Iran. J Orthop Sci 2012;17:521-5. [Crossref] [PubMed]
7. Jaeschke R, Singer J, Guyatt GH. Measurement of health status: ascertaining the minimal clinically important difference. Control Clin Trials 1989;10:407-15. [Crossref] [PubMed]
8. Brozek JL, Guyatt GH, Schunemann HJ. How a wellgrounded minimal important difference can enhance transparency of labelling claims and improve interpretation of a patient reported outcome measure. Health Qual Life Outcomes 2006;4:69. [Crossref] [PubMed]
9. Revicki D, Hays RD, Cella D, et al. Recommended methods for determining responsiveness and minimally important differences for patient-reported outcomes. J Clin Epidemiol 2008;61:102-9. [Crossref] [PubMed]
10. Garg B, Nagraja UB, Jayaswal A. Microendoscopic versus open discectomy for lumbar disc herniation: a prospective randomised study. J Orthop Surg (Hong Kong) 2011;19:30-4. [Crossref] [PubMed]
11. Mousavi SJ, Parnianpour M, Mehdian H, et al. The oswestry disability index, the roland-morris disability questionnaire, and the quebec back pain disability scale: Translation and validation studies of the Iranian versions. Spine 2006;31:E454-9. [Crossref] [PubMed]
12. Copay AG, Glassman SD, Subach BR, et al. Minimum clinically important difference in lumbar spine surgery patients: a choice of methods using the Oswestry Disability Index, Medical Outcomes Study questionnaire Short Form 36, and pain scales. Spine J 2008;8:968-74. [Crossref] [PubMed]
13. Jaana M, Tamim H, Paré G, et al. Key IT management issues in hospitals: results of a Delphi study in Canada. Int J Med Inform 2011;80:828-40. [Crossref] [PubMed]
14. de Vet HC, Bouter LM, Bezemer PD, et al. Reproducibility and responsiveness of evaluative outcome measures. Int J Technol Assess Health Care 2001;17:479-87. [PubMed]
15. Copay AG, Subach BR, Glassman SD, et al. Understanding the minimum clinically important difference: a review of concepts and methods. Spine J 2007;7:541-6. [Crossref] [PubMed]
16. Ma C, Wu S, Xiao L, et al. Responsiveness of the Chinese version of the Oswestry disability index in patients with chronic low back pain. Eur Spine J 2011;20:475-81. [Crossref] [PubMed]
17. Irrgang JJ, Anderson AF, Boland AL, et al. Responsiveness of the International Knee Documentation Committee Subjective Knee Form. Am J Sports Med 2006;34:1567-73. [Crossref] [PubMed]
18. Fawcett T. An introduction to ROC analysis. Pattern Recognit Lett 2006;27:861-74. [Crossref]
19. Asher AL, Chotai S, Devin CJ, et al. Inadequacy of 3-month Oswestry Disability Index outcome for assessing individual longer-term patient experience after lumbar spine surgery. J Neurosurg Spine 2016;25:170-80. [Crossref] [PubMed]
20. Stratford PW, Binkley JM, Riddle DL. Health status measures: strategies and analytic methods for assessing change scores. Phys Ther 1996;76:1109-23. [Crossref] [PubMed]
21. Solberg T, Johnsen LG, Nygaard ØP, et al. Can we define success criteria for lumbar disc surgery?: estimates for a substantial amount of improvement in core outcome measures. Acta Orthop 2013;84:196-201. [Crossref] [PubMed]
22. Glassman SD, Copay AG, Berven SH, et al. Defining substantial clinical benefit following lumbar spine arthrodesis. J Bone Joint Surg (Am) 2008;90:1839-47. [Crossref] [PubMed]
23. Copay AG, Martin MM, Subach BR, et al. Assessment of spine surgery outcomes: nconsistency of change amongst outcome measurements. Spine J 2010;10:291-6. [Crossref] [PubMed]
24. Fukui M, Chiba K, Kawakami M, et al. JOA Back Pain Evaluation Questionnaire (JOABPEQ)/JOA Cervical Myelopathy Evaluation Questionnaire (JOACMEQ). The report on the development of revised versions. April 16, 2007. The Subcommittee of the Clinical Outcome Committee of the Japanese Orthopaedic Association on Low Back Pain and Cervical Myelopathy Evaluation. J Orthop Sci 2009;14:348-65. [Crossref] [PubMed]
25. Cook CE. Clinimetrics Corner: The Minimal Clinically Important Change Score (MCID): A Necessary Pretense. J Man Manip Ther 2008;16:E82-3. [Crossref] [PubMed]