Introduction
As patients with metastatic cancer live longer, the prevalence of brain metastases is increasing [1]. However, many clinical trials of systemic therapies continue to exclude patients with brain metastases or require that CNS lesions undergo local therapy, i.e. stereotactic radiosurgery (SRS), whole brain radiation, or surgery, prior to enrollment. Nevertheless, in recent years a few trials have been conducted to explore specifically the activity of targeted or immune therapies in patients with previously untreated brain metastases [2]. Still, not only does the number of studies in this area remain limited, but interpretation of their results has been complicated by the heterogeneous criteria used to evaluate the response of brain metastases to treatment [3].
For extra-cerebral tumors, standardized response criteria were first proposed by the World Health Organization in 1981 [4]. These have largely been replaced by the simpler response evaluation criteria in solid tumors (RECIST), which are now widely used in the assessment of most solid tumors [5, 6]. However, this has not been the case in neuro-oncology. Historically, the MacDonald criteria were used for the assessment of high-grade gliomas [7], and the response assessment in neuro-oncology (RANO) group recently proposed updates to these criteria [8].
Advertisement
Brain metastases have characteristics of both their extracranial primaries as well as primary brain tumors. Similar to extra-cerebral metastases, and unlike gliomas, these lesions are typically smaller, more regular in shape, and often multiple in number. However, like gliomas, symptomatology depends not only on lesion size but also location within the brain, with potential for significant effects on neurological function, and brain metastases similarly often require management using corticosteroids to control peri-lesional edema. Because of this overlap, both RECIST and RANO high-grade glioma (RANO-HGG) criteria have limitations when used to evaluate brain metastases response during clinical trials. Key issues include minimum size for target lesions being 10 mm, accuracy in definitions of response and progression based on percentage change in lesions, distinguishing true tumor progression from treatment effect, and incorporation of clinical status of the patient.
In view of the considerable variability among available response criteria and the concerns unique to the CNS, the RANO brain metastases (RANO-BM) working group recently proposed new response criteria for evaluating brain metastases in clinical trials [9]. These criteria were essentially formed from the merging of key features of RANO-HGG and RECIST 1.1 judged by the RANO-BM group to be the most relevant to brain metastasis response assessment. Thus, the RANO-BM criteria continue to use unidimensional measurement of tumor size (from RECIST 1.1), require measureable disease to be at least 10 mm in diameter (from both RECIST 1.1 and RANO-HGG), allow up to five lesions to be followed as targets (from RANO-HGG), and incorporate clinical status and steroid use when determining response or progression (from RANO-HGG). The main new feature of RANO-BM is the recognition that CNS and non-CNS response to systemic therapy can be and often is discordant. Prior response criteria did not make specific provisions for this: RECIST historically considered all organ sites together, while RANO-HGG does not account for extracranial disease assessment as this is typically not important with primary brain tumors. Consequently, the RANO-BM group recommends separate assessment of CNS and non-CNS compartments, with RANO-BM used to evaluate CNS disease, and RECIST 1.1 still used to evaluate non-CNS disease.
Pembrolizumab is a PD-1 inhibitor that has been approved for the treatment of metastatic melanoma and non-small cell lung cancer (NSCLC). A phase II clinical trial studying the effect of pembrolizumab in patients with previously untreated or progressive melanoma or NSCLC brain metastases was initiated at our institution prior to publication of the RANO-BM criteria (NCT02085070). To report trial outcomes, a modification of RECIST 1.1 (modified RECIST, mRECIST) was created. Here we report a comparison between RANO-BM, standard RECIST 1.1, RANO-HGG, and mRECIST, when used to evaluate CNS response of patients from this trial.
Patients and methods
Patients
Patients were accrued to an ongoing clinical trial of pembrolizumab for patients with melanoma or NSCLC with untreated or growing brain metastases (NCT02085070). Experience with the first 36 patients (18 from each cohort of the trial, melanoma or NSCLC) has been reported [10], and these same 36 patients were included in this comparison study. Of the melanoma patients, the median age was 65 (range 41–85), 67% were male, median ECOG PS was 1 (range 0–1), 33% had BRAF mutations, and 67% had prior local therapy for brain metastases. Of the patients with NSCLC, the median age was 59 (range 33–82), 67% were female, median ECOG PS was 1 (range 0–1), and 56% had prior local therapy for brain metastases. Any clinically concerning brain metastases (e.g. due to lesion location or size, and/or potential need for steroids due to significant perilesional edema on baseline MRI) were treated using local therapy (e.g. surgery or stereotactic radiosurgery) prior to trial entry. Given the use of local therapy, these particular lesions were subsequently considered not evaluable for the purposes of the trial. All patients received pembrolizumab at 10 mg/kg intravenously every two weeks while on trial. The primary endpoint of NCT02085070 was brain metastasis response rate as assessed by modified RECIST (mRECIST). The trial was approved by the Yale University Institutional Review Board, and all patients provided written informed consent.
Advertisement
MRI tumor measurement
Patients were enrolled onto the clinical trial after detection of previously untreated or progressing brain metastases on an initial baseline MRI conducted up to 4 weeks before starting pembrolizumab therapy. Routine MRI sequences were obtained which included among others a thin cut, isotropic (0.9 mm resolution) 3D volumetric T-1 MPRAGE acquisition, obtained after administration of 0.1 ml/kg of gadobutrol (Gadavist®). Thereafter, the images were reconstructed with an effective slice thickness of 2.5 mm to assess metastasis size. An early MRI scan of the brain was obtained 4 weeks after starting therapy for patient safety (i.e. to confirm no imminent risk in the case of a rapidly progressing lesion). Efficacy MRI brain scans were then conducted every 8 weeks after initiating therapy, using standard T1-weighted gadolinium-enhanced MRI with similar parameters as described above. Both 1D and 2D tumor measurements were conducted prospectively by a single CAQ certified neuroradiologist at the Yale Cancer Center (AM), with 5 years of experience.
Response assessment
Response on the clinical trial was determined using our institutional modification of RECIST (mRECIST), which allowed for CNS target lesions to be ≥5 mm in maximum diameter (or twice the MRI slice thickness if slick thickness was ≥2.5 mm), and up to five target lesions in the brain could be included. Other non-target lesions were prospectively followed. For the current comparison study we also assessed responses using standard RECIST 1.1 criteria [6], RANO-HGG [8], and RANO-BM [9], at each time point that mRECIST was used. Table 1 summarizes each of these approaches and the criteria for complete response (CR), partial response (PR), stable disease (SD), and progressive disease (PD). Of note, RANO-BM provides additional guidelines for evaluation of immunotherapy, which we included in this analysis. For each set of criteria, if a patient did not meet criteria for measureable disease, they were designated as ineligible. A number of patients also discontinued trial therapy before the first efficacy MRI brain scan—these patients were designated as unevaluable by any criteria. Patients were censored at the data cut-off date of June 30, 2015.
Table 1
Summary of differences between mRECIST, RECIST 1.1, RANO-HGG, and RANO-BM assessment criteria
mRECIST | RECIST 1.1 | RANO-HGG | RANO-BM | |
|---|---|---|---|---|
Measureable lesions | ≥5 mm in one diameter | ≥10 mm in one diameter | ≥10 mm in two diameters | ≥10 mm in one diameter |
Target lesions | 5 | 2 | 5 | 5 |
Complete response (CR) | Complete disappearance of all lesions | Complete disappearance of all lesions No steroid use Clinical status stable/improved | ||
Partial response (PR) | ≥30% decrease in SLD compared to baseline | ≥50% decrease in SPD compared to baseline Stable/less steroids Clinical status stable/improved | ≥30% decrease in SLD compared to baseline Stable/less steroids Clinical status stable/improved | |
Progressive disease (PD)a
| ≥20% increase in SLD compared to nadir new lesions | ≥20% increase in SLD compared to nadir, with minimum absolute increase of 5 mm new lesions | ≥25% increase in SPD compared to nadir Significant T2 signal increase New lesions Clinical status worse | ≥20% increase in SLD compared to nadir, with minimum absolute increase of 5 mm in one lesion Significant T2 signal increase Clinical status worse |
Stable disease | Does not meet criteria for CR, PR, or PD | |||
Confirmation | Confirmation by consecutive scans at least 4 weeks apart required for CR and PR | |||
Statistical analyses
A weighted kappa analysis was performed to assess the level of agreement between each set of criteria in determining best CNS response, using Fleiss-Cohen quadratic weights, as described in other work [11]. Agreement between two response criteria sets was categorized as poor (κw < 0), slight (κw = 0–0.20), fair (κw = 0.21–0.40), moderate (κw = 0.41–0.60), substantial (κw = 0.61–0.80), and almost perfect (κw > 0.80). Patients deemed ineligible or unevaluable were excluded as needed for each pairwise analysis.
Results
A total of 91 brain metastases were observed as target lesions in these 36 patients on the clinical trial. The median number of metastases observed as targets per patient was 2 (range, 1–5). The median longest diameter of target lesions at baseline was 8.5 mm (range, 5–20 mm). Lesions with local treatment prior to enrollment on clinical trial were not considered evaluable and were not included in the tumor measurements unless they had clear evidence of progression since local treatment.
Owing to differences in both size and number of target lesions allowed, 30, 29, and 38 lesions were followed in the current comparison analysis, for RECIST 1.1, RANO-HGG, and RANO-BM, respectively. Thirteen patients (9 NSCLC and 4 melanoma) did not have any lesions ≥10 mm in diameter, making them ineligible by RECIST 1.1 and RANO-BM, although they were fully eligible by the mRECIST criteria which was used for the trial. An additional six patients (3 NSCLC and 3 melanoma) did not have any lesions that were ≥10 mm in diameter in two dimensions, and would therefore have been ineligible had RANO-HGG been used, resulting in a total of 19 patients ineligible by RANO-HGG. The remaining 17 patients (6 NSCLC and 11 melanoma) were eligible by all four criteria sets. Although it appeared that melanoma patients were more likely to be eligible by all four criteria sets, this difference was not statistically significant (p = 0.18, χ² test).
The best CNS response to pembrolizumab according to each criteria set is shown in Table 2. Eight patients were deemed unevaluable in the CNS because of either rapid extracerebral progression (seven patients) or intralesional hemorrhage requiring stereotactic radiosurgery before the first efficacy scan (one patient). Of the remainder, excluding cases of ineligibility, response evaluation was identical for all patients except one, who had a best response of PD by RANO-HGG, but SD by the other three criteria sets. Concordance among the criteria was therefore excellent, as shown in Table 3, with κw ranging from 0.955 to 1.000. Notably, had RECIST 1.1 or RANO-BM been used to determine trial eligibility, five patients who went on to have an objective response (three with CR) would not have been eligible for enrollment. If RANO-HGG had been used, this increased to eight patients who would not have been eligible for enrollment (four who went on to have a CR). These patients have had durable responses, and all continue to respond at the time of this comparison analysis.
Table 2
Best CNS response according to mRECIST, RECIST 1.1, RANO-HGG, and RANO-BM in patients with melanoma or NSCLC treated with pembrolizumab
Response | mRECIST | RECIST 1.1 | RANO-HGG | RANO-BM |
|---|---|---|---|---|
Ineligible | N/A | 13 | 19 | 13 |
Unevaluable | 8/36 (22%) | 5/23 (22%) | 4/17 (24%) | 5/23 (22%) |
ORR | 10/36 (28%) | 5/23 (22%) | 2/17 (12%) | 5/23 (22%) |
CR | 4/36 (11%) | 1/23 (4%) | 0/17 (0%) | 1/23 (4%) |
PR | 6/36 (17%) | 4/23 (17%) | 2/17 (12%) | 4/23 (17%) |
SD | 4/36 (11%) | 2/23 (9%) | 0/17 (0%) | 2/23 (9%) |
PD | 14/36 (39%) | 11/23 (48%) | 11/17 (65%) | 11/23 (48%) |
Advertisement
Table 3
Concordance among the different criteria in assessing best CNS response
mRECIST | RECIST 1.1 | RANO-HGG | RANO-BM | |
|---|---|---|---|---|
mRECIST | 1.000 | 0.955 | 1.000 | |
RECIST 1.1 | 18/18 (100%) | 0.955 | 1.000 | |
RANO-HGG | 12/13 (92%) | 12/13 (92%) | 0.955 | |
RANO-BM | 18/18 (100%) | 18/18 (100%) | 12/13 (92%) |
While there were a few instances in which differences in criteria for selection of target lesions affected evaluation of target lesions (e.g. one patient had SD in five target lesions by mRECIST, but PR in one target lesion by standard RECIST), overall evaluation of CNS disease (target + non-target lesions) was not affected by target selection in any case.
The only discordant patient had one identifiable lesion at baseline. On the first efficacy scan, this lesion met RANO-HGG criteria for PD, but did not meet criteria for PD by mRECIST, RECIST 1.1, or RANO-BM, as the longest diameter had not increased by 20% (Fig. 1). With no appreciable non-target disease, his overall response was therefore SD by mRECIST, RECIST 1.1, and RANO-BM. On the subsequent efficacy scan, the lesion continued to grow, and the patient was then determined to have PD by all criteria. Ultimately, the lesion was treated with SRS and he was taken off trial.
Fig. 1
Brain MRI scans of the discordant patient, demonstrating growth in the target lesion. a at baseline, b at 8 weeks, c at 16 weeks
×
![]()
Advertisement
Discussion
Clinical trials evaluating systemic treatment of brain metastases are likely to become more common as survival improves in cancer patients and brain metastases become more prevalent. Given the need to protect the neurological safety of the patient but also to create a simplified method by which to evaluate lesional response, the RANO working group has proposed the RANO-BM criteria. The pembrolizumab study NCT02085070 was initiated prior to publication of these RANO-BM criteria, and it was conducted with a modified version of RECIST 1.1 criteria (mRECIST), providing an opportunity to prospectively assess RANO-BM and compare results with other response criteria.
The biggest difference between response criteria evaluated in this comparison paper was the definition for measureable lesions. The lower 5 mm cutoff for the minimum diameter of CNS lesions used to enroll patients onto the clinical trial with pembrolizumab (NCT02085070) resulted in the inclusion of 13 patients who would not have been eligible by standard RECIST 1.1 or RANO-BM criteria, and an additional six patients who would not have been eligible by RANO-HGG. Of note, more than a third of these additional patients (who only had lesions between 5 and 10 mm) had durable responses to treatment on the clinical trial. While the intrinsic uncertainty of measurement of small lesions could call into question the reproducibility of these results, we did use high resolution MR imaging with slice thickness of 2 mm or less, and all responses were confirmed with subsequent scans. Moreover, many of these responders had a complete response, which requires the unequivocal complete disappearance of any visible disease. This comparison paper therefore suggests that lowering the size requirement of target lesions to 5 mm when using MRIs with cuts ≤2 mm can facilitate accrual to similar trials and provide access to experimental therapies for patients with a smaller disease volume in the CNS.
For the patients who were eligible for comparison among the various response criteria, concordance of response was excellent, with κw ranging from 0.955 to 1.000. The only case of discordance involved the use of 2D measurement in RANO-HGG; the longest diameter of the single target lesion increased by <20% but the perpendicular diameter increased by >70%, resulting in >100% increase in the 2D product. The ultimate course of the patient was progressive disease, and use of 1D measurements only resulted in an 8 week delay in determining disease progression. However, while it may therefore be tempting to suggest that 2D measurement is more sensitive to lesional changes than 1D measurement, this must be balanced against the increased measurement variability observed with 2D versus 1D measurement [11]. Automated 3D radiographic response evaluation may ultimately allow for true volumetric measurement, which has been shown to be more sensitive to tumor changes and less subject to variation than any linear methods [12]. However, until this becomes readily and widely available, this comparison suggests that 1D and 2D measurements are essentially equivalent, with 1D measurements simpler to conduct.
Other differences between response criteria included number of target lesions, incorporation of clinical status and steroid use, thresholds for progression and response, inclusion of peri-lesional T2/FLAIR signal changes, and inclusion of measurements of new lesions for determination of progression (in RANO-BM only). None of these ultimately affected results for patients in the current comparative analysis. There were a number of cases in which choice and number of target lesions did affect evaluation of target lesions, but in each case there was unequivocal non-target progression, making any differences in target lesion evaluation moot. With regards to clinical status and steroid use, given that we enrolled patients with smaller lesions and prophylactically treated potentially concerning lesions prior to trial entry with local therapy, the study population was less likely to have or develop neurological compromise or require steroids. As the trial involved immunotherapy, all patients were also required to be off steroids prior to enrollment. Inclusion of clinical status and steroid use in evaluation of response was therefore ultimately irrelevant in this comparison analysis.
Advertisement
One final limitation of all the current criteria that is not addressed by this comparative analysis is how to handle lesions that were previously treated with local therapy prior to trial entry. Given such phenomena as pseudoprogression [13] and radiation necrosis [14], this is likely to be relevant in future studies with immunotherapy and requires further investigation and improved imaging methods, as distinguishing true progression from such treatment effect remains difficult [9].
Limiting our study was the relatively small number of patients. This impacted our ability to study whether the differences we found between response criteria reached statistical significance. Given the small and subtle differences found, future studies attempting to discern statistical significance will likely need to be much larger to obtain sufficient statistical power. In addition, the RANO group recently has published the immunotherapy RANO (iRANO) criteria [15]. While these are not standalone criteria, they do provide additional guidelines for existing RANO criteria (e.g. RANO-HGG, RANO-BM) that are specific to immunotherapy response assessment. The essence of iRANO pertains to the differentiation of progressive disease from pseudoprogression, through the confirmation of initial imaging suspicious for progression via repeat imaging 3 months later. While including iRANO in this comparison study could have been informative, our trial was designed before the publication of iRANO, and so we did not plan in our protocol for confirmatory scans. Given that patients deemed to have progressive disease were often quickly treated with local therapy, such confirmatory imaging is not available to us, making it impossible to determine how this new guideline might have affected response assessment. The applicability of iRANO therefore could not be explored in this analysis, but should be included in future studies as the use of immunotherapy becomes more widespread.
In conclusion, as applied to this clinical trial, the most notable difference among the four criteria (RECIST 1.1, RANO-HGG, RANO-BM, and our institutional modification of RECIST) was the definition of measureable lesions. The use of a lower 5 mm cutoff in our mRECIST criteria allowed enrollment of significantly more patients than if we had used one of the other response criteria. Inclusion of these additional patients provided them access to a new therapy, and over a third had confirmed, durable responses. Other differences among the response criteria had little effect in this comparison analysis, resulting in high concordance across all four criteria sets. Our data support modification of the RANO-BM criteria to include lesions 5–10 mm in diameter with MRI slices of ≤2 mm to facilitate enrollment without compromising validity and possibly improving patient safety.
Funding
This analysis did not receive any funding support. NCT02085070 received funding support from Merck and the Yale Cancer Center, but these funders did not have a role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation of the manuscript; or decision to submit the manuscript for publication. Merck was involved in review and approval of the manuscript.
Compliance with ethical standards
Conflict of interest
JMQ reports funding from the NIH-NCATS CTSA-TL1 Medical Student Research Fellowship outside the submitted work. JBY reports funding from twenty-first Century Oncology outside the submitted work. SBG reports grants from Merck during the conduct of this study, and grants from AstraZenenca and personal fees from Clovis outside the submitted work. HMK reports grants from Merck during the conduct of this study, and consulting fees from Regeneron, Bioclinica, Alexion, and Prometheus outside the submitted work. No other disclosures are reported.
Ethical approval
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.