This is a more of my own summary for the joint work on BMD with my manager, Xiaoyi, at Corteva. We conduct research on statistical analyses of avian reproduction study for risk assessment.

Last updated on 03-08-2023.

Intro

US EPA and EFSA both published new guidelines on how to derive a Reference Point (RP) for risk assessment. Previously, NOEC (No Observed Effect Concentration) was a commonly used approach to find a safe dose. This approach is simple and straightforward but it is limited as it cannot derive any dose other than the ones administered in the experiment and the statistical power highly depends on the sample size and allocation method. The criticism of NOEC is not new (Laskowski, 1995) and over the last ten years, the principal approach of locating safe dose had shifted from NOEC to Benchmark Dose (BMD) analysis. The benefits of using BMD approach include taking advantage of the shape of the entire dose-response curve, extending estimated reference point to any dose level (extrapolation would be a carveat, but we will discuss this later), and obtaining a better understanding of the nature of dose-response relationship. US EPA explains BMD analysis as follow:

“The BMD is a chemical dose or concentration that produces a predetermined change in the response rate of an adverse effect, such as weight loss or tumor incidence. The BMD is a range, rather than a fixed number. For example, the benchmark dose (lower confidence limit) (BMDL) can be regarded as a dose where the observable physical effect is less than the predetermined benchmark response (BMR). BMD methods are used by the U.S. EPA and throughout the world for dose-response analyses to support chemical risk assessments and regulatory actions.” link

And EFSA describes BMD in a similar fashion by stating “the benchmark dose (BMD) is the minimum dose of a substance that produces a clear, low level health risk, usually in the range of a 1-10% change in a specific toxic effect such as cancer induction.” link

Previously, frequentist BMD analysis was a answer to derive safe dose by fitting multiple non-linear models and selecting the winning model which fits the date the best. Due to the potential inconsistency between results from different models, more integrative approaches have been suggested, like lowest lower bound (more conservative), model averaging with weighted average BMD/BMDL, etc. Model averaging was then promoted as it can integrate all models which fit data well and summarize the information provided by individuals models. With some mild assumptions, it can be shown that model averaging is more robust in the sense that it can better balance the bias and variance of estimated BMD. At the end of 2022, Bayesian BMD analysis along with Bayesian model averaging was on fire. Similar to frequentist approach, Bayesian BMD analysis aims to fit Bayesian non-linear models, and compute the credible interval from posterior distribution of BMD. Compared with frequentist non-linear models fitting, Bayesian approach can naturally handle the uncertainty of the estimated BMD with posterior sampling.

As Model Averaging (MA) is a very powerful methodology, I will discuss the idea of model averaging a bit more here. There are two branches of model averaging. The first one is Weighted Average BMD (WA-BMD). Under this branch, the WA-BMD is the weighted average of estimated BMD’s from selected models, where weights are typically determined by goodness-of-fit statistics. So if the data tells us a certain model is better, we assign a larger weight to it. Likewise, WA-BDMDL is defined in the same way. However, it’s very challenging to prove that WA-BMD is actually a consistent estimator without assuming true model is included. The second branch is Averaging Function BMD (AF-BMD). That is to say, the final estimated curve is the weighted average of all curves fitted by selected models. And inference of BMD is based on the weighted average curve. Intuitively, such an approach is to using all good models to approximate the true dose-response curve. However, there is no literature proving that the approximation error can be bounded or go to zero as sample size increases. The good news is AF-BMD performs well in practice with empirical assessment.

Current issues

There are still much active discussion or disagreement relating to BMD analysis. In this review, we aim to focus on four points at issue: 1) the choice of BMR and significance level in general framework of BMD; 2) formulation for individual models; 3) the choice of MA methodology; 4) the inconsistency between different software.

As recent updates on the guidelines for avian study suggested using the point estimate of BMD (instead of BMDL) as reference endpoint for risk assessment, I personally think this would make more sense as at the end of the day, the estimand is BMD. BMDL is still important to be included in the report because it gives practitioners an interval estimation of the BMD. Also, the interval indicates the variability in the dataset. When the interval is wide, we may raise more concerns on the point estimate and the quality of the data.

Another critical updates from EFSA’s end is replacing Frequentist BMD analysis with Bayesian BMD analysis. With our experience on BMD analysis of avian reproduction study, Bayesian BMD analysis has several advantages including