I received an e-mail from Mr. Edward McDonald that contained some specific questions about the calibration. I referred him to the web board and stated I would use this as a forum for answering his questions.
November 10, 2016, Edward McDonald wrote:
Thank you very much for your reply. I just recently became aware of the web board mentioned in your email. In fact my first attempt to contact you was unsuccessful because I used an old email address from one of your presentations that was included on the web board. I notice that you as well as the other two professionals that are reviewing the ECFTX model have many questions/comments. I too have many questions. As I live in Polk County the conclusions that are reached via the model will have a direct impact on me and other Polk County citizens. It is important to me that the model is done right and that it's limitations and uncertainties are clearly stated. The following is my original email that was not delivered successfully.
I am a retired Mechanical Engineer that has been following and commenting on Polk County's water issues for many years. Though I will never have the in-depth understanding of modeling that you do, my background does allow for a basic understanding of the approach used. My interest in the ECFTX model is nearly boundless, but for the purpose of this email I would like to limit my comments to calibration. From what I can determine, calibration is also important to you.
Why is calibration of a model necessary? My thoughts are that it is way to "fill-in-the-blanks"; i.e. a way of accounting for missing data. I have the following two thoughts regarding calibration and I was hoping that you could take a few minutes to correct my thinking.
1. Calibration would not be required if the real world could be accurately represented. In other words, if all required input data was available.
2. When very little data is available, calibration of the model is relatively easy but the uncertainty of such a model would be high.
Thank you for your time.
Dear Mr. McDonald,
Thank you for your interest. I’ll try to answer your basic question, “why is calibration necessary?” and then comment on your 2 statements.
Calibration of a groundwater model is defined as the adjustment of model inputs, such as aquifer/aquitard parameters, inflows/outflows, and boundary conditions, such that the model output (water levels and flows) provides an acceptable match to observed conditions for the same water levels and flows. In essence, the process works as follows. The modeler initially develops a conceptual model of the hydrogeologic system, which is his/her understanding of system based on data that have been collected, observations of system response through time, and professional experience. This conceptual model is converted to a numerical model by assigning all the necessary inputs that define the system. These inputs come from a variety of different sources, are of variable quality, and may only be representative of a limited spatial (or vertical) or temporal domain. A limitation that is unique to groundwater models is that most data are “point data”, that is they are measured at a single point, and therefore are strictly only representative of that point. To be practical, however, data points are generally assumed to be representative of a larger area and their values are either extrapolated or interpolated with others to represent a much larger area. An acceptable match between modeled and observed conditions is rarely achieved when the numerical model is run for the first time using the initial conceptual model and initial parameter estimates. The inability to satisfactorily match observed conditions with the model indicates an error in the conceptual model. The modeler seeks to improve the conceptual model through the calibration process: adjusting uncertain parameter values or distributions, changing the influence of boundary conditions, etc. A distinct advantage that the model has over other data analysis techniques is that it allows the effect of the inputs to interact with one another and develop a response/solution. This interaction cannot be evaluated when data are evaluated independently of one another. The initial model run, even if erroneous, can provide clues to the modeler as to where the deficiencies in the conceptual model lie. The modeler works through the process by sequentially adjusting model inputs until an acceptable match to observed conditions is achieved. The calibration process can either be a manual trial-and-error process, can be automated to mathematically minimize differences between model and observed conditions, or can be a combination of both.
To directly answer your question, calibration is necessary because of uncertainty in the data and what they represent. Available data have differing levels of uncertainty associated with them and ranges from relatively low (municipal pumping) to relatively high (distribution of vertical hydraulic conductivity within confining beds). Just as with calibration, there are a variety of techniques available for dealing with uncertainty. These range from conducting sensitivity analysis to understand the importance of individual parameters on the model output to a stochastic analysis that assigns probability density functions to parameters and ultimately outputs the probability of a certain outcome (e.g. 90% confidence that a drawdown of x feet will not be exceeded at location A).
You added two statements at the end of your letter, to which I generally agree, but have a few comments, which I have listed after your italicized statement
1. Calibration would not be required if the real world could be accurately represented. In other words, if all required input data was available. Agree. However in my 37 years of experience in developing and reviewing groundwater models, I have never come across a case where all the required input data was available and highly certain.
2. When very little data is available, calibration of the model is relatively easy but the uncertainty of such a model would be high. Again, I generally agree. However, I am not quite clear on what is included in your term of “data”. I can visualize a situation where there is little input data available, but there is a lot of observational data available. In this case, the model calibration could be quite difficult as one attempts to match a complex flow field. The situation you state would probably also be true with a fair amount of input data, but limited observational data. In either case, I believe you are correct that results would be uncertain.
I hope my thoughts on your question and comments are helpful. I encourage you to be a part of the modeling process by participating in the web board and attending scheduled meetings/teleconferences.
Peter F. Andersen