SWFWMD WebForum

mbarcelo
In person meeting to discuss Peer Review Comments received to date on the Conceptual Model Document
Quote 0 0
mbarcelo
Meeting 4 agenda
Quote 0 0
mbarcelo
Meeting 4 presentations
Quote 0 0
tdesmara
Attached is a more complete selection of model cross-sections - a supplement to "20161017__ECFTX_LayerElevs__forPeerReviewMtg.pptx" above.
Quote 0 0
emcdotomb
Apparently there have been at least four meetings where the three peer review experts have made comments to the ECFTX model. Where can the minutes to these meetings be found?
Quote 0 0
emcdotomb
Is the AFSIRS approach to estimating ET curremtly used throughout Florida by all five WMDs? Is it used though out the US? Is it the accepted "gold standard" through out the world? My research indicates that it was mostly worked on in the early 2000's. Has there been any improvements, enhancements, modifications, etc. in the last 10 years? If so what are they?
Quote 0 0
pete.andersen
I received an e-mail from Mr. Edward McDonald that contained some specific questions about the calibration.  I referred him to the web board and stated I would use this as a forum for answering his questions.

November 10, 2016, Edward McDonald wrote:

 

Mr. Andersen,

 

Thank you very much for your reply. I just recently became aware of the web board mentioned in your email. In fact my first attempt to contact you was unsuccessful because I used an old email address from one of your presentations that was included on the web board. I notice that you as well as the other two professionals that are reviewing the ECFTX model have many questions/comments. I too have many questions. As I live in Polk County the conclusions that are reached via the model will have a direct impact on me and other Polk County citizens. It is important to me that the model is done right and that it's limitations and uncertainties are clearly stated. The following is my original email that was not delivered successfully.

 

I am a retired Mechanical Engineer that has been following and commenting on Polk County's water issues for many years. Though I will never have the in-depth understanding of modeling that you do, my background does allow for a basic understanding of the approach used.  My interest in the ECFTX model is nearly boundless, but for the purpose of this email I would like to limit my comments to calibration. From what I can determine, calibration is also important to you.

 

Why is calibration of a model necessary? My thoughts are that it is way to "fill-in-the-blanks"; i.e. a way of accounting for missing data. I have the following two thoughts regarding calibration and I was hoping that you could take a few minutes to correct my thinking.

 

1. Calibration would not be required if the real world could be accurately represented. In other words, if all required input data was available.

 

2. When very little data is available, calibration of the model is relatively easy but the uncertainty of such a model would be high.

 

Thank you for your time.

 

Edward McDonald

Auburndale, FL


My Reply:

Dear Mr. McDonald,

Thank you for your interest.  I’ll try to answer your basic question, “why is calibration necessary?” and then comment on your 2 statements.

Calibration of a groundwater model is defined as the adjustment of model inputs, such as aquifer/aquitard parameters, inflows/outflows, and boundary conditions, such that the model output (water levels and flows) provides an acceptable match to observed conditions for the same water levels and flows.  In essence, the process works as follows.  The modeler initially develops a conceptual model of the hydrogeologic system, which is his/her understanding of system based on data that have been collected, observations of system response through time, and professional experience.  This conceptual model is converted to a numerical model by assigning all the necessary inputs that define the system.  These inputs come from a variety of different sources, are of variable quality, and may only be representative of a limited spatial (or vertical) or temporal domain.  A limitation that is unique to groundwater models is that most data are “point data”, that is they are measured at a single point, and therefore are strictly only representative of that point.  To be practical, however, data points are generally assumed to be representative of a larger area and their values are either extrapolated or interpolated with others to represent a much larger area. An acceptable match between modeled and observed conditions is rarely achieved when the numerical model is run for the first time using the initial conceptual model and initial parameter estimates.  The inability to satisfactorily match observed conditions with the model indicates an error in the conceptual model.  The modeler seeks to improve the conceptual model through the calibration process: adjusting uncertain parameter values or distributions, changing the influence of boundary conditions, etc.  A distinct advantage that the model has over other data analysis techniques is that it allows the effect of the inputs to interact with one another and develop a response/solution. This interaction cannot be evaluated when data are evaluated independently of one another.  The initial model run, even if erroneous, can provide clues to the modeler as to where the deficiencies in the conceptual model lie.  The modeler works through the process by sequentially adjusting model inputs until an acceptable match to observed conditions is achieved.  The calibration process can either be a manual trial-and-error process, can be automated to mathematically minimize differences between model and observed conditions, or can be a combination of both.

To directly answer your question, calibration is necessary because of uncertainty in the data and what they represent.  Available data have differing levels of uncertainty associated with them and ranges from relatively low (municipal pumping) to relatively high (distribution of vertical hydraulic conductivity within confining beds).  Just as with calibration, there are a variety of techniques available for dealing with uncertainty.  These range from conducting sensitivity analysis to understand the importance of individual parameters on the model output to a stochastic analysis that assigns probability density functions to parameters and ultimately outputs the probability of a certain outcome (e.g. 90% confidence that a drawdown of x feet will not be exceeded at location A).

You added two statements at the end of your letter, to which I generally agree, but have a few comments, which I have listed after your italicized statement

1. Calibration would not be required if the real world could be accurately represented. In other words, if all required input data was available.  Agree.  However in my 37 years of experience in developing and reviewing groundwater models, I have never come across a case where all the required input data was available and highly certain.

2. When very little data is available, calibration of the model is relatively easy but the uncertainty of such a model would be high.  Again, I generally agree.  However, I am not quite clear on what is included in your term of “data”.  I can visualize a situation where there is little input data available, but there is a lot of observational data available.  In this case, the model calibration could be quite difficult as one attempts to match a complex flow field.  The situation you state would probably also be true with a fair amount of input data, but limited observational data.  In either case, I believe you are correct that results would be uncertain.

I hope my thoughts on your question and comments are helpful.  I encourage you to be a part of the modeling process by participating in the web board and attending scheduled meetings/teleconferences.

Best Regards,

Peter F. Andersen




Quote 0 0
emcdotomb
First, Mr. Anderson, thank you very much for your very detailed reply to my questions. Your response confirmed and enhanced my basic understanding of model calibration. I would like you to follow-up on one point that you made and that is the difference between input data and observational data. To me, input data is anything that the model uses to simulate or approximate the real world. In other words, it is known information about the world that you are trying to model. I don't know how this differs from observational data.

One final comment involves a sensitivity analysis. I believe it is true that all input data is not of equal importance or weight. Isn't it true that some input parameters can vary wildly and have very little impact on the critical outcomes (output of the model) while small changes in others can have significant impacts?
Quote 0 0
Mark Stewart
Reference by Mehl and Hill (2010) that examines model grid effects on fluxes between SAS and rivers and lakes
Quote 0 0
pete.andersen
During yesterday's teleconference I mentioned that I would provide a more formal listing of the items that I felt needed some further discussion of where we ended up following the in-person meeting.  These are:
1) transient and steady state calibration.  My understanding was that a steady state calibration would be performed initially, followed by a transient calibration.  I am unclear regarding the selected year for the steady state calibration and details on the period and duration of the transient simulation.  Further, the handling of any "spin-up" period is not clear.  We also discussed breaking the transient period into spin-up, calibration, and "validation" periods.
2) PEST.  Similarly, I don't have a clear idea of what was decided regarding how PEST would be used.  There seemed to be some question of whether PEST could even be used effectively, given model size, # of parameters, and duration.
3) LFA discretization.  There was unresolved discussion of how the LFA should be represented.  There were some the felt that the proposed amount of layering was not justified and the additional cells would only slow the calibration effort.
4) Representation of Rivers and Lakes.  Mark Stewart had some cautionary words regarding the use of river baseflow as a calibration target.  In addition, the use of river cells to represent lakes seemed inconsistent with the way they might be represented using the lake package. 
5) Required accuracy versus attainable accuracy.  Some metrics for calibration were discussed, but it wasn't clear to me if these were what the modelers thought could be attained or whether these were the required accuracy to meet the objectives of the model.

As I mentioned yesterday, I think it would be useful to document the current plan and understanding, recognizing that it may change as the modeling progresses.
Quote 0 0
pete.andersen
This is a belated reply to Mr. Ed McDonald's post of November 14.  Please excuse the tardiness of this reply, I was not checking the web board regularly and this was compounded by the Holiday.

Your first question was what I meant by input data and observational data, and if the two aren't the same thing.  My distinction has to do with the application of the model:  there are parameters, geometries, and boundary conditions that are entered into the model data sets.  These are things like hydraulic conductivities, recharge rates, river stages, etc.  I refer to these as "input data".  The model is run using these input data and the model computes heads and drawdowns.  I refer to these heads and drawdowns as "observational data".  You are correct that they are all "data", the distinction is whether they are input directly or used as a target for the model.

Your second question was about sensitivity analysis.  Yes, it is true that there are some data that have very little influence on model results while others have a big influence.  A useful outcome of sensitivity analysis is to identify the sensitivity of model inputs and relate these to "importance".  The model sensitivity analysis can in this way be used to design a data collection program that focuses on the "important" data.  A cautionary note: the model can be insensitive to a parameter for the calibration (past) phase, but could be sensitive to it for the predictive (future) phase.  An example of this would be hydraulic conductivity of a lakebed: the model may not be particularly sensitive to this in an unstressed past condition, but could be very sensitive in a future stressed (nearby pumping) condition.

Hope this helps!
Quote 0 0
emcdotomb
Thanks again, Mr. Andersen. for your very thoughtful response. In general, it's has been my experience that the folks responsible for the CFWI RWSP (including the modeling work) are not interested in hearing alternative points of view from the general public. That's why I am counting on you and the other two reviewers of the model to "keep-them-honest". The perfect example of my complaint is that I have asked for and never received any sort of minutes to the meetings that you have had in regards to the modeling effort. You and your colleagues have prepared many written questions and I have no idea how, if or when they were addressed.  I worked in the professional field for over thirty years and I can't imagine a technical meeting without written records of what was discussed, the answers given and the "next-step" direction. There is a great deal of time, money and politics at stake with the CFWI and the ECFTX model is at the heart of everything.
Quote 0 0
mbarcelo
Meeting 4 minutes and attendee list
Quote 0 0

Add a Website Forum to your website.

Southwest Florida Water Management District home page

Home Page  •  Who We Are & What We Do  •  Search & Site Map  •  Contact Us  •  Privacy & Disclaimer  •  © Copyright  •  pdf Download PDF Reader