Some Issues Affecting Comparison of
Computations and Experiments

Do the boundary conditions exactly match that of the experiment?

Computational results are determined by the boundary conditions that are applied. Often we use constant velocity or temperature boundary conditions, but do we really know that the value is constant over the whole patch? Do we really know the boundary condition value itself? Similar concerns exist for the type of boundary condition applied. For example, thermal boundaries are often set to be adiabatic, but unless that boundary is perfectly insulated there will be a heat loss/gain. Think about every boundary condition setting (velocity, pressure temperature, turbulence quantity, mixture composition, etc.) and ask how much confidence do you have that the setting you are making is an exact match to the experimental conditions.

The advantage with trend based analysis is that you often keep these settings the same for every simulation, thus small uncertainties do not affect the resulting trend.

Do the material properties exactly match that of the experiment?

You know that your solids are made of aluminum and the gas flowing over them is air. But do you know the properties of that aluminum? Do they change with temperature? Do you know the properties of the air? Is it perfectly dry air or humid? Think about every property setting (density, thermal conductivity, specific heat, viscosity, etc.) and ask how much confidence do you have that the setting you are making is an exact match to the experimental conditions.

The advantage with trend based analysis is that you often keep these settings the same for every simulation, thus small uncertainties do not affect the resulting trend.

Did you simplify the geometry to make gridding easier?

Real world geometries are often very complex. Maybe there are slots or holes that are quite small and we choose to ignore them with the assumption that they do not affect the results. Perhaps the flow passes by a bolt head but who wants to grid that bolt head? Castings often have fillets that are quite small and will often be left out of the grid system. Some geometries that are "almost" symmetric or axi-symmetric will be modeled that way to save computational resources.

With trend based analysis the grid system is usually constant so any simplifications to the geometry are the same for every simulation and while these simplifications may degrade the accuracy it does not affect the resulting trend.

How accurate are the experimental measurements?

Experimental results are subject to measurement uncertainties. Thermocouples, pressure transducers, velocimeters, etc. all have an uncertainty to their values. If that uncertainty is +/- 1% then one should not expect the computation to get any closer than the error band of the measurement.

With trend based analysis we do not concern ourselves so much with exactly matching the experiment. Instead we seek to match trends in the results.

Are the experimental results repeatable?

Computational results will always produce the same exact result (with the same given inputs). Sometimes experimental results can not be reproduced exactly. This can be for a wide variety of reasons (changes in operation conditions, measurement uncertainties, etc). If you have experimental results that cannot be reproduced to within 5%, then you should not expect the computational result to match that closely.

Trend based analysis keeps most inputs constant and may vary only a single parameter to asses that parameter's impact on the results.

Did the measurements in the experiment affect the results?

Some measurement techniques can affect the result they are trying to measure. For example a pitot probe in a small enough channel can block enough of the flow to affect the velocity measurement. This may be one reason you may not get good match between your experimental measurement and your computation result.

The nice thing about all simulation results is that the measurements are all "non-intrusive".

Do the proper computational models exist and have they been used?

Several of the techniques used in computational modeling are "models" of physical behavior (e.g., turbulence, surface reactions, momentum resistances, etc.). Most of these models are based on empirical relationships and many require inputs to define the model behavior (reaction rate constants for surface reactions for example). There is often alot of uncertainty associated with the input values for these models and that will therefore produce uncertainty in the computational results.

Trend based analysis keeps these model parameters constant while we look at the trend of the results.

Is the solution grid independent?

Your computational grid system is nothing more than a discretization of the real geometry that you are trying to model. Changes to the grid system (number of points, skewness, grid clustering, etc.) may change the results. It is often difficult or impossible to keep changing (refining) the grid system until you obtain results that are grid independent. Ideally you would double the number of grid points and rerun the solution. If the results change you double again until you reach the point where the results stop changing.

With trend based analysis the grid system is kept constant so while the inaccuracies of discretization may affect direct comparison to the experiment, they do not adversely affect the trends produced.

Is the solution fully converged?

In order to make use of any computational results we need to make sure that the simulation has converged. If you are only able to get 1 order or less of convergence then the results are not trustworthy and will often be unphysical. Only after 3-5 orders of convergence can the results be trusted. The difference between 3 and 5 orders of convergence may be just 1% in the results. See also the past ESI-USERS tip on convergence for more information about assessing converergence.

With trend based analysis you should converge to the same level of accuracy for every simulation to ensure that convergence level is not affecting the trends.