Interpretation of validation results

Best estimate vs. forecast

The forecast is the deterministic part of the TOPAZ simulation, which starts on the bulletin date. Thus, there is one 10 day forecast for each bulletin date. The best estimate is one long time series, built from the one-week ensemble runs which tie together the analysis and the state on the following bulletin date (the bulletin lags the corresponding analysis by one week).

The forecast is initialized with an analysis which is valid one week before the bulletin date. Results from the first simulation week of the forecast, which leads up to the bulletin date, are not validated, and consequently not displayed here.

Bias, RMS values

Daily results for biases and root-mean square (RMS) differences are average values for the domain where the model results and sea ice thickness data (SMOS, CryoSat) overlap. Since the data are subject to quality filters (see validation algorithm for details), the mutual domain for observations and model results is always nearly identical to the former.

Note that regions where the observations and the model results are both indicative of open ocean (ice free) conditions are not taken into account when bias and RMS difference are computed (relevant for the processing of SMOS data). The persistence lines in the presentations of validation vs. SMOS data show bias and RMS results if the SMOS data from the inital date is retained throughout the period. All bias values are computed by subtracting observational data from the corresponding model results.

The bulletin date is written with green letters in the x-axis labels. If present, thin lines bridge periods with missing SMOS data in between. (Line plots are not provided for the validation vs. CryoSat data.

Thickness category results

The areas which are associated with each of the four SMOS thickness categories are stored when the validation metrics are computed. They are then presented as time series for each of the products (the best estimate, the forecast and the SMOS data).

Furthermore, the category grid cell count match-up between observations and model results are provided in the form of contingency tables for every second day of each bulletin validation for the SMOS validation. For the validation vs. CryoSat data, contingency tables are produced for the best estimate product, and for selected forecast ranges.

Scatter diagrams and quantile/quantile plots

For one of the days in the bulletin, all grid cell pairs of SMOS observations and model results are presented in scatter diagrams. For the CryoSat data, the aggregated gridded data for the 2-week period under consideration form the basis for the scatter diagrams. Also included in these figures are a blue and a red line than run parallell to the "perfect forecast" black line (the diagonal). They are determined so that 90% of all observation-model pairs are between these lines, with 5% on each side is off the region between the lines.

For the quantile/quantile plots (QQ-plots), observations and model results where sorted by increasing values. Then circles where drawn for every 100th value pair, starting with pair no. 50 for the TOPAZ/SMOS comparison .

Note that grids where observations and model results both indicate open water conditions were removed when drawing these figures (relevant for the SMOS data).

Sea ice thickness maps

Finally, we present sea ice thickness maps for all products. The results are contoured according to the thickness category definition (see validation algorithm for details). Note that the region impacted by the quality filter, or without data, is displayed in dark grey. As shown, the regions with low quality observations were masked in both model products, in order to facilitate a 1-to-1 comparison.