# Refined Index Of Agreement

In this case, the baseline of the μ denominator is defined on the basis of both and. However, this last index presents a number of serious defects that are illustrated in the analysis below. the last term is proportional to the covariance between X and Y. One way to create an index that explicitly contains this term of covariance in the denominator and limit it to always being positive: in this case, the denominator μ is defined by adding up the differences of all points of X and Y compared to the average of X. The original version was based on gridded deviations, but was later modified 15 using absolute deviations, arguing that MAD (or MAE in this case, because it refers to errors between forecasts and observations instead of deviation) is a more natural measure of average errors and less ambiguous than RMSD (or RMSE)12. Another refinement of the index16 was intended to remove the predictions from the denominator, but as others have argued14, this amounts to resizing the expression of the coefficient of effectiveness, while the interesting reference point is lost. Again, these indices do not meet the requirement for symmetry. For the indices mentioned in Section 0 that meet the symmetry criteria, it is proposed to compare metric performance at intermediate intervals: Mielkes and permutation-based indices, Watterson`s M index, and Ji-Gallo`s AC-agreement coefficient. To do this, an artificial record is created. Two independent random vectors of samples using 0 and standard deviation of 1 are produced first and entirely decoral with a decomposition of Cholesky18.19.

These two vectors and are then re-combined to generate two new vectors, and which have some imposed correlation (details are explained in the Additional Information section). The X and Y datasets are thus generated by the alignment of all vectors and vectors generated for correlations of 1 to 1 or all vectors (they are shown in Figure 1 for certain correlation values). With X and Y, compliance metrics for vector pairs can be calculated and compared with equal averages and variances, but with a correlation of 1 to 1. To assess the behaviour of metrics in case of data distortion, the Y data set is further disturbed by the introduction of various systematic additive distortions (in practice by adding b to the data) and by systematic proportional biases (by multiplying the data per m), as shown in Figure 1. These systematic additive and proportional effects can also interact with each other, either to compensate for disagreement or to accentuate it. The advantage of this property is that the value of the index can be immediately compared to r, which is a metric with which most practitioners are familiar.