We’ve learned that R square is not the best goodness of fit because one can manipulate its static as close as possible to 1 by adding additional independent variables. As alternatives to that, one can use adjusted R-square or AIC since these two statistics give penalty on the manipulation of including more independent variables.
In time series model, economist use ARMA (p, q) model to find the best forecasting model of interest where p is order of AR component and q is order of MA component. ARMA combines the autoregressive part with moving average. For simplicity ARMA (1, 1) indicates AR (1) + MA (1). In regression model form, that is y (t) = m + a*y (t-1) +e (t)-b*e (t-1) +error term. Now let’s assume that there are 3 competing models. The first one is ARMA (1, 1). The second one is ARMA (2, 4). Last one is ARMA (2, 12). The last model might contain yearly effect because its lag of MA component corresponds to 12.Due to the nature of different lags of time series model, each model has slight different number of observations. ARMA (1, 1) will have 4 more observations that that of ARMA (2, 4). In case of ARMA (2, 12) model, the number of observations seems to differ substantially compared to two other models.
Conditioned on all other else criteria such as residual analysis, normality test and MSE etc being similar in model selection, is the smallest value in AIC still the best in selecting a model among competing ones? The regression model we learned in the class did not have different number of observations. However, in time series model, the number of observations can be different as I showed above. So I wonder AIC is still good goodness of fit in selecting the model. If not, what kind of other goodness of fit we can use in this case?
I appreciate your answer.