From the overview:

Ratings produced in Quality Rating and Improvement Systems (QRIS) are intended to provide meaningful information to parents and policymakers about the quality of early care and education programs. Given this central purpose of ratings, it is important to establish and follow guidelines for creating QRIS ratings that can promote their validity and integrity. The purpose of this brief is to demonstrate how using the principles of scale development can support the development of QRIS ratings.

The brief summarizes an analysis that uses the data from six large studies of early care and education to simulate state QRIS ratings. The results suggest that QRIS ratings can achieve their desired goal of predicting gains in child outcomes when attention is paid to the psychometric principles of scale development including: dimensionality (ensuring that a scale represents one, not multiple dimensions), selecting items with strong evidence, and scoring items using established criteria for cut points. The analysis provided significant validation —albeit modest, in terms of the strength of associations—of almost all of the carefully selected quality measures of classroom experiences. This finding was observed when measures were analyzed as individual quality indicators and when they were combined into a summary QRIS rating of classroom experiences. It also held true whether focusing only on structural indicators, or including both structural and process indicators of quality.

The analysis provides an example but not a blueprint for developing QRIS ratings. The authors conclude that ratings can be strengthened by using a careful approach to construction of the ratings that takes into account the content and evidence base for selected quality indicators.