The National Geospatial Data Asset (NGDA) Lifecycle Maturity Assessment (LMA) is regularly performed by the NGDA Dataset Managers to identify lifecycle stages where additional actions may be needed to increase the NGDA Dataset maturity. This assists in developing the strategic and implementation actions for the management of the NGDA Dataset. Each Lifecycle Maturity Assessment has a set of questions and processes that will be evaluated to continue to improve the assessment and the usability of the results.
The 2015 Lifecycle Maturity Assessment (LMA) maturity levels ARE NOT comparable to the 2017 maturity levels due to a change from using a weighted rating algorithm in 2015 to a normalized rating algorithm in 2017. In some cases, where the reported maturity level did not change between 2015 and 2017, the calculated level that is reflected in the dashboard did change as a result of the new mathematical method. While this methodology change prevents direct comparison of the results from the 2015 to the 2017 LMAs, the change was made to normalize the maturity level values so future LMA assessment results will be directly comparable over time, starting with the 2017 results. For more details on the LMA maturity levels and the changes between 2015 and future year’s LMAs, see below.
However, the assessment of maturity and the calculation of results of the 2015 LMA introduced an additional potential result change that does not reflect an actual change in maturity between the 2015 LMA and subsequent year’s LMAs (see recommendation 4 and its effects below).
The 2015 LMA was the first year an assessment of National Geospatial Data Assets (NGDA) was performed. It established a baseline maturity level for each of the NGDA Datasets in the federal geospatial portfolio. An analysis of the 2015 LMA results and process was performed to identify ways to improve the LMA workflow, increase efficiency as well as decrease reporting burden. The implementation of recommended improvements are the factors that prevent direct comparisons between the 2015 and 2017 (and subsequent) LMAs without looking at the NGDA Dataset Reports for the years the LMA was conducted. The five recommendations, and changes associated with their 2017 implementation, are described below:
- Maintaining the OMB Circular A-16 seven stages of the geospatial data lifecycle.
- Maintaining the Maturity Matrix that describes the Maturity Characteristics for all lifecycle stages
- Identifying NGDA Datasets as Active or Static (Recommendation 3 above).
- In 2015, 8 NGDA Datasets were identified as static. In 2017, the LMA included the definitions of active and static datasets, clarifying text supporting those definitions, and the time frame in which to consider a response (i.e. when the dataset was active vs. static). This resulted in 17 NGDA Datasets identified as static for 2017. Therefore, these NGDAs may have different responses in 2015, 2017 and beyond.
- In 2015, each lifecycle stage question had a varying number of possible responses. If a question had two responses each response was weighted at 50%. If a question had 5 responses each response had a weight of 20%. This led to some questions being unequally weighted when calculating the maturity level..
- In 2017, each lifecycle stage question was normalized with six responses for each question. Each response is directly aligned with the Maturity Matrix so that the least mature response was a level 0 = no activity to level 5 = established and optimized. This makes each question equal in weight to each other based on an equal number of responses for each question and standardizes the levels of maturity. Maturity for each stage, as well as for the roll-up for all stages to obtain overall NGDA Dataset maturity, was also normalized and defined in 2017 LMA How to Calculate Maturity.
- Due to the changes in weighting and subsequent rounding in 2017, the numeric thresholds changed, and in a number of cases results in a “false” change in a maturity level. Since the possible number of responses to a question changed in 2017, reweighting the 2015 results would not offer a fair comparison since there is no way of determining if the dataset would have been rated differently if additional response options had been offered.
- In a few instances, more well-defined clarifying text to better explain questions led NGDA Dataset Managers to reconsider their responses from 2015.
Implementation of these recommendations should yield more comparable results for future NGDA Lifecycle Management Assessments.