Do You Know How to Avoid Rankings Data Integrity Issues?

Written by Rodney G. Alsup and published by the MBA News Digest on July 26, 2018.

Lies, damned lies, erroneous, inaccurate, false, misreported, falsified, inflated are all terms used in recent articles to describe the Temple University ranking’s scandal. An outside review by the law firm Jones Day found that Temple inflated student test scores and GPAs and knowingly provided false information to U.S. News and World Report. Also, disbanding a longstanding committee charged with ensuring the accuracy of rankings data created an absence of checks and balances which enabled data inflation and information falsification.  While it is difficult to eliminate 100% of the risk associated with an individual or group of individuals intentionally misrepresenting self-reported information and data to a rankings organization, there are steps you can take to minimize the risk.

What are the immediate action steps?

There are four immediate action steps program managers and deans should consider.

1. Document the current reporting process. Written procedures show the steps being followed and provide an audit trail.
2. Have the campus internal auditors review the current reporting process. They train internal auditors to find process weaknesses and how to make improvements. Also, their independence and objectivity reduces the risk of creating a report based on inflated and false data or information.
3. Change the reporting process based on the internal auditors’ recommendations. The internal report identifies process weakness and recommended improvements. You should change written procedures and reporting processes based on improvement recommendations.
4. Request that the internal auditors conduct annual reviews of the internal reporting processes. Annual reviews help make sure that program managers, staff, faculty, and deans follow the documented reporting processes.

What are the annual reporting steps?

Providing self-reported data and information to a data rankings organization should follow a pre-defined set of steps. Here are six suggested steps:

1. Do a trend analysis of key data items used to prepare the report. Start with a trend analysis using the data someone will use for the report and make a comparison to earlier reporting periods. Look for anomalies and investigate each looking for errors, omissions, or intentional misstatements.
2. Prepare a draft report for submission. Draft reports offer the opportunity to see how the data will look to others.
3. Have the program manager review the report, the trend analysis and the internal auditors’ recent report. Review the report looking for things that look inconsistent with earlier reports. Discuss the findings with the report’s preparer.
4. Have a committee review the report, the trend analysis and the internal auditors’ recent report. Review the report looking for things that look inconsistent with earlier reports. Discuss the findings with the report’s preparer and the program manager.
5. Have the dean review the report, the trend analysis and the internal auditors’ recent report. Review the report looking for things that look inconsistent with earlier reports. Discuss the findings with the report’s preparer, the program manager, and a representative of the committee.
6. Submit the report.

What about collusion?

All is for naught if there’s collusion among the players above and their aim is to misrepresent self-reported information and data to a rankings organization to make their program look better. But, you can reduce the risk of collusion and unintentional errors when annual reporting processes are in place and being reviewed by others.

Join the Conversation!

Question: Does your program have internal auditors review internal controls and processes? If so, please share how this works with your program. You can provide your answers by adding a comment or you can email me at RGAlsup@MBANewsDigest.com.