Reform Competitive Grantmaking
The key to improving federal competitive grantmaking is to replace committee judgment of narrative applications with objective evaluation of logical proxies for the purpose of the program. A reformed system will 1) save the government hundreds of man-hours, 2) will reduce the lag between congressional appropriation and program implementation to a nominal duration, and 3) will foster transparency into federal decision-making. ...more »
The key to improving federal competitive grantmaking is to replace committee judgment of narrative applications with objective evaluation of logical proxies for the purpose of the program. A reformed system will 1) save the government hundreds of man-hours, 2) will reduce the lag between congressional appropriation and program implementation to a nominal duration, and 3) will foster transparency into federal decision-making.
There are two fundamental flaws in federal competitive grant-making: A) pretending to have the capacity to measure capacity and B) relying on arbitrary thresholds.
Flaw A: Pretending to have the capacity to measure capacity.
A recent loan guarantee underwriter selection committee spent several days, perhaps weeks, selecting a firm to run a public offering of securities guaranteed by the program. To make the selection, the committee considered two criteria: the cost to the borrowers and the capacity of the firms. The selection of the underwriter should have been a very simple competition to run. Most federal competitions include hundreds of competitors, several criteria, and an abstract objective. The underwriter selection included just six competitors, two criteria (capacity and cost), and the goal was simple and concrete.
As the committee waded into judging the capacity of the firms to run the public offering, it was suggested that the committee establish baseline thresholds that would qualify or disqualify the firms based on objective indicia of experience and that the committee refrain from attempting to assign scores to the firms’ levels of experience. The committee opted instead to employ the standard federal mechanism for judging capacity: assign scores. Scores were based on an objective, first-hand proxy for capacity: market share. This is preferable to the committee basing the scores on an analysis of the narrative statements that were provided in the applications, because narrative statements are a subjective, second-hand proxy for actual capacity. However preferable it may have been to narrative analysis, the proxy for capacity that the committee chose disproportionately rewarded the firms with high market shares in public offerings of federally guaranteed securities, when the proxy scores demonstrated that most, if not all, of the applicants were fully capable of marketing the securities.
The second criterion in the competition, cost, was much more straightforward than capacity. The cost to the borrowers was presented neatly wrapped in an objective, first-hand proxy: the bids as measured in dollars. Upon factoring the two criterion scores together, the applicant with the highest cost to the borrowers and to the federal government emerged victorious as a result of the weighting for capacity, aka market share.
The backward outcome was unsurprising given that the federal strategy for making the selection was akin to asking which fast food chain, McDonald's or Burger King, can better sell 2000 hamburgers. Either of them can do it, so let's certify them and compete the job on cost rather than capacity.
Flaw B: Relying on arbitrary thresholds.
A recent federal competition established a threshold that limited applications to projects that would assist 100 or more residential units. A non-profit organization that held option on a 98 unit facility and wanted to rehabilitate the facility into senior apartments inquired into the purpose of the threshold. The response from the federal government: "You can’t turn around a neighborhood with fewer than 100 units.”
A second threshold in the same competition limited applications to those requesting a minimum $5 million subsidy. As a consequence, an applicant that could rehabilitate 100 properties with $10,000 per unit subsidy must apply for $50,000 per unit subsidy or not apply at all.
Thresholds are absolutely necessary for federal competitions. But arbitrary thresholds are debilitating.
Federal competitive grantmaking should abide this lifecycle:
1) ESTABLISH AN APPROPRIATE PROXY FOR THE GOAL, OR BENEFIT.
2) ESTABLISH AN APPROPRIATE PROXY FOR THE COST.
3) ESTABLISH APPROPRIATE PROXIES TO QUALIFY APPLICANTS.
4) RANK APPLICATIONS ON A COST/BENEFIT BASIS.
5) AWARD FEDERAL FUNDS UNTIL EXHAUSTED.
6) MONITOR THE RECIPIENTS FOR FULFILLMENT OF THE APPLICATION TERMS.
Consider the application of this lifecycle to a prior competitive federal grant.
1) ESTABLISH AN APPROPRIATE PROXY FOR THE GOAL, OR BENEFIT. The purpose of the legislation was to stem the tide of foreclosures and abandonment to turn neighborhoods around from decline toward independent viability. Thus, the benefit that Congress sought was neighborhood turnaround. To measure neighborhood turnaround, it is necessary to establish an objective and first-hand proxy for the concept of “turnaround.” If the momentum driving the decline is composed of foreclosed and abandoned properties, then the natural indicator of turnaround is the return of these properties to productive use. But before this seems too simple, consider that neighborhoods are not all created equal. Ten properties rehabilitated in one neighborhood could stave off disaster, while 100 properties rehabilitated in another neighborhood could have minimal impact. Therefore, it would be inappropriate to measure turnaround purely in the number of properties rehabilitated. A more accurate proxy would be the share of the decline that is put into productive reuse. Still, even this should not conclude the development of the proxy.
Imagine a neighborhood with 500 F/A (foreclosed or abandoned) properties of a total 1000 properties in the neighborhood. A proposal to flip 100 of those 500 F/A properties would reduce the rate of decline from 50% to 40%, which composes a 20% improvement (100/500). Now imagine a neighborhood with 60 F/A properties of a total 300 properties. A proposal to flip 12 properties would reduce the rate of decline to 16%, which also composes a 20% improvement (12/60). Despite having a common share of decline that is put into productive reuse, the two neighborhoods started with drastically different rates of decline (50% and 20% respectively) that cannot fairly be ignored. It would be prudent to account for the fact that the latter neighborhood concludes the exercise far closer to independent viability than does the former, because of their respective starting rates of decline. To incorporate this momentum toward independent viability into the proxy, the share of the decline that is put into productive reuse should be divided by the original rate of decline. The result will closely represent true neighborhood turnaround. For the hypothetical neighborhoods, the final benefit proxy scores are 0.4 and 1.0; that is 20%/50% and 20%/20%.
In a competition designed in this manner the highest scores will be achieved by applications that propose to assist those communities that are closest to independent viability. For example the benefit proxy for resolving 100% of decline in an area with a 6% rate of decline would be 16.7, much higher than the benefit proxy score achieved by either hypothetical neighborhood presented above. In this way, the federal government will subsidize sustainable impact. This reflects wise investment of public funds . . . to a point.
While the federal government should favor tipping neighborhoods into independent viability and disfavor pouring money into cash hogs that will cycle back into decline regardless of the subsidy, the legislation presumes that the neighborhoods that will earn federal investment are those with meaningful decline. Just one foreclosure in a neighborhood of 5000 homes does not threaten a neighborhood’s viability. To account for this, it is necessary to establish a threshold rate of foreclosure and abandonment, based on empirical evidence, that represents meaningful decline.
Identifying thresholds is where the federal government most often resorts to arbitrary assignments. Rather than resort to administrative ease, federal agencies should consult with industry and academic experts to determine the rate at which actual decline takes root. Is it one percent foreclosure and abandonment? Five percent? Seven percent?
A threshold would also be necessary to measure and track progress across the country and to prevent applicants from gerrymandering target neighborhoods to artificially inflate their proxy scores. Given that data collection is carried out on census-based geographic boundaries, it is necessary to define “neighborhood” not as a naturally occurring neighborhood, but instead as a target area comprised of one or more complete census tract block groups. Setting this threshold enables the oversight that is necessary for diligent administration of a multi-billion dollar federal program. It may not be perfect for practical development, but it is a reality of government.
These two logical and necessary thresholds render arbitrary thresholds like minimum grant size and minimum property count unnecessary. And by avoiding arbitrary thresholds, the federal government avoids inadvertently disqualifying wholesale groups of capable applicants and worthy applications, e.g. the senior housing project described above.
It is reasonable to question why the federal government should not explicitly favor rehabilitating 1000 properties of 2000 F/A properties in a 5000 property neighborhood over rehabilitating 10 properties of 20F/A properties in a 50 home neighborhood, by accounting for the count of properties rehabilitated in the benefit proxy score. We don’t need to, because economics does. Economies of scale will naturally favor larger neighborhoods with more properties and the cost proxy will inherently account for the federal preference to do as much and as many as possible. Since both achieve a benefit proxy score of score 1.25, or 50%/40%, it is likely that the former project will be able to seize on economies of scale to rehabilitate 1000 properties for an average cost that is far less than the average cost to rehabilitate ten properties, and this advantage will be reflected by the cost proxy score. If the former project requires greater per unit subsidy, then it will score lower than the ten unit project, as it should. The federal government should direct public resources to investments that provide for the greatest return on investment.
2) ESTABLISH AN APPROPRIATE PROXY FOR THE COST. Establishing a cost proxy is far simpler than establishing a benefit proxy. The cost proxy should be the amount of federal subsidy requested in the application divided by the number of properties proposed to be rehabilitated (to capture economies of scale). The results can then be divided by 10,000 to keep the figures manageable.
To achieve the cost/benefit analysis score, divide the benefit proxy by the cost proxy. For example, assume two applications score benefit proxies of 1.25. If one applicant proposes to rehabilitate 10 properties for $200,000 and another applicant proposes to rehabilitate 1000 properties for $1,000,000, then the respective cost/benefit scores would be .625 and 1.25 (1.25/$200,000/10 properties/10,000 and 1.25/$1,000,000/1000 properties/10,000).
3) ESTABLISH APPROPRIATE PROXIES TO QUALIFY APPLICANTS. To separate qualified actors from unqualified actors and avoid attempting to rank applicants on capacity, it is necessary to establish several thresholds. A threshold will be necessary to qualify for financial soundness, another will be necessary for demonstrated capacity to execute the line of business proposed, and another will be necessary for the ratio of the federal award to the annual operating revenue of the entity, to prevent the federal government from dramatically shifting the competitive positions of private market actors. All applicants, regardless of legal structure or taxing authority, should be evaluated for the same qualifications.
In contrast to the cost/benefit analysis, the qualification of applicants is most appropriately approached as a threshold rather than scored and ranked. Most simply, this is because the federal government should only provide public resources to organizations that clearly possess adequate capacity to substantially execute their proposed programs. If every applicant is fully qualified to execute its own proposed project, then determining which applicant among the pool can better implement its project is immaterial, like asking whether McDonald's or Burger King can sell 2000 hamburgers better.
Similar to the cost/benefit analysis, the qualification thresholds should be based on empirical evidence. The debt to equity ratio, capital reserves, and profit margin that demonstrates financial health should be based on the best estimates of industry experts. The number of years necessary to demonstrate capacity in a certain line of business is industry specific knowledge as well.
4) RANK APPLICATIONS ON A COST/BENEFIT BASIS. Scoring and ranking the applications should occur in real-time, so preliminary announcement of the awardees can be announced immediately upon closing the competition. The applications should be aggregated into a database and the scores auto-populated based on the applicant entries. Awardees can begin planning and executing agreements immediately. Certification of applicant qualifications can occur parallel to creation of the grant agreements and establishment of the lines of credit. Note that despite being auto-generated in real-time, this method of scoring does not fail to account for the factors that are debated for weeks and months by human scorers.
The method inherently captures leverage, because applicants will recognize that their effort to earn the federal subsidy will be aided by capturing and dedicating as much non-federal capital to their proposal as is possible in order to drive up the number of properties that it can rehabilitate per federal dollar. Debates about which commitments count as leverage and which commitments do not become obsolete. The purpose of rewarding leverage in a grant competition is to encourage the federal funds to be used as subsidy rather than capital. This method achieves that purpose.
The method captures capacity, because applicants will be required to demonstrate historic success in the line of business that they propose to carry out, just to qualify for the competition. Capacity typically freezes federal scoring panels in their tracks. Does capacity mean capacity generally? Does it mean capacity with respect to the proposed course of action? Are we grading on a curve, where applicants are relative to one another, or against a set of objective measures of capacity?
The method will capture the extent of need because applicants will be able to do more in areas with steeper decline, up to a critical level. Applications will naturally sprout from geographies where dramatic results can be achieved.
Though the crucial factors remain imbedded in the scoring method, the former prejudices and biases and delays that accompany human scoring are all eliminated. Individuals bring preconceived notions to the scoring table that color every score they assign. These notions have no fair place in distribution of federal funds. A common prejudice is the substitution of narrative style for capacity. While there is possibly tenuous association between these two characteristics, it is almost universally employed as a perfect substitution by scoring panels. The scoring method described above eradicates the undue influence of bias, preconceived notions, and, of course, narrative style.
No longer must applicants put projects on hold for months at a time to determine whether they have earned a federal subsidy. Nor do they have to pour thousands of dollars and hundreds of hours into creating a dense application that serves no lasting purpose. In recognition of the need to respond quickly to changes in market dynamics, the scoring method described above shifts the focus from consistency with an aging plan, to achievement of results by whatever means that local circumstance dictates to be feasible.
5) AWARD FEDERAL FUNDS UNTIL EXHAUSTED. Upon verifying that the capacity submissions are accurate, the federal agencies can execute contracts with the highest scoring qualified applicants. The competition does not conclude here; it commences here. The contracts should have performance targets and compliance clauses that permit the federal government to seize funds in the event that a recipient fails to perform. Funds that are recaptured can be recycled to the next highest scoring qualified applicant.
Awards should not be made for amounts greater or less than the subsidy requested in the application. Awarding more or less than the request amount assumes that the applicant can put the funds to use, without any appropriate assurance. Exactly the opposite should be assumed. Proposed outcomes should be assumed to be infeasible with even $1 less than the amount requested. Adopting a practice of awarding partial requests encourages applicants to artificially inflate the amounts that they request in order to distort their proxy scores. Awarding bonuses encourages applicants to illegitimately participate in a market that does not require intervention. Federal funds should act to fill market gaps, not to supplant private investment.
6) Monitor the recipients for fulfillment of the application terms.
« less full details »