SBIR and multiple award winners[1]

Charles Wessner and Robin Gaster

May 2008

One consistent criticism of the SBIR program is that some firms “game” the system by winning numerous awards but then fail to commercialize. These frequent award winners (often referred to as “SBIR mills,”) are criticized for using funds that could be better spent on firms more closely focused on a primary objective of the program, namely commercialization of their research.

This critique has gained some traction; some Congressional staffers believe that limits should be placed on the number of awards a single firm can win. NSF firms are limited to four applications annually.

We believe both the critique and the resulting policy adjustments are profoundly misplaced: the problem of the “mills” is indeed a myth, for several interlocking reasons:

1          How many mills are there? And how many awards do they get?

Agencies do not systematically publish their own lists of multiple award winners. And the tech-Net database is not guaranteed accurate, especially as searches by company name may understate extent of award concentration. Still, Tech-Net data provide at least an approximation of the overall situation at all agencies:

 Table 1. Multiple award winners, by number of awards, 1992-2005[2]

 Looking across all five major SBIR awarding agencies,[3] we find that according to the SBA, there were between 1992 and 2005 14,824 Phase II awards made, to a total of 6,386 firms. Two firms received more than 100 awards . Overall , 28 firms received at least 28 Phase II awards during this period – an average of at least 2 per year. Only 15 firms received at least 3 awards per year. [check].

Collectively, the top 20 award winners received 5.7% of all Phase II awards during this period; the top 50 received 13.6%.  In contrast, the remaining 86% of awards were spread across 6,336 companies.

Clearly, some companies do win more awards than others. The most prolific won about 11 awards per year – a sizeable amount, generating revenues on the order of $3.5M annually.

To put this amount into perspective, DoD maintains a database of research, development,. Testing and evaluation (RDT&E) contracts.[4] In 2004, EDS was the top contract winner, with an astounding 10,065 contracts valued at well over $1.5 billion. The top 10 averaged more than 6,000 contracts each, and more than $7 billion in annual revenue.  

2          What commercial results do they generate?

One critique leveled against SBIR “mills” is that these are essentially contract research operations, providing sufficient services to attract more SBIR awards, but never making any real effort to commercialize their work independently of the SBIR program.

This argument is misleading, on two grounds. First, contract research is itself potentially valuable to agencies in meeting their specific research needs – a specified Congressional objective for the SBIR program.  Second, the data show that the mills do in fact in aggregate commercialize.

Over time, some firms have built substantial and versatile research capabilities, and have become reliable research vendors to government agencies.  When called on, these firms can provide specific research in response to closely articulated questions whose answers are needed by the government, especially in the procurement agencies. Some of these answers are extremely valuable, closing off potentially expensive lines of futile research, for example, or providing informed, reliable answers in a timely and cost-effective manner. The legitimate role of these firms in the SBIR program is discussed in section 3 below.

So far as commercialization itself is concerned, the plain fact is this: looked at in aggregate, firms with the largest numbers of awards generate more commercial results, not less, than firms with few awards.

The following table is taken from the DoD CCR database, which closely tracks commercial outcomes related to SBIR awards at DoD and other agencies. It is the most accurate available gauge of outcomes at DoD, and provides some insight into activities at other agencies as well. However, as the “mills” argument is applied most often to firms preponderantly operating within DoD, it is especially relevant her:

Table 2.

Source: DoD Company Commercialization Reports database

These results are telling. They indicate that on average Phase II awards to firms with the most awards in fact generate more sales/revenues per project than do awards made to any other group.

There are a number of reasons why this might be so – such firms are, typically, larger than firms with a handful of awards. They have often been in business longer, and have developed a better marketing network, especially within the somewhat arcane environment of DoD acquisitions.

But the fact remains –according to the survey data, companies with the most awards, criticized for being service organizations not commercializers, in fact commercialize more than companies with fewer awards.

3          Agency mission and knowledge effects: meeting other SBIR objectives

The “mills” argument is also flawed because it takes a single goal of the program as the only goal. This approach has been explicitly rejected by Congress in setting program goals.

Criticism of the “mills” implies that the value of SBIR awards can be judged solely in terms of commercial results. But this is only one objective among the four detailed by Congress in the legislation. And at DoD in particular – which accounts for half of the overall program – a successful SBIR is not one that generates commercial results in the sense of sales on the commercial market, but one that contributes to technology that is eventually built into DoD weapons systems.

The Academy reviewed the relationship between SBIR and agency mission. It focused on three elements:

In all of the Academy’s analysis – and indeed in that provided by those critical of SBIR “mills” - there is no indication that multiple award winners provide less value to the agencies than other firms. If anything, the evidence – both in terms of repeat business (i.e. customer satisfaction) and of successful commercialization – suggests that the agencies are generally satisfied with the performance of these firms.

Beyond agency mission, the Academy also looked at knowledge effects, and found that there is no evidence that multiple award winners generate fewer patents, publications, and other indicators of technical knowledge than do firms with fewer awards, on a per award basis. 

4          Ongoing eligibility

It is worth noting that some of the most prolific award winners are no longer eligible for future SBIR awards. Foster-Miller, for example, the most prolific of all, was acquired by a foreign corporation, making the firm ineligible for future awards.

Firms become ineligible because they are acquired, because they become too large to meet SBA size guidelines for participation, because their ownership structure changes in some specific ways, and they become predominantly institution-owned.

Firms that are as successful as the most prolific SBIR award winners may be especially attractive acquisition targets, as the profile of the program continues to rise, and potential purchasers give more credence to the validation effect that accompanies SBIR awards.

5          Why the cure is worse than the disease

Finally, we should consider possible effects of solutions to the “problem” of the “mills.”

Most proposed solutions are simple – they suggest that firms be allowed to apply (or in some circumstances, win) no more than a set number of awards in a given fiscal year once it has reached a certain threshold of total awards won.

This simple solution will not work for firms closely focused on winning SBIR awards. Firms can easily reconstitute themselves - for example by reincorporating under a different name. They can start subsidiaries. They can subcontract. All of these tools are likely to help determined firms evade quantitative limits.

But such limits are also a bad idea. They raise a stream of questions and concerns:

There are only a few of the practical questions. Failing to answer them effectively could mean that a limit-based rule could have highly negative consequences for the agencies as well as the firms affected.

6          Conclusions

Critiques of the SBIR” mills” are largely based on a misunderstanding of the purposes of the SBIR program, and on the absence of important data. Together, as the National Academy SBIR studies clearly demonstrate, there is no good case for imposing further limitations on access to the SBIR programs for small businesses based on the number of awards they have won in the past or the number of current applications pending. The only meaningful criteria are the needs of the agency, the quality of the proposal, and the past performance of the applicant.

Further, imposing such limits carries with it a significant degree of risk for the program. These limits would likely prove hard to design and harder yet to administer. That is why the Academy has recommended that the only existing limit –imposed at NSF primarily for administrative convenience – be discontinued. And it is why we conclude that such limits are not in the national interest.