Tom Kalil, a pivotal science policy appointee in the Clinton and Obama administrations, recently went on the Hear This Idea podcast, a node in the Effective Altruist network. The episode is well worth a listen.
A couple of items jumped out at me:
Tom talked about building a "marketplace for outcomes." That makes a lot of intuitive sense, particularly with the success areas like rocket launch milestone payments in helping drive down $/kg costs to get to orbit dramatically. I can also see tools like advance market commitments, the policy lever used to great effect in Operation Warp Speed, being extremely effective in accelerating market transformation towards for example zero or negative carbon emission concretes. More broadly though I would have loved to see a discussion of the challenges with social impact bonds and pay for performance frameworks. Those saw a great deal of excitement in certain public and philanthropic circles a decade ago. Actually finding crisp problems that are amenable to clear unambiguous measurement that actually continues to relate to the desired impact long term is rather difficult within the crooked timber of humanity. See Goodhart's law and the idea of cobra effects:
"The results of a perverse incentive scheme are also sometimes called cobra effects, where people are incentivized to make a problem worse. This name was coined by economist Horst Siebert based on an anecdote taken from the British Raj.[2][3] The British government, concerned about the number of venomous cobras in Delhi, offered a bounty for every dead cobra. Initially, this was a successful strategy; large numbers of snakes were killed for the reward. Eventually, however, people began to breed cobras for the income. When the government became aware of this, the reward program was scrapped. The cobra breeders set their snakes free, leading to an overall increase in the wild cobra population.
As an aside, I do believe firmly that the concept has merit very broadly to address public problems; it's still just early. The biggest success of the bipartisan progressive reform movement a century ago, which saw the development of ubiquitous innovations like professionally managed budgets and standard operating procedures for engineered infrastructure can be thought as increasing the rigor of public inputs. Today, with the increasingly acknowledged state capacity crisis and difficulty in building infrastructure, the need to focus on outcomes is extremely clear.
Looking more deeply at how to achieve that goal of focusing on results, measuring outputs along a single metric can be part of the problem. A wise boss and former CFO of a multi-billion public agency once told me that the best measure of municipal management came from the credit rating agencies. Note those don't use a single or fixed set of quantitative metrics but rather a combination of qualitative and quantitative factors through a set of rigorous rubrics and standardized protocols. Something similar would be worth exploring to help realize the promise of measuring outcomes. Here developing a "marketplace for measuring outputs" would be a requisite first step. Today we live in a golden age of computational social science and there is a vast frontier to for institutional innovation, incentive experimentation and opportunity to be creative with bounties, micro-grants, quadratic funding and the like. Here is a great vision in that vein from Steve Randy Waldman writing on Interfluidity:
Ultimately, we should want to generate a reusable, distributed, permanent, and ever-expanding web of science, including conjectures, verifications, modifications, and refutations, and reanalyses as new data arrives. Social science should become a reified public commons. It should be possible to build new analyses from any stage of old work, by recruiting raw data into new projects, by running alternative models on already cleaned-up or normalized data tables, by using an old model's estimates to generate inputs to simulations or new analyses.
I also firmly agree with Tom that there's a huge need for translating insights from the realm of data or digital tools to on the ground positive impacts that address civic challenges. There's a lot of opportunity for scaling up many of the successes from the Obama era in public technology across the country with creative bottom up levers. There is tons of funding for better technology and now there is a golden opportunity to reallocate those public dollars where cities, states and other subnational agencies spend way too much money on sometimes truly terrible digital technology. As a general rule its suboptimal and as civic tech's fairy godmother Jen Palka writes, the vendor ecosystem still largely hasn't changed that much.
The supplier base for government tech, for example, is not all that different from when I started Code for America in 2010. Anduril has made strides at the DoD for sure, and some startups have done some interesting stuff. Companies like Nava, a public benefit corporation whose costs and outcomes in contracting with agencies like CMS and VA should make it a strong choice across civilian government, is now over 500 people. But CGI Federal, just to pick one of the incumbents, is 90,000. Lockheed Martin is 122,000. Anduril is 3,500. It’s not like we’ve seen some massive shakeup.
Here Tom's points about policy entrepreneurship and incentives are particularly powerful for civic tech, which largely has focused on programmatic areas like bringing tech talent into government.
The podcast touches on lots of other items of interest such as metascience, challenge prizes[1] and how to have agency in influencing positive impacts as a person without formal authority. Recommended!
[1] As an aside, I also loved the book Longitude. Unlike Tom I was in I believe middle school when I read it. Hearing his story about how that catalyzed challenge.gov is super cool and inspiring!
