Research about research

We are theoretical researchers. We are scientists, but we don’t work in a lab. Our tools are a computer and a blackboard. People like us invented the computer that you are using to read this text. People like us invented the very notion of computation. We identify and solve problems. We are mathematicians, computer scientists and theoretical physicists.

More often than not, we cannot solve the problems which we pose. More often than not, while trying to crack them, we bump into new, more fascinating questions. It is by following those wild inspirations that we discover anything worth discovering. If we are doing our job well, we should not know what we will be working on in six months’ time.

When we apply for public funding to carry our research, though, we face a big problem: most funding agencies demand a 2-5 year research project. In the grant proposal, we are expected to list all the questions we intend to answer in the next few years.

Of course, that is not how theoretical science works: one cannot “plan” discovering mathematical calculus, quantum cryptography or neural networks. All one can do is create the right environment, and those discoveries will just happen.

Agencies also demand to know our “methodology”. Namely, they want us to explain how we intend to prove this or that theorem. The honest answer is that we don’t know. If we did, the theorem would be proven already, and we would not be applying for funds to crack it.

For the working theorist, applying for research funds is therefore a long and unethical task. It involves concocting an elaborate fantasy where we pretend to know what we are going to discover in the next few years and how. This takes a lot of time away from our research, around one month for the most important grants. Most importantly, it involves lying in an official document.

We have reached this situation because, up to now, research policies have been based more on political fashion than on solid science. To progress beyond this point, we need an open scientific debate on research funding practices, where the scientific method is applied to the problem, i.e., with hypothesis, models, and experiments.

This is what we do in our paper [1]. We propose a research funding scheme by which each research unit (be it a single scientist, a group leader or a whole institute) applies for funding, but does not specify how much. The decision of how much funds (if any) must be awarded to each unit is taken by the funding agency, based on the recent scientific activity of the unit and the prior funding which such a unit was enjoying.

Of course, we work under the assumption that the agency has a clear notion of what it is about science that it wants to sponsor. There are many ways to quantify or evaluate scientific productivity, and deciding which one suits best reflects a political stance towards research. We personally advocate for evaluation methods based on peer-review rather than, e.g., bibliometric data. However, as we show in [1], once an agreed measure of scientific productivity is adopted, the distribution of research funds is no longer a political problem, but a mathematical one.

A mathematical theory of research activity

We start by modeling the research system as a collection of agents or research units, each of which possesses a “scientific productivity function”, indicating how much scientific production we can expect from a given research unit, when we hand it a given amount of funds for research. We allow productivity functions to be probabilistic and time-dependent. They are also secret, i.e., neither the research agency nor the scientists themselves know how they look like.

Scientific productivity of research unit i, as a function of the term funding x. It can be argued that this function (or its average value, if probabilistic) must be approximately increasing and concave.

Relying on our mathematical model of the research activity, we show that there exist systematic procedures to decide the budget distribution at each grant call with the property that the total productivity of the research community will be frequently not far off its optimal value.

The simplest of such procedures is what we call “the rule of three”, by which the funds xk+1 for each research unit i at grant call k+1 are proportional to the research output gik of the unit during the kth term. If the total budget for science during the (k+1)th term is X euros, this means that

The returns of this research policy must be compared with those of “excellence” schemes, whereby, under equal research outcomes, researchers which were funded in the past have a greater chance of receiving further funds. As we show, the latter policies very likely converge to configurations where the total scientific productivity is an arbitrarily small fraction of the maximum achievable by the research system. They are hence riskier than the rule of three.

In [1] we also study to what extent research policies can be cheated by dishonest research units. We conclude, for example, that hacks of the rule of three would require either influencing the evaluation stage or a coalition of research units.

In sum, what we propose is a radical reform of the current grant system. This reform is not a magical recipe for all the problems of scientific research. By itself, it won’t eliminate focus on popular topics, short-term goals, and conservative research. On the other hand, it won’t force theorists to engage in unethical practices, its funding decisions will be transparent and it won’t require the applicant to waste months of working time in writing project proposals. Moreover, we argue, using mathematical models, that it has the potential to steer the scientific community to a situation of maximal scientific productivity.

We hope that our work serves as the starting point for an academic (i.e., not political and definitely not administrative) debate on the way to manage publicly funded science.

[1] M. Navascués and C. Budroni, arXiv:1810.11387.