Sound policies are based on sound evidence, and nowhere is its definition, procurement and interpretation more complex than in the field of Culture, where the benefits of a good decision are outweighed only by the risks of making a bad one.
Sarah Thelwall’s ‘Inside-out’ methodology aims to unpack the ways in which financial data is aggregated, providing a simple typology which sets out the uses and limitations of different data aggregation methods and identify the appropriate math for the various end uses of the data. This apparently simple proposal is disconcertingly novel: The ‘Outside-in’ approach which constructs aggregate performance indicators without due attention to the actual structure of cultural provision and production is all too often a path of least resistance. Moreover, the proposal has wider implications for the use of data which, it is to be hoped, will prove both salutary and instructive.
Perhaps the top requirement, for evidence to be useful, is its relevance. Having known and worked with Sarah for many years both during and after my time with the Economics Intelligence Unit of the Greater London Authority, I have never ceased to be impressed with her ability to tease out, through direct interaction with artists, agencies, and providers, exactly what kind of data arts enterprises need to know in order to evaluate and benchmark their performance and plan accordingly.
Sarah’s new report takes this work to the next stage with a simple question: how should that data be interpreted? It is a well-known truism that data is not the plural of anecdote: equally true is that information is not the singular of data. The reductionist temptation to reduce a large mass of data to a single, catchy, one-dimensional indicator – be it box-office, attendance, profit, GDP, or any of a familiar range of single numbers – besets the world of arts just as much as the world of economics. However, precisely because of the complexity and variety of the variables that affect, and represent, what an artist does, the scope for reductionist-induced damage is
probably greater in the sphere of cultural management than anywhere else.
One not-uncommon reaction is to eschew measurement of any kind, relying on intuition, experience and judgement in preference to numbers of any kind. Certainly, expertise has its place in arts management and may even be the primary condition of success. However, we live in a world in which, whether we like it or not, numbers are used. Granting authorities, policy-makers and many others work with a vast mass of numerical indicators, and are hard put to make the best use of them. Sarah’s simple point, illustrated with examples which any intelligent administrator will be able to follow, is that these data have structure: they refer to enterprises,
and activities, which are differentiated into various categories – for example, those that rent space and those that do not. The diagnosis, and recognition of these categories is essential, in order to make effective use of the numbers, and – Sarah shows – can be achieved as often as not by simply interrogating the data properly.
A simple point, but not an easy one, either to operationalise, or to recognise. It may be argued that correct classification does not constitute policy-making nirvana, and this is true. Classification is also a mundane activity which is easy to dismiss as an obsessive and arcane substitute for simply making good judgements. Yet many of science’s greatest achievements are arguably rooted in, and certainly preceded by, adequate classification: it suffices to recall Mendeleev’s table of the elements, or Linnaeus’ system of species.
This report is small, and hence easy to digest. Big things, sometimes best, however come in small packages. This is one of them.
Research Director, Geopolitical Economy Research Group, University of Manitoba