ERA “not” arbitrary or anti-academic

The launch of the inaugural ERA national report has generated much discussion in recent weeks. I expect nothing less. Good public policy stems from a robust, respectful and open exchange of information. Good research is no different. And that is the philosophy on which ERA was built.e you to make the case. Some criticisms I must firmly reject. I cannot accept, for example, that ERA is arbitrary or anti-academic. Any researcher who has sought tenure or applied for a grant knows evaluation is a fact of life. Nobody ever won an academic appointment or promotion without having their work weighed against the standards and norms of their discipline. And it is precisely those standards and norms that ERA employs to gauge research quality – tailored for each discipline, and evaluated by expert scholars. ERA depends on the very means of comparison that disciplines use themselves, and presents the information in a fair, accessible and systematic manner. It is equally remarkable to me that ERA is condemned for being retrospective. ERA is not a road-map. It is the foundation on which a good road-map can be built. Without the famous crystal ball, we cannot assess future performance. But we can use past performance to set future expectations – and we will. I must also reject suggestions that ERA lacks the benefit of international expertise. International consultation has been prioritised from the outset, and that is reflected in the data ERA gathers together. For 101 of the 157 fields of research, international citation rates supplied by a multinational publisher were used as world benchmarks. This is a standard and validated form of assessment in these fields. For the remaining 56 disciplines where citation data are unavailable or unreliable, more than 500 experts from Australia and overseas reviewed a sample of publications (selected by universities) and then rated them according to criteria describing quality in terms of international comparators. International members were represented on committees, and leading international experts offered their advice in the development of the ERA methodology. Having said all that, I do see several areas where we can improve the instrument, in consultation with the research community. Many criticisms to date have centred on ERA’s disciplinary focus, and the difficulty of capturing interdisciplinary research within that framework. I am a strong advocate for collaboration across research fields, and I am determined to give those who undertake it the recognition they deserve. The first round of ERA has allowed us to gauge the volume of research in danger of slipping through the cracks. As the national report shows, the extent of interdisciplinary collaboration varies across research fields. The links are already deep and complex. Our challenge is to work with researchers in each discipline to respond to that complexity – without constructing an expensive and onerous regime that serves nobody’s best interests. I am also conscious of the need to continually improve how we define and measure research outputs. The strategy we adopt should provide clear incentives for researchers to undertake world-class research – whatever form that might take. Work that looks good on paper should also be good in practice. The ERA 2010 initiative used a range of indicators and proxies to support the evaluation of research excellence. One of these indicators was tiered outlet rankings, including ranked journals for all disciplines, and ranked conferences for several disciplines where conferences make up a large proportion of research output. I emphasise that these are far from the exclusive ratings measures used in the 2010 report, and arguably in some fields they were less important than other established measures. It is vital to ensure our ranked journal list stands up to scrutiny. That is why the ARC sought sector input through several phases of direct consultation with more than 700 discipline experts and representative groups. The four Learned Academies and various discipline peak bodies developed the original draft list in late 2007. The ARC also conducted two phases of public consultation in June 2008 and August 2009. In the end, the ERA 2010 exercise collected data on 333,000 research outputs, the most comprehensive list ever built. We have invested heavily to get this right, and we will renew consultations in advance of ERA 2012. And I will welcome reasoned suggestions to incorporate further indicators. Additional indicators for applied research are matters for discussion, for instance, although I note that even on current indicators many disciplines with a strong applied focus have performed well. Any new indicators must be able to tell us about research quality – not just research activity, and not just raw uptake of research outcomes, but quality. ERA is all about driving up the standard of research, and not promoting busywork, rewarding expensiveness, or providing incentives to shift mediocre product. I am proud of what we have achieved thus far. It would not have been possible without the goodwill of the university community. I thank you for sharing my commitment to excellence.