"As a young academic, I am reliably informed that the landscape of scholarly communication is not what it was 20 years ago. But, despite all that has changed, it seems that we still largely rely upon the same tired and narrow measures of quality and academic impact - namely, citation counts and journal impact factors.
As someone who has used the internet in almost every aspect of their academic work to date, it's hard for me to ignore the fact that these mechanisms, in predating the web, largely ignore its effects.
By holding up these measures as incentives, we appear to have our eye firmly fixed on the hammer and not the nail, adjusting our research habits in order to maximise scores and ignoring issues such as why we publish in the first place."
(Matthew Gamble, 28 July 2011, Times Higher Education)
"The European Academy of Design took a pioneering step in their 1999 biennial conference by including an exhibition of 'practice-based research'. This was refereed in the same way as conventional papers and a number of interesting exhibits were produced, demonstrating a diversity of work and connections between the methods and aims of the exhibitors and those of conventionally published research. In fact the conference award for 'best paper' (on a vote by all delegates) went to one of the exhibitors. Unfortunately the EAD exhibition did not result in a permanent record of the research thus 'published' so the exhibits did not contribute to the recorded body of knowledge and provided no exemplars for future researchers.
A further problem with the EAD exhibition, held in England, was that all the exhibits originated in the UK. Given the difficulty of transporting exhibition materials over long distances, it was reasonable to assume that the format inhibited international contributions and this was reinforced at the 2001 EAD conference in Portugal where exhibits were invited but only one was forthcoming (a graphic design exhibit from Australia) possibly because the ideas of practice-based research were less prevalent in the host country.
Against this background, the Design Research Society decided to include an exhibition in their 2002 Conference, 'Common Ground', held at Brunel University in England. This was an experimental activity and there was uncertainty about whether suitable research exhibits would be forthcoming, how to referee them and how to provide a permanent record. However it was felt that this experiment needed to go further than the preceding EAD venture and make a permanent contribution to our understanding of this form of dissemination. "
(Chris Rust and Alec Robertson, 2003)
1). RUST, C. and ROBERTSON, A. (2003). Show or tell? Opportunities, problems and methods of the exhibition as a form of research dissemination. In: Proceedings of 5th European Academy of Design Conference, Barcelona, April 2003.
"The motivations that lead researchers to publish in different formats – particularly in scholarly journals – differ significantly across disciplines. Researchers in the sciences are more likely to see publication in a learned journal as a ‘natural’ means of communication with their desired audience, while their colleagues in engineering, the humanities and the social sciences are more likely to see it as meeting essentially external requirements for research assessment and career advancement.
In these latter disciplines, therefore, the rise of journals is more closely associated with an environment where there is increasing emphasis on measuring, assessing, and evaluating research, its outputs and impact."
(HEFCE on behalf of JISC, UK, 2009)
"The proposals for a new approach to the assessment and funding of research - set out last year in the Higher Education Funding Council for England's consultation paper on the research excellence framework - have sparked more than a few rows.
Much of the conflict has revolved around whether or not the economic and social impact of research should feature in the regime that will replace the research assessment exercise. ...
Our starting point should be to remember that the RAE was deeply flawed. It was dominated by vested interests, was embarrassingly subjective and seriously undervalued those scholars who bridge the worlds of academe and practice.
The REF is, then, a major step forward from the RAE not least because it broadens the definition of research. To suggest, as the REF does, that research is 'a process of investigation leading to new insights effectively shared' invites all scholars to think afresh about how they communicate their research findings and to whom. ...
Yes, there are challenges in research impact assessment. New thinking, around, say, research 'possibilities' is needed. But once academics recognise that research findings should be 'shared', we have made a significant step forward. By definition we are now discussing research impact or, at least, potential research impact.
However, the intellectual argument relating to research impact, rather like the debate about the expansion of university public engagement activities, goes much deeper than a discussion of how scholars can improve the manner in which they communicate with different audiences - important as this is.
Rather it concerns a reshaping, for some disciplines at least, of the way scholarship is conceived. It heralds a move towards the notion of 'engaged scholarship'. Many UK academics - medics are a classic example - are already actively engaged with stakeholders outside the campus in the process of defining research questions and co-producing new knowledge.
This is not to suggest that all scholars should be 'engaged scholars' - indeed, that would be a bad thing. But the research impact debate can open up the possibility of broadening the definition of scholarship."
(Robin Hambleton, 4 February 2010, Times Higher Education)
"The Research Excellence Framework (REF) is the new system for assessing the quality of research in UK higher education institutions (HEIs). In previous years, research quality has been assessed periodically through the Research Assessment Exercise (RAE).
We are currently consulting on proposals for the new framework. We will issue guidance on the REF in 2010 after completing the consultation. The first REF exercise is due to be completed in 2013.
We are working in collaboration with the Scottish Funding Council, the Higher Education Funding Council for Wales, the Department for Employment and Learning (Northern Ireland), and with a wide range of stakeholders to develop the new framework. Representatives of the four funding councils sit on a steering group which oversees its development.
The REF will focus on three elements, which together reflect the key characteristics of research excellence. These are:
* Outputs: The primary focus of the REF will be to identify excellent research of all kinds. This will be assessed through a process of expert review, informed by citation information in subjects where robust data are available (for example, in medicine and science).
* Impact: Significant additional recognition will be given where researchers build on excellent research to deliver demonstrable benefits to the economy, society, public policy, culture and quality of life. Impacts will be assessed through a case-study approach that will be tested in a pilot exercise.
* Environment: The REF will take account of the quality of the research environment in supporting a continuing flow of excellent research and its effective dissemination and application."
(Higher Education Funding Council for England)