Eradicate teaching evaluations

Following the document by Angela Jancius on "Battling the Neoliberalization of University Life" which I reported in my previous post, I would like to highlight how the performative thinking, which pervades academia in UK, is rooted in corporativism.

Indicators of teaching performance stem ideas of performativity, breaking solidarities and local systems of trust. Such indicators may come to be used predominantly for image management and protection of interests at various levels in the organization by the elite caste, rather than for genuine and tangible student benefits. As such, they must be eradicated.

Audit systems do not empower students and lecturers in taking control of the curriculum, its meaning, content and the student-academia relationship, but instead serve only as control mechanisms for the elite caste.

It is a usurpation driven by the consumer paradigms emerging in HE, the marketisation and massification of HE underpinned by the free-market ideology embraced by the UK government.


Below is a critical review of the methodologies currently employed in the universities in the UK for assessment of teaching performance.

Long post!

***

From places for the intellectual elite to preserve their culture, universities have become centres for mass education. Much debate has ensued about whether the growth of flexible organisations has served to reinforce distinctions between elite and mass HE and their respective collegial and managerial cultures in the UK.

With primary functions as centres for cultural production, for producing epistemological research and for the provision of training to meet changing occupational needs, the societal role of HE institutions is reflected in the official DfES perspective, as a resource for competitive knowledge societies exemplifying an overarching commoditised, ‘enterprise system’ of education in which changes in economy drive developments in education. Indeed, following the alignment of Universities in the UK with the private sector the overriding demand is to improve efficiency while maintaining quality wherein HE must bring return on investment to society.

This leads to a chase for the best means of creating and measuring desired outcomes and a conceptualisation of roles whereby students become consumers and lecturers a resource for the institution.

Changes in the discourse on quality have also forced universities to become increasingly competitive and performance driven and respond to employer requirements. Post-’92 universities in particular must rely on HEFCE allocation of funds, enjoy less autonomy than older universities and strive constantly to demonstrate return on investment to stakeholders.

A changing government agenda has thus meant a redefinition of HE management and quality of education, its control, measurement and value, with the 2004 Higher Education Act citing further themes for improvement.

Quality Assurance and Improvement
A macro-level analysis reveals emphasis on accountability and efficiency within the agendas shaping strategies for HE management, with the emergence of a new language of audit aimed at instilling private sector values through the technology of accountancy.

Since the 1980s, the focus of educational effectiveness and QA has moved towards outcomes like excellence in learning and teaching, employability, widening participation and equal opportunity, while government initiatives have devolved control of the sector to agencies including the QAA, to which universities are accountable via processes of audit and inspection.

Attention is drawn towards to the interplay of external policy and institutional aims in setting agendas for educational quality and improvement. In particular, quality assurance and enhancement should be built upon specification of educational purposes, aims, objectives and standards and depend on appreciation of the underlying context and meaning of quality underpinning external methodology as well as of how people learn, interact, sustain, develop or destroy a culture.

Researchers have accordingly analysed various conceptions of quality which might inform QA policy with value-for-money and fitness-for-purpose most associated with the QAA’s key processes. While the former is seen in measures of accountability achieved through QAA assessment and audit processes, fitness-for-purpose is instead associated with processes such as the validation of courses and with the notion of ‘prospective QA’; in other words, improvement.

This more formative approach is concerned with reviewing how well the entire institution works in achieving its missions, as opposed to meaninglessly quantifying aspects of the system.

However, doubts are raised about prevailing processes for measuring performance against each institution’s stated objectives, in examining how mission statements have become marketing tools rather than statements of strategic purpose which can be realistically obtained.

Indeed, QAA audit and assessment are discussed elsewhere as the exercising of power from above revealing their values to be in contrast to those supporting educational development and their policing methods to be of threat to quality, or able only to institutionalise compliance and accountability.

Educational improvement is thus linked to commensurable increase exemplified by permanent struggle, elusive satisfactory goals and ambiguous measurement procedures. With this come themes of imperialism and domination towards HE, with quality effectively colonised by consumerism and short-term effectiveness. This raises lingering questions regarding what is actually to be improved, in what ways and for whose benefit and shifts focus towards instilling a quality regime and culture based on improvement, self‑regulation and meaningfulness for its participants.

Strategic Management and quality assurance
Strategic Management is an integrating mechanism in modern Universities. Activities of an organisation must be considered in light of the context in which it operates, plus all the mechanisms available in order to achieve aims and steer towards opportunities for improvement. In the case of a university, the main strategic aim of educational improvement should be analysed within the context created by the QAA policies and the increasingly competitive environment, leading university executives to adopt strategic management as in the private sector. Strategies and policies aimed at the enhancement of the quality of the provision of an academic institution should thus be read in concomitance to the overall organisation development aims and strategies.

Institutional change is thus linked to wider changes in society at a national and international level to the point that it becomes difficult to discern which shapes which. Indeed, suggestions that managers shape quality policy in universities are questionable under the spectre of QAA operations.

The academy is increasingly paralleled to an enterprise system, responsible for transforming a variety of inputs (e.g. students’ time, teachers’ time, consumables, equipment and buildings) into knowledge products, usually in the form of qualified people and intellectual property.

Good quality educational provision should as such transform students into well qualified individuals, although this correlation is not straightforward and the mechanisms of such transformations are often obscure. They are, however, ‘tuned’ by busy university executives who see the educational provisions of their institutions at the service of preset agendas, like employability.

This transformational model of HE (input-processes-output) in the UK amplifies the importance of measures of effectiveness and efficiency in the new neo-liberal enterprise systems of HE; accommodating the idea that processes targeted to the enhancement of educational provision can be subject to the law of reason and as such, efficiency and effectiveness can be measured by grade descriptors or indicators at different points in the transformation process. Enhancement of the processes is carried out after an evaluation phase and is characterised by various rationalisations of the system. This all assumes that input, processes and outcomes in educational organisations are well-defined and tightly coupled with one another, requiring a rational-technicist approach to the structuring of decision-making.

Power, control and performativity
Influenced by the performative thinking evident within external quality policies, the development of indicators of teaching performance and the language in which they are couched becomes institutionalised. Consequently, such indicators may come to be used predominantly for image management and protection of interests at various levels in the organization by the elite caste, rather than for genuine and tangible student benefits.

This finds resonance with criticisms of audit systems in HE as ‘political technologies’, which redefine social relationships in terms of power sharing and control between scrutinizers and observed. Processes of auditing, that is evaluating whether or not targets are met, evoke ideas of scrutiny, examination and inspection as a public event, which may not be concerned only with the quality of performance.

Instead, such processes may well be concerned only with the systems that are in place to govern quality, by displacing local structures of trust. Consequently, any form of local organization, where individuals can autonomously and professionally engage in a discourse on quality, is unthinkable.

Student Evaluation
Student evaluation of teaching is a method used throughout HE institutions to assess teaching quality, which should identify areas for improvement to the educational provision after evaluation. By identifying areas of curriculum, teaching or assessment which may need improving it is possible to have an impact on the overall perceived or actual quality of the educational provision. Such evaluations tend to focus upon systematically measuring satisfaction with teaching methods, materials and resources, often using questionnaires. Accordingly, their adoption has been most heavily informed by quality audits and teaching quality assessment exercises required by external agencies. The data gathered is often used as an indicator of educational quality, primarily through evaluating the performance of teaching staff.

At various universities, students participate in course evaluation by completing questionnaires at the end of a module or course. The feedback gathered is then scrutinised by School managers with summaries later provided to teaching staff and students. The evaluation of student satisfaction is also carried out at institutional level with a student end of year survey, which gathers evaluations on teaching and resources/facilities quality perceptions. In this way, macro policies can be translated into micro practices with performance criteria centrally-determined.

Such performance indicators provide grounds to managerial imperatives for the facilitation of a culture of commitment to them. Such a strategy of introducing arbitrary performance indicators internally, in contraposition to the externally determined is, a strategy of domination in the assessment of teaching quality, justified in terms of the open-endedness of continuous improvement.

Moreover, it should be noted that, although post-course evaluation questionnaires have attracted the nickname ‘happiness sheets’ because they tend to encourage students to provide the information students believe staff want to see, aware that their feedback can have no impact on improving their own studies, many universities continue to employ such methods within their strategies for educational improvement. To make matters worse, universities do not appear to have in place mechanisms to counteract the tendency of such evaluations to measure charisma and the inseparability of student feedback from individual learning conceptions or to put into context comments which may be either superficially optimistic or unfairly critical towards individual teachers.

There are indications that the data obtained from the questionnaires may be used to exercise subjugation and punitive oppression towards staff who do not fit the standard profile and expectations of the University’s management; who may be ‘maverick but effective performers’.

Certainly, inappropriate use of such evaluations to evaluate performance of teaching staff and identify poor teaching for remediation has led to suggestions that student evaluations, which, as a quality measure, are predominantly driven by bureaucracy and ideas about what quality represents rather than by educational concerns have lost their power to instigate educational improvement at all. Their value in bringing about worthwhile change to the curriculum, rather than merely identifying under-performers, is in this light questionable.

Further, student views should only be used as part of a dialogue aimed at reviewing the curriculum and under no circumstances should they be used to judge the personal performance of individual members of staff, it is difficult to see this in practice.

Other criticisms regarding the ability of lecturer behaviour or course characteristics to provide valid indicators about student learning in light of the nature of teaching as multidimensional and of learning as problematic, uncertain and relative, contribute to doubts that such procedures can capture the subjective aspects of quality and learning required to develop teaching practice.

Such summative approaches are of little value for teaching compared to alternative formative methods that could be adopted, aimed at informing individual teaching practice.