Mr. Test Writer,
Does your test really test what it should test?
It is all apparent that testing in general and language testing in particular in Nepal have been plagued with a bundle of issues. Some of the blazing issues that call for immediate intervention include memory and content based (English language) tests, slipshod marking system, lack of post result analysis, derogatory influence of exams on education, guess paper based exams, abolishment of entrance exams, issue of remarking papers, escalating cheating and so forth. Unfortunately, exam authorities have turned deaf ear to the issues. I put a few questions before Professor Tirth Raj Khaniya, a well known expert on the subject of testing if his visionary perspectives and long gained expertise could offer some guidelines to trim them down. Here are his insights and observations in his own words.
Please also feel free to put your comments, no matter you like or you feel difficult to digest the views expressed in the interview.
1. Professor Khaniya, you have been involved in testing and evaluation for a long time. We do acknowledge your contribution in exam reforms in Nepal. However, if you look at the current trends in testing in Nepal, are you satisfied with what and how we have been testing?
Your question reminds me of the time when, after completing my PH. D. from University of Edinburgh, I was flying back to Kathmandu from London in the then RNAC flight thinking of how I would be working on bringing about reforms in the SLC Examination in particular (my area of investigation) and examination system of Nepal in general. As a matter of fact, I was able to influence some authorities involved in the management of the SLC exam to initiate some reform activities. I must say that there have been several changes in the SLC exam since then. However, I should also confess that I have not been effective in bringing about the reforms I wanted to bring about in the SLC Exam. What it is that I wanted the SLC to be is both an exit exam, as it is now (administrative), as well as a research process (academic) to bring about reform in the whole school sector of education. What it means is using the SLC exam as evidence of school performance and post analysis of the exam would give us sufficient information about what kind of intervention is needed in the school sector reform so as to make the system better. Doing this every year would obviously build the system better. It is mainly because of our failure in utilizing useful information available from the post analysis of the academic aspect of the SLC exam, even after spending a lot of money on school education, accomplishment is very little.
In response to your query, I must say that my academic pursuit forces me to work on making examinations as powerful instruments for causing learning, which is possible only when we produce good exams, but the situation in which I and we have been working leads me/us to be with the community that understand(s) exam only as an administrative tool. In that sense I have not been able to accomplish my mission so far. I will continue working towards this direction. The new generation is very positive about such changes. I am optimistic about it.
2. When we look at the tests made in Nepal-whether be them for the English language or for other subjects/disciplines, we find most of the questions can be answered only if students have committed the contents well. Are we supposed to test that at all?
You have rightly touched on the main problem of testing in Nepal. As a matter of fact, we do not care what questions we ask, what answer our questions will stimulate the learners to produce, how to develop a question, how a learner will answer our question, etc. Especially in language testing, asking a question is more difficult than answering it if we really appreciate the art of questioning.
In response to your concern, I would say that the whole lot of the people involved in testing in the present set up does not have adequate orientation to testing. Their main source of knowledge about testing is that they took exams during their study period. But in modern education, testing has become an independent discipline and thus people who want to work in the field of testing need training on it. Without training they cannot appreciate that a question cannot be a question simply because of its structure and a question mark, rather the one who asks a question needs to know what he is asking for (i.e. the expected answer or response or performance). On most occasions, I have seen people setting questions in such a way that there is no way for creative and innovative answers. To give you an example, in most question papers on the top, you find the statement saying “Candidates are required to give their answers in their own words as far as practicable” whereas you do not find even a single question in the paper not asked in the previous exams. How can the teacher expect original answers from students when he has not used even a single word in the whole set of question paper on his own? The argument is that in order to stimulate students to produce creative and innovative answers, you need to ask such questions and this is possible if testing procedures (e.g. specification) are truly followed.
3. You have written a book entitled ‘Exams for Enhanced Learning’. Contradictorily, tests are criticized for exerting deleterious influence on education and for not being able to assess the actual competence of a person. If so, why should test exist at all? Do not we have any alternative to it?
In my views, what we test, if done professionally, represents what we want our learners to learn. It makes the learner clear what they should strive for. In addition, testing is associated with intrinsic and extrinsic values which make the learner work hard. The combination of all these has a lot to do with how teachers prepare students for testing, how students prepare for it and how parents support them for getting through it. Then if we have good tests, working for such tests will enhance students’ learning. By the same token, if it happens to be a bad test, students are bound to suffer from its derogatory influence. The argument is that derogatory influence of an exam is not inherent, it depends upon whether the test is good or bad. A good test allows students to deal with what they are supposed to learn, and through testing their learning becomes powerful.
The main crux is what we ask students to produce and how we want them to demonstrate. Since a human brain is intelligent, and can tell a lie, as you said, it may be difficult to test real competence. However, a good test can lead a learner to truly perform tasks which can be accepted as an indicator of the learner’s competence.
My argument about exams is that at least for a foreseeable future, exams are likely to survive because even the advanced technology could not find a substitute for an exam. If so, why not prepare for using the power of exams which leads towards better learning, and it is possible if we design good tests
4. Many allege that exams in Nepal have virtually remained guess-paper based? Why is it so? How can we check it?
As I said above, students’ behavior in an exam is shaped up by the questions we ask. When we ask questions borrowed from previous exam papers, commercial notes and guide books, we are forcing the students to be guess paper based. We can change the situation only when we make our students know that questions will not be based on market materials, and marking will be done on the basis of a marking scheme not on the basis of the answers copied from different sources.
5. Specifically, what are English language testing issues in Nepal? What efforts are to be made in order to check them?
There are several issues in language testing in Nepal. One of the major issues, for example, is that we need to be clear about what we want to test- language or content. For me it is language not content. If we want to test language we should not put a pressure on the students about contents. Content in language is merely a way of eliciting language. Lack of knowledge about this concept makes the whole process of testing complicated. We do not understand the difference between asking Nepalese students to write on Pashupatinath and The White House. When students do not write, we do not know whether it is because of lack of knowledge or language. In language testing we need to make sure that when students do not give us proper answer it is because of language not because of content.
6. Particularly what tips or suggestions do you provide to test writers so that they unveil creative and critical abilities of the students?
What we need to understand is testing should not be treated as an isolated activity; rather it is a comprehensive activity. What I mean by this is that testing cannot be improved only by changing tests. Before we produce a changed test in an exam hall, we need to present it to the classroom in the beginning of the academic year so that the teachers and students understand what they are expected of; be it creative or innovative or practical or something they need to demonstrate. Once students and teachers know what they need to demonstrate in the exam, they will work for that and when what we want them to learn and what they need to demonstrate in order to pass an exam become the same, then we can concentrate on strengthening creative and critical abilities of the students through and for the exam.
7. Language Testing has been established as a separate discipline elsewhere and that has systematized the evaluation of language learning from perspectives to practices. What about ours? Do not you think we need to make it more organized here in Nepal?
Everywhere language testing is emerging as an independent discipline. In a short period of time, let’s say, after 1960s, it has grown in such a way that it is like any other disciplines which have long history and rich literature. Nepal’s case is the same. When we were students in Nepal, there was no concept of language testing. We were trained through measurement and evaluation concepts under pure education courses. Now we have a 50 marks’ course for language testing. I agree with you that we need to do more like forming language testing groups, publishing language testing journals, organizing special activities for training and sharing focusing on language testing, etc. The increasing popularity of language testing is creating sufficient spaces for doing what you are proposing.
8. Often fingers are lifted on the marking system in Tribhuvan University and elsewhere. Many departments have been set on fire alleging subjective and careless examination of the papers? What do you have to say on this?
You are right. Our department was set on fire two times during my headship. I see problems not only in marking, but in the whole process of enrolling students for teaching and testing. Recalling that time, I think the students who were serious and regular in the class were not the ones who damaged the department. Those who led the vandalism were guest students but strong enough to exploit the situation. Saying so, I also agree with you that our testing involves subjectivity and carelessness. We need to provide some sort of professional development courses to all who are involved in teaching and testing about language testing. We happen to wrongly believe that good teachers are good testers. Once questions are developed professionally, many anomalies can be sorted out
9. Students of TU and Higher Secondary Board have expressed their disgruntlement over the marking system time and again. Universities in the world do have the provision of remarking system. Do not you think TU should introduce this system at least for the students’ satisfaction and also to show the fairness of exams?
Many universities have a provision for an appeal when students do not accept results for any reason. In order to satisfy them, the provision for remarking is a way to give them justice. There is another advantage of remarking, that is, making teachers feel responsible for what they do. When teachers know that they are not the final authority for marking, when they know that the answer sheet may be examined again, they would to be more serious and do the job sensibly.
10. Tribhuvan University abolished entrance exams in some disciplines? How justified was the decision? Is there any university in the world which holds open admission?
It is unfortunate that the TU abolished entrance exam in some disciplines. As a matter of fact, it comes under university’s autonomy- a university making decisions on who to be allowed for admission and who not- my argument is, it is a matter of deciding on the quality of students for enrollment. When a university has no say over what kind of students it wants to invite and select, how can it talk about the quality of its product? Our student leaders also feel proud being able to allow those who do not merit for admission. They do not understand how it damages university’s credibility. In my view, this decision should be revisited.
11. Last but far from the least, cheating has been rampant in Nepal-whether be it SLC or Higher Secondary or even exams of university? It has become a matter of headache for one and all. Why do you think has it happened? What do you think could be the best solution to check it? Can anything be done on the part of test writers to check cheating?
I am of the opinion that cheating is a byproduct. Students go for cheating because cheating is possible in the exam, they have seen their friends passing exams by cheating, questions are asked based on previous exam papers and published materials available in the market, etc. If we do not ask cheatable questions, if we effectively communicate to them that cheating cannot lead them to pass an exam, if we ask them questions answers of which cannot be supplied through cheating, and if we make them that cheated answers would not be awarded marks, I think they would not cheat.
Yes, you are right, we can stop cheating by improving how questions are asked and how answers are expected. But this has to be clearly communicated to teachers and students and classroom teaching has to be improved.
Thank you so much for illuminating us with your viewpoints.
About Professor Khaniya
Tirth Raj Khaniya is a familiar name in the arena of Nepalese education and ELT both. A Ph. D. in Language Testing from University of Edinburgh, UK. Dr. Khaniya led several examination and education reform taskforces. Currently, a Professor of English Education, he teaches language testing in the Department of English Education, TU. To his credit, he has a number of books and articles published in national and international journals on Nepalese education and ELT. New Horizons in Education in Nepal (2007) and Examination for Enhanced Learning (2005) are his highly acclaimed works.