Testing the Tests

In a couple of weeks, we will witness the largest gathering of humans at the Kumbha Mela at Allahabad, and in a few months after that we will be witness to the greatest ‘Testing Tamasha’ of the world. Millions of young learners will be taking numerous tests, ranging from engineering, medical to admission to various courses of various Universities. These will lead to joys of success, disappointment of failure, some will feel that their life is ruined and a few driven to suicide as well.

An important question that arises is how accurate and reliable are these tests? What are the attributes of a good test? Should there be an element of accountability of test administrators. As things stand, protecting the secrecy of the test papers and confidentiality of the evaluation system is considered most important and various provisions of the Indian Penal Code are invoked occasionally, but the exam and test providers are not required to live up to any standard of relevance, quality and external review. The entire principles of external verification and working to specified quality standards stand totally abdicated. This is true almost all over the world for most of the tests.

The canonical principle adopted is that if a number of candidates are asked exactly the same questions and required to answer them without any help in the same prescribed time, under proctored conditions, then those who can answer the most to the satisfaction and expectations of the examiners are the best. However the fundamental question of whether the attributes and qualities being sought to be tested have value, and more importantly what is the degree of accuracy and reliability of the scores assigned to Individual test takers.

Almost 4 decades ago in a book on evaluation by Edwin Harper, he summarised the results of some interesting research in how examiners evaluate the scripts assigned to them. He reported that there is a wide variation in the marks given by different examiners to the same script, that led to the conclusion that ‘examiners do not agree with each other’. Even more interesting was the outcome of another experiment where the same answer script was sent to the same examiner again after several months. It was found that there was a significant variation in the scores. This leads to the inference that ‘ examiners do not agree with themselves’.

We have made much progress in moving towards consumer protection, right to information and rights of children to a safe learning environment. We have a great awareness of the negative effects of junk food and serious punishments for adulteration and fake goods.

Shouldn’t fake tests meet the same fate.

The Fairtest organisation in the US has made a scathing observation that the quality of pet food in the US is better regulated than the tests being administered to the children.

There is a whole body of knowledge and good practices for assessment, including item analysis on the basis of facility, reliability and discrimination index, and quality examinations do declare relevant parameters to organisations who use their tests.

But in India we distinguish between students on one mark difference when the error of measurement itself would be much more.

As we come towards the end of 2012, we have the good fortune that the Vice Chancellor of Delhi University is a distinguished Mathematician, the director of NCERT is also a mathematician, and the Chairman of CBSE is an IITian. There can be no better time to acknowledge the importance of educational measurements and systematically improve upon the random unreliable assessment to one where we know what we are measuring and how reliably and accurately.

About mmpant

Prof. M.M.Pant has a Ph.D in Computational Physics, along with a Professional Law Degree, and has been a practitioner in the fields of Law, IT enabled education and IT implementation. Drawing upon his experience in world class international institutions and having taught in various modes of Face-to-Face, Distance Learning and Technology Enhanced Training, Prof. Pant is now exploring the nature of institutions which will be successors to the IITs, which represented the 1960s, IIMs, which represented the 1970 and Open Universities which were the rage of 1980s & 90s. He believes that the convergence between various media and technologies would fundamentally alter the way learning would be created, packaged, and delivered to learners. His current activities are all directed toward actual implementation of these new age educational initiatives that transform education in the post Internet post WTO era.. Prof. Pant, has been a Former Pro-Vice Chancellor, Indira Gandhi National Open University (IGNOU) and has been on the faculty of IIT – Kanpur (the premier Engineering institution in India), MLNR Engineering College and Faculty & Visiting Professor - University of Western Ontario-Canada. He has been visiting scientist to research centers in Italy, England, Germany & Sweden and has delivered international lectures with about 80 papers published. During his association of almost 15 years with the IGNOU, Prof. Pant has served as the Director Computing and has been the Member of All Bodies (i.e. School boards, Academic council, Planning board, Finance committee and the Board of management). With his interest in Law, backed with practice of Law in a High Court, and his basic training in Science and IT, Prof. Pant has been particularly interested in the Cyber Law, Patent & trade mark issues, Intellectual Property Rights (IPR) issues etc. and has been involved with many activities, conferences on “Law & IT” Prof. Pant is presently; • Advisor to Media Lab Asia - Chairman of working group on ICT for Education, chairman of PRSG handling projects on ICT for education. • Lead Consultant for an ADB funded project for ICT in Basic Education in Uzbekistan • Member of the drafting Group for India’s National Policy on ICT in education • Chairman of the group creating books for class 11 and 12 students on ‘Computers and Communication Technology’ appointed by the NCERT • Preparing a ‘Theme Paper” for the NCTE in the area of ICT and Teacher Training • Advisor and mentor to several leading Indian and Multi-national Companies in the area of education. Prof. Pant has in the recent past been ; • Member – Board of Management – I I T, Delhi for 6 years (two consecutive terms) • One-man committee to create the Project Report & Legislation for Delhi IT-enabled Open University • Advisor to the Delhi Government on Asian Network of Major Cities Project (ANMC-21) distance learning project in association with Tokyo Metropolitan Government. • Chairman Board of Studies, All India Management Association With his mission to create and implement new business opportunities in the area of e-learning & learning facilitation, Prof. Pant has promoted Planet EDU Pvt. Ltd., as its Founder & Chairman, along with a team of highly experienced and skilled professionals from Education & Training, Operations, IT and Finance.
This entry was posted in Uncategorized. Bookmark the permalink.

Leave a Reply