What are vanity metrics?

Posted on October 26, 2020 by mmpant

What are vanity metrics?

What are metrics?

About 250 years ago  Lord Kelvin said:  “I often say that when you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind; it may be the beginning of knowledge, but you have scarcely in your thoughts advanced to the state of Science, whatever the matter may be.” 

In today’s age of big data and data analytics, with its deluge of data, it becomes important to assign meaning to the data. 

Metrics are quantitative values of commonly used measures for assessing, comparing, and tracking performance or progress. Often a group of metrics will be used to build a dashboard that management or analysts review regularly to maintain performance. 

In this  digital age, it’s become easier than ever for educators  to track the various metrics of their students. However, the metrics commonly tracked by educators can often be non-factors in the actual success of the educational efforts, while the useful ones may go untracked.

Vanity metrics?

Here is a very good example of a vanity metric :  

CBSE, in collaboration with Intel, organised the ‘AI For Youth’ Virtual Symposium from October 13-17, 2020.

During the symposium a Guinness World RecordsTM attempt was made for ‘Most users to take an online Artificial Intelligence lesson in 24 hours’ on October 13 &14, 2020

 To know more about Symposium – https://bit.ly/3d0AwNW For any other queries – help@aiforyouth.org

Vanity metrics can be fickle, misleading, and appealing for all the wrong reasons. Vanity metrics are metrics that make you look good to others ( like being in the Guinness book of records) but do not help you understand your own performance in a way that informs future strategies.

Vanity metrics should be  contrasted with actionable metrics which is data that helps you make decisions and helps your mission  reach its goals. 

How many views make a YouTube video a success? How about 1.5 million? That’s how many views a video an organization, DoSomething.org, posted in 2011 got. It featured some well-known YouTube celebrities, who asked young people to donate their used sports equipment to youth in need. It was twice as popular as any other video that Dosomething.org had posted to date. Success! Then came the data report: only eight viewers had signed up to donate equipment, and zero actually donated.

Characteristics/ attributes/ features of vanity metrics: 

It can be very hard to sift through data and figure out what actually helps versus what just looks good. If you’re unsure if a metric is a vanity metric, a quick shortcut is to ask: “Can this metric lead to a course of action or inform a decision?” If the answer is “no” or “I don’t know,” then you should probably re-evaluate it.

Smart, actionable metrics help you make a decision. They provide feedback and context for what your goal  is and whether or not you’re moving towards it. 

Another clue is whether or not you can manage cause and effect within your data. Observing random occurrences isn’t helpful. 

Is the data a real reflection of the situation?

Often, data can be manipulated or spruced up with extra effort. Data fallacies abound. Vanity metrics may be hard to identify. The central question to ask yourself when considering a metric is whether or not it will help your business achieve its goals.

In direct opposition to vanity metrics, there exist actionable metrics. Actionable metrics are actual, useful statistics that can provide educators with reliable data that they can then use to objectively improve their eLearning experiences, making actionable metrics far more desirable than vanity metrics.

Examples of vanity metrics in education: 

Any metric can be a vanity metric, it all depends on the analysis. However, below are some of the most common examples of metrics prone to being vanity metrics. The education market space is full of vanity metrics, such as:

  • Institutional Ranking in various ranking systems
  • Area of the campus 
  • Number of books in the library 
  • Salary Packages of your graduating students
  • GER in higher education
  • Percentage of GDP spent by the Government on education

Some common vanity metrics in relation to eLearning include:

  • Module completion rate.
  • Course completion rate.
  • Module views.
  • Course views.
  • Number of questions answered.
  • Number of words typed.

But do these metrics measure what really matters? Do they convey the achievement of your mission? Do they build credibility for the delivery of your value proposition and brand promise?

Parents enthusiastically embrace traditional talking points (e.g., test scores or elite college lists), as they go out into the community as ambassadors for the school–or at least to defend their school decision. These are numbers they can understand and correlate to the common language of other parents.

Bad metrics are most common when there is  lack of clarity on what success looks like and doesn’t have strong, accurate feedback loops to access progress. 

Here are some examples of what could be real actionable educational metrics?

I am sure that you have thought of many other important parameters that could really be used for meaningful and actionable learning metrics. Look forward to your sharing them. 

About mmpant

Prof. M.M.Pant has a Ph.D in Computational Physics, along with a Professional Law Degree, and has been a practitioner in the fields of Law, IT enabled education and IT implementation. Drawing upon his experience in world class international institutions and having taught in various modes of Face-to-Face, Distance Learning and Technology Enhanced Training, Prof. Pant is now exploring the nature of institutions which will be successors to the IITs, which represented the 1960s, IIMs, which represented the 1970 and Open Universities which were the rage of 1980s & 90s. He believes that the convergence between various media and technologies would fundamentally alter the way learning would be created, packaged, and delivered to learners. His current activities are all directed toward actual implementation of these new age educational initiatives that transform education in the post Internet post WTO era.. Prof. Pant, has been a Former Pro-Vice Chancellor, Indira Gandhi National Open University (IGNOU) and has been on the faculty of IIT – Kanpur (the premier Engineering institution in India), MLNR Engineering College and Faculty & Visiting Professor - University of Western Ontario-Canada. He has been visiting scientist to research centers in Italy, England, Germany & Sweden and has delivered international lectures with about 80 papers published. During his association of almost 15 years with the IGNOU, Prof. Pant has served as the Director Computing and has been the Member of All Bodies (i.e. School boards, Academic council, Planning board, Finance committee and the Board of management). With his interest in Law, backed with practice of Law in a High Court, and his basic training in Science and IT, Prof. Pant has been particularly interested in the Cyber Law, Patent & trade mark issues, Intellectual Property Rights (IPR) issues etc. and has been involved with many activities, conferences on “Law & IT” Prof. Pant is presently; • Advisor to Media Lab Asia - Chairman of working group on ICT for Education, chairman of PRSG handling projects on ICT for education. • Lead Consultant for an ADB funded project for ICT in Basic Education in Uzbekistan • Member of the drafting Group for India’s National Policy on ICT in education • Chairman of the group creating books for class 11 and 12 students on ‘Computers and Communication Technology’ appointed by the NCERT • Preparing a ‘Theme Paper” for the NCTE in the area of ICT and Teacher Training • Advisor and mentor to several leading Indian and Multi-national Companies in the area of education. Prof. Pant has in the recent past been ; • Member – Board of Management – I I T, Delhi for 6 years (two consecutive terms) • One-man committee to create the Project Report & Legislation for Delhi IT-enabled Open University • Advisor to the Delhi Government on Asian Network of Major Cities Project (ANMC-21) distance learning project in association with Tokyo Metropolitan Government. • Chairman Board of Studies, All India Management Association With his mission to create and implement new business opportunities in the area of e-learning & learning facilitation, Prof. Pant has promoted Planet EDU Pvt. Ltd., as its Founder & Chairman, along with a team of highly experienced and skilled professionals from Education & Training, Operations, IT and Finance.
This entry was posted in Uncategorized. Bookmark the permalink.

Leave a Reply