Posts Tagged Statistics
While preparing some presentations for HR and recruitment peers I researched and re-thought the trends (aka buzzwords :-)) which are currently discussed in the HR and online community. I will add my thoughts over the next weeks (hopefully not months) and start with Big Data. Also thanks to @carllsons who included this topic in his presentation at the Social Media in HR Summit a few weeks ago.
Generally speaking, Big Data is…
… a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools. The challenges include capture, curation, storage, search, sharing, analysis, and visualization. The trend to larger data sets is due to the additional information derivable from analysis of a single large set of related data, as compared to separate smaller sets with the same total amount of data, allowing correlations to be found to “spot business trends, determine quality of research, prevent diseases, link legal citations, combat crime, and determine real-time roadway traffic conditions” (according to the omniscient Wikipedia).
Simply said: it’s just a huge amount of numbers to which statistical analysis tools are applied. In my view, the “only” new about Big Data is the quality of data which is now available through the ubiquitous use of technology and social media, implicating also that there is a huge quantity of data available in areas where in the past one has to use spot tests and samples. What is not new: the law of averages and statistics in general.
My experience: proper statistical thinking is new to most people and HR folks anyway, so it’s time to get your understanding of numbers right 🙂 The biggest misunderstanding that I find in business, economics and society seems that people mix-up correlation and causality (sometimes intentionally). And that empirical evidence is exactly that: evidence. Meaning that it is illegitimate to draw final conclusions on a single case/object/person from empirical correlations in data sets. I will come back to both later.
Let’s take a closer look into the application and implications of Big Data in HR:
A recent WSJ article showed some examples of how Big Data is already used by employers:
What you find there is, that the companies are working a lot with “likelihoods” to select candidates or employees. This is exactly what usually is called “discrimination” – in the negative connotation of the word. They look at a group of people, defined by a certain common criteria and draw conclusions on individuals regarding another criterion. Even if you could have some (statistical) confidence in this kind of discrimination, most Western societies tried to get rid of this kind of thinking (at least when it comes to age, gender, sexual orientation,…). While employers are very careful to link themselves to this kind of discrimination, Big Data opens the door for other sorts:
But Kenexa, which tested 30 million applicants last year for thousands of clients, has found that a lengthy commute raises the risk of attrition in call-center and fast-food jobs. It asks applicants for call-center and fast-food jobs to describe their commute by picking options ranging from “less than 10 minutes” to “more than 45 minutes.” The longer the commute, the lower their recommendation score for these jobs, says Jeff Weekley, who oversees the assessments.
So someone who is eagerly looking to find employment in a low paid job and is willing to commute considerably has a smaller chance of being offered a job in this industry.
As this example shows, use of quantitative data isn’t rocket science at all but doesn’t necessarily lead to better qualitative decisions.
While I base this argument on ethical (or corporate governance) reasons and the basics of statistics, an article in the Harvard Business Review challenges Big Data in more technical aspects. Companies lack the capacity to intelligently analyze the data and they lack the systems and data structures to provide the usable data even though such data exists.
Big Data is also one of the topics which will be discussed in the upcoming HR Tech Europe in Amsterdam. Here you find examples that Big Data is not only a trend for recruitment but also considered in reward and recognition. I am sure there are already examples where it is used for development out there.
So besides the sure fact that there will be the use of Big Data for HR purposes in the future, there are some key aspects which I think are worth considering:
- Data Protection: is the Big Data really anonymized? This is also one of the major concerns regarding the mother of all Big Data users: Google
- Illegitimate deduction and wrong assumption of correlations, which might lead to…
- … threats of discrimination directly or indirectly and therefore the violation of corporate governance principles.
I would be curious to get your thoughts regarding the correct and clever use of Big Data in HR – maybe we can discuss this soon at the HR Tech Europe?
- ￼Little Data: How traditional statistical ideas remain relevant in a big-data world (andrewgelman.com)
- Big data: all you need to know (zdnet.com)
- Big Data’s neglected topic: How to secure it | TechRepublic (isykes.wordpress.com)
- Big Data: Impacting All Areas of a Business | Anjul Bhambhri | DATAVERSITY (isykes.wordpress.com)