Looking beyond Big Data

 

Data is a force that matters in shaping decision-making and developing policies. Data has gained in significance in the field of education and training, particularly in areas that used to be neglected in terms of investment and international cooperation, such as vocational education and training (VET) and adult learning programmes.

As American statistician W. Edwards Deming famously said: “Without data, you’re just another person with an opinion”.

Data is important, said Mihaylo Milovanovitch, a Senior Human Capital Development (HCD) Expert - Coordinator for System Change and Lifelong Learning at ETF, as at the simplest level, it allows for informed decisions.

“Data injects knowledge into discourse. In policy-making, evidence is just one aspect taken into consideration when decisions are made, as there are other considerations – feasibility, politics, and so on – but evidence allows for a choice. If you don’t have data, you don’t have the choice to make informed decisions, or at least know what you’re ignoring as a trade-off,” he said.

A second reason concerning data’s importance is that it allows for the creation of a basis for building a consensus around an idea or policy, which may also involve countering a bad idea.

Thirdly, data can help build trust. “It is a good way to promote accountability and to make beneficiaries trust that what is going on is the right thing to do,” he said.

But for data to be truly useful, it must be rooted in the context of what is trying to be achieved. “If you don’t invest in facilitating or revealing the context, you will never know what the data really means, be it the data analysis or for those affected by the data. It is extremely important to talk about data and invest the same effort in capturing the context as in generating data,” said Milovanovitch.

The limits to Big Data

Over the past decade, there has been a flurry of discussion around data gathering, including the upsides and pitfalls of how it is used. This has been spurred on by the explosive growth of Big Data through increased computing power, advanced software tools and data storage (such as through the cloud).

“There is a renaissance of reflection after years of Big Data usage - What is missing? What are the factors influencing it, and the willingness of practitioners to use the data they have? There is a realisation that it is not just about having the data, or one type of data, but the context and conditions of how data is generated, and how it is supposed to be used. How data was used to support policies wasn’t such an issue before,” he explained.

At the educational level, the limitations of Big Data are shaping how and what data should be gathered.

“Big Data covers a whole system or whole populations, but this kind of data has limits to what you can do with it to improve the fate of individual students, or how a school is doing. Big Data allows for a bird’s eye view and for big presentations, but, in reality, it is not enough to guide reforms. It does not tell you what to change specifically, like this or that to have a certain result,” said Milovanovitch.

Neglected areas

A more specific focus on what data is needed to guide decision-making has been essential for developing segments of education that have been “neglected until recently, like VET,” he said.

Since the introduction of the Torino Process in 2010, evidence-based analysis of VET policies in ETF partner countries has been developed to provide a snapshot of the state of development of VET systems every few years.

There have been challenges to implementing policies, with a common complaint from countries about the lack of data and evidence.

Milovanovitch said the ETF’s message is that countries can get evidence if they have the right approach to data gathering and are flexible. “In most of what we are trying to monitor, it is possible to have the evidence. Countries need to find the right way to generate it,” he said.

This involves being open to using different types of evidence that mix the qualitative and quantitative and carve out a space for people concerned by the evidence to have a say.

A sound methodology is equally crucial but it “should not shy away from combining bits and pieces of what is needed. Because life is so colourful, and policies are messy, you may need to combine things that might not seem combinable at first glance,” he said.

This takes on added importance with a ‘moving target’ like monitoring priorities in education.

“We have this situation where life gallops ahead, policies try to respond to developments, and people try to develop evidence, usually as the last step, which is a challenge, as this really should happen at the beginning, to know what needs to be looked at,” he said.

A variety of sources

At the pragmatic level, the ETF is working to include data from a wide diversity of sources to provide both the micro and the macro view.

“Both information streams – national and international data – need to come together for a single message. This is a first step to acknowledge that information coming from national sources is as important as international data,” said Milovanovitch.

Yet while gathering data is an apparent challenge, countries also have more data than they often realise, as information is not always properly coordinated or collated in one place to be analysed, he added.

There can be unexpected positive outcomes from the requirement to share information under the Torino Process, which needs strong communication between central and regional governments.

In Ukraine (prior to the conflict), Tunisia and Kazakhstan the “process of generating evidence turned out to be an instrument for facilitating dialogue between the centre and regions, and horizontally, such as with stakeholders. Educators appreciated this and used the collection of information to talk in ways they didn’t do before,” said Milovanovitch.

The ETF is working to support countries’ efforts in collecting evidence and in analysis, and is actively allocating resources to ensure that the data generated is reliable. “We care about involving our partner countries as if we don’t do that, and they don’t work with us, we have a problem,” he said.