From raw data to real insights

Facebook
Twitter
WhatsApp
Telegram
Email

“The real price of everything, what everything really costs to the man who wants to acquire it, is the toil and trouble of acquiring it.”

– Adam Smith, 1723 – 1790, Scottish economist

WHEN it comes to supermarket shopping, I have been implementing a strategy for the past decade: I consistently visit a variety of supermarkets and never go to the same one twice in a row.

Different supermarkets have different advantages, but a large part of it is price. It’s not always easy to figure out if an item is genuinely ‘on sale’ or simply cheaper elsewhere.

Enter the recently announced KitaJaga app. It was previously an app that allowed you to find people looking for contributions during the pandemic, but now it includes a way to compare grocery prices in shops around your area, and it even plots them on a map for you.

Sounds fantastic, right? Well, not everyone thinks so. People on social media have accused KitaJaga of ripping off PriceCatcher, a government website run by the Domestic Trade and Cost of Living Ministry. They argue that the data and even the images used are exactly the same.

How can you get away with just ripping off someone else’s website when they’re both based on open data?

If this sounds familiar, think back to the CovidNow dashboard, which provided Covid-19 related statistics in Malaysia. Controversy raised its argumentative head when it was revealed that the people who built the website did so for free and allowed the government to take over its operation. But they could only do so because the government gave free access to the data.

See also  Chomsky's take on total global dominance – a revisit

In particular, this data is not some big Portable Document Format (PDF) file with an infographic on it. Rather, it is kept in a format that is not very human-friendly to read, but for computers, they can just suck the data up and do what they need to do with it (in this case, make pretty interactive graphs).

CovidNow has now transformed into KKMNow, with more charts and data on blood and organ donation. All the data driving these dashboards is freely accessible at the Health Ministry’s repository at code-hosting platform GitHub (github.com/MoH-Malaysia).

The KitaJaga app’s data on supermarket prices comes from a dataset hosted on OpenDOSM (open.dosm.gov.my/), provided by the Department of Statistics Malaysia.

OpenDOSM houses 264 data sets, everything from how much currency is in circulation at the moment, and demographics by geographical area in Malaysia to the consumer price index, and inflation by state and category.

If you think about it, it’s amazing. The open data concept has long been the norm in the scientific community, where sharing research results is crucial for advancing discoveries. But for the rest of the world, it’s a relatively recent development.

In 2015, I participated in a group of influential technologists, economists, and visionaries who convened in Sebastopol, California, to discuss the idea of making government data openly available and accessible to the public. This meeting led to the creation of principles that would guide the open data movement, including emphasising open access, machine readability, non-discrimination, and the use of open standards and licenses.

See also  Affinity to divinity in humanity

The idea was simple: The public should be able to freely access the data they have a stake in, so they can use it to make better decisions.

The open data movement gained momentum in 2009 when the US government launched Data.gov, followed by the United Kingdom (UK) and Singapore governments in 2010 and 2011 respectively. Malaysia opened up in 2015 with data.gov.my.

So everything’s great, right?

Not quite. In 2021, a discussion paper by think tank Khazanah Research Institute titled, “Open Government Data in Malaysia: Landscape, Challenges and Aspirations” identified several issues. Probably the most glaring one is that individual government agencies don’t have to share their data if they don’t want to. Why not? That year, I suggested agencies don’t want to try hard to protect private data because it’s a lot of work with no upside for them. Open data needs protection of privacy, plus a lot more on top of that.

Another related issue is that there is no centralised policy framework pushing such agencies towards open data. This results in inconsistencies in implementation, not the least that you have to pay for some of the data access. And then there’s the question of whether different government databases can talk to each other or not. Even the catalogue of what data an agency holds may not be accessible to other agencies without authorisation.

Despite these challenges, it is a credit that KKMNow and OpenDOSM can offer free, up-to-date, and machine-readable data. Yet, once data is made freely available, people may become less interested in it.

See also  Rewarding trip for grandniece’s wedding

For example, when the Ministry of Health (MoH) was holding press conferences to announce Covid-19 updates, reporters would always ask about case numbers. Every announcement immediately resulted in headlines like, “Numbers go up since yesterday”, or “Fewer cases over the weekend” (even if calling two days data a “trend” is really stretching the definition).

However, when the ministry started to release its data openly, it seemed like there was actually less news about case numbers going up and down. And any reporter asking for them would just be guided to the repository and invited to help themselves to the data.

I think it’s like going to a restaurant and ordering a waffle, and being led to a kitchen counter and told you can make one yourself with eggs, flour, and milk. It’s a bit harder to do, and if it doesn’t come out well, you only have yourself to blame.

The painful truth is that data is data, and it needs to be organised, analysed, and presented in the right way before it becomes information. And then making sure that information is acted on properly is another job in itself.

Hopefully in the future, people like me won’t have to go on a physical supermarket tour to find the best value. Instead, we could just pull out an app to compare prices – even if it’s one that we’ve built ourselves.

Download from Apple Store or Play Store.