What Data Analytics Can Do for You
Nearly everyone, from the world of business to academia and government, seems to be talking about Big Data. Endless pieces of information, from billions of sources, are being collected and evaluated, often in real time. It’s one of the most important features of our current technological climate. It has tremendous repercussions for business and scientific innovation, but also for individual privacy.
Big data is generally defined as a data set that is so large that innovative forms of processing are required to capture, analyze, and manage it.
Every time you go online, your digital footprints leave a trail of data: tweets, social media posts, emails, search engine queries, websites visited. When you shop in a mall, more data about your consumer choices is collected as well. That trail of information, along with everyone else’s, is gathered into a massive big data environment, where trends and patterns and profiles can be collated and correlated by sophisticated data analysis techniques.
A 2001 research group identified the three primary characteristics of big data as volume, velocity, and variety.
Volume is the amount of data, measured in today’s benchmarks of petabytes or even exabytes. For context: the digital archives of the US Library of Congress was reported to contain around 74 terabytes of information in 2009; in 2011, the US National Security Agency boasted of collecting this much information every six hours.
Velocity is the speed at which data moves. Big data is often generated, processed, and analysed in real time.
Variety is the range of data sources for data. Big data can capture structured and unstructured data in all formats from a vast array of sources, both online and offline.
Two additional descriptors have since been added: veracity – the quality of the data – and variability – its consistency, and thus reliability.
Big data is growing exponentially on all of these fronts.
Big data is now crucial to commercial success. The consumer data being leveraged for corporate benefit is far more detailed, and more individualized, than ever before. The savvy use of big data analytics can add another “V” factor – value – by giving companies a competitive edge. Businesses large and small are realising that if they don’t develop big data strategies, they’ll get left behind.
Insights into trends are where organizations can really take advantage of big data analytics; companies that catch new trends can stay ahead of the curve by maximizing distribution.
Building Your Analytics Capacity
Big data analytics has rapidly become so ubiquitous that all companies will need to engage with it in some shape or form. But if you’re new to big data, how best can you go about it? Data analysis is a complex and specialized field; how you initially choose to engage with it will depend on the current level of expertise in your IT department. As your capacity and experience grows, you may want to explore more advanced options.
Here are three stages of data analytics maturity for you to consider:
1. Get intelligence on trends
A good place to start with data analytics is to subscribe to reports on data trends, or to a trends-based web service.
These services, such as Microsoft Insights, can provide valuable research data relevant to your business operations. Big data allows researchers to collect and analyze various industry trends and patterns, draw conclusions, and publish or sell the results. These reports are available across all industries, from health, retail, and technology, to real estate. You can sign up for pre-packaged reports, or hire a consulting firm to provide more customized data.
For example:
A family-run pharmacy may purchase consumer reports about wearable medical device trends, to better understand buying habits and assist their purchasing decision-making. For instance, they could access detailed information about the demographics and health profiles of people who purchase different wearable medical devices, and compare this to their own customer base. The reports give them access to far more data than they could collect through their own business, and provide analysis beyond their own technical capacity.
2. Optimize Online Engagement
Once your IT department is more familiar with data analytics, you might decide to upgrade to a web analytics service. This is a more hands-on approach to investigating your specific customer base and how it interacts with your business. The big advantage is that these services are highly customizable; however, this means that you need to know exactly what you’re looking for to get the most from what they offer.
Analytics services, provided by companies like Amazon, Google, or Twitter, allow you to monitor and analyze how long users spend on your website, which pages they read, where they came from, and other useful information. These services may appear more rudimentary than the analysis performed by professionals, but they are customized to your business, rather than to general industry trends. Even Google Analytics, which is free, can provide surprisingly granular data on user behaviour.
Web analytics help developers optimize a website’s potential for customer engagement. For example:
A vacation rental company could improve user experience by finding out which types of properties website visitors look for at different times of year, and featuring them on their home page. Analytics could also reveal how to improve Search Engine Optimization (SEO) to help their website display more prominently in search engine results and attract new clients. Knowing where their visitors come from helps them make the most of their advertising budget. All of these insights into their audience helps them tailor future campaigns.
3. Build Your Own Data Infrastructure
Later, you may choose to build your own data analysis infrastructure, by integrating a software framework for the distributed storage and processing of massive data sets into your current large-scale data management system. The open-source framework Hadoop is often the best option. Organizations that regularly deal with large amounts of data will benefit from this dynamic and interactive approach, which allows incoming information to be integrated into an existing database. This can help build customer profiles, improving marketing and sales.
For example:
A retail chain integrates Hadoop into the server holding the data from their rewards program and website analytics. Data on customer purchases, combined with location, website engagement, and publicly available social media content, allows the retailer to understand the preferences of very specific customer demographics: for instance, environmentally conscious, single, urban, professional women in their 30s, or working-class, suburban dads in their 40s who are soccer fans. These customer profiles would enable both online targeted marketing and better distribution to retail locations.
Building a data analysis infrastructure is a complex process, but it can produce invaluable information. Depending on the data source and the consent provided, companies may be able to share or sell data with third parties, but even if they choose not to, the internal use of data analytics can be well worth the investment.
Once you are collecting and analyzing data, you will need to be aware of the privacy implications of big data analytics. Organizations operating at this level will need a robust privacy program to ensure that they are in compliance with regulatory requirements and public expectations. My next post will discuss strategies for managing the privacy risks of big data.
This article is based on my book, Privacy In Design: A Practical Guide to Corporate Compliance.