Recent digital trends — Big Data, and the Cloud, and their business impacts
In today’s world, we are progressively adding to the amount of digital data whether as individuals or firms across the value chain. For example, during the manufacturing process, factories are equipped with sensors that continuously assess or track the performance of machinery and parts that move through the assembly process. Due to this, the world’s stock of information has been growing, opening doors to new digital technologies and services such as Big Data and Cloud services.
Big data
The trend: digitization of information in large volumes
While in the 90s, more than 95 percent of stored information on the planet was analog, only 5 percent was stored in the digital format. Then, at the turn of the century, in 2000, digital accounted for 25 percent of the total stored information at around 55*10¹⁸ bytes or 55 exabytes. By 2003, digital and analog data were equally balanced at 50% and in 2007 digital information reached 94% of the total world storage capacity at 300*10¹⁸ bytes or 300 exabytes. An IDC and Seagate whitepaper forecasts that by the end of 2020, we will reach 40*10²¹ or 40 zettabytes in stored digital information and expects 175*10²¹ or 175 z-bytes .
Definition — volume, variety, velocity
Big data has three characteristics that define it:
- Big data is created in large volumes — as indicated in the previous trend section. For example, every minute 156 million emails and 452,000 tweets are sent worldwide.
- Big data has variety — data is not only collected during structured interactions when you filling in an online form or conducting a transaction via a credit card but more so is stored in unstructured difficult-to-mine formats, images, speeches in various languages or videos.
- Big data has high velocity, it’s fast — today, we communicate with others all across the world in near real-time by chat or even video call.
Big data means that there are large loads of data, in zettabytes, that come in structured and unstructured formats, and can be exchanged in real-time, which then contributes to the growth of the volume of this data.
The business value of Big Data
This growing load of digital information is usually stored on a laptop, a mobile phone, or server with an IP address and these devices are globally connected in a network. This means that there exists the ability to link different pieces of information, correlate them, and build inferences, almost effortlessly, leaving a multiplier effect on the value you can extract from it.
But just because a company has a bunch of servers where it dumps every single piece of data, doesn’t make it a big data business. A company needs to be able to translate that data into a competitive advantage to create value and business impact from it. The most common uses of big data by firms include:
- Personalization of offering — companies such as Amazon are obvious examples. But Brick and Mortar companies also use Big Data to get to know their customers and offer customized solutions such as discount vouchers from a nearby supermarket.
- Fraud reduction — for example, credit card companies analyze millions of transactions to identify unusual patterns, and therefore reduce fraud in real-time.
- Predictive maintenance — for example when a company uses the data it collects about operations to predict performance issues before they even happen which is extremely valuable in asset-intensive industries, for example in the Oil and Gas industry.
For example, the competitive advantage of Netflix is not only just making videos available online but also to improve the whole experience of discovering videos of interest. For this, Netflix collects enormous amounts of data and analyzes users’ watching habits to generate personalized recommendations and offerings. Netflix also analyzes what people like to watch and why, and uses this data as a basis to produce its series and movies.
The challenge: customer trust and Big Data
Trust in a data and analytic sense as it applies to personal data or PII, is defined in two ways:
- First is what are the legal and regulatory rules that define how an organization collects, stores, stewards, forgets, and uses consumer data and what rules need to be followed along those dimensions.
- Second is the consumer sentiment definition of trust, meaning, do I trust this organization to steward my data in ways that I deem safe, effective, and careful? This is a much more subtle and nuanced definition of trust as it’s contextual and has to do with consumer understanding and perceptions and very little to do with regulations.
A majority of corporations are extremely bad in managing the consumer sentiment aspect of digital trust, and this could have a couple of reasons:
- Firstly, historically, when data has been stolen or when companies have used data in new ways, there has been no negative implications,
- Secondly, only recently has the combination of storage, data and analytic tools, and the cloud-enabled stock of data to be used in new and powerful ways. In other words, today, data flows, while it did not in the past.
If consumers are creeped out by corporate data misuse, they end to act in ways that are brand destroying and revenue harmful. Surveys suggests that in the first year of data misuse — in today’s level of understanding of what a data misuse is by the mass market — can create a 7- 8% drop in revenues, and in year two, this figure drops to 2–3%.
Today, when there is data misuse, ~20% of the population perceives it as one, but these figures are increasing as consumers are educated. And if the figure increases from 20% to 40%, the 7–8% drop in revenues will mount up to 14–16%, which is a significant figure.
Additionally, if consumers trust an organization with their data, then they are 7–10 times more likely to let you the firm do new stuff with their data. So there is both a sustainable competitive advantage in being a trusted steward of data, and there is a serious brand and revenue hit risk of inadvertently crossing the mistrust line.
So the challenge for corporations is to make a shift from a pull-based model, whereby consumers are pulling their personal information in, to a push-based model, where the company ensures that the user understands what data is stored and used, and when the firms process the data in new ways, they provide room for users to engage with the firm in some form or fashion. The big mental shift here is understanding that data misuse has real consequences and making the pivot from a non-transparent to a transparently user engaging, data corporation, is key to progress in the digital age.
Shifting to ‘the Cloud’
The amount of available data is growing and it needs to be stored and processed somewhere, and this is where Cloud service comes to play.
The trend: supercomputers and communication speed
Int the last two decades, advanced processing power has flown more into the supercomputer versus our personal devices, and communication speed doubled its growth rate when we entered the 21st century, whereas, processing power has been growing roughly at the same rate since the 90s. Therefore, it has become relatively more efficient for us o waste a small amount of time transferring data to save a lot of processing time by using the supercomputer, in the cloud. This is the shift to the cloud.
The shift to cloud services will be creating a lucrative market. the total cloud market services industry will be growing fast and is projected to reach 350 billions dollars in 2022.
Definition — IaaS, PaaS, SaaS, BPO
Depending on how deep we want to be involved, one can be involved with four layers of cloud services:
- Infrastructure as a Service, IaaS — for example, Amazon Web Services (AWS) is an infrastructure used to store photos, files, descriptions, etc. along with their backup and security, on a cloud-based server. In technical terms, IaaS, are online services that provide high-level APIs used to dereference various low-level details of underlying network infrastructure such as physical computing resources, location, data partitioning, scaling, security, backup, etc.
- Platform as a Service, PaaS — or sometimes referred to as Application as a Service (AaaS), for example, Google App Engine, provide a platform allowing customers to develop, run, and manage applications without the complexity of building and maintaining the infrastructure typically associated with developing and launching an app.
- Software as a Service, SaaS — or sometimes referred to as ‘on-demand services’, is a software licensing and delivery model in which software is licensed on a subscription basis and is centrally hosted, such as CRM tools.
- Business Process Outsourcing, BPO — or sometimes referred to as Business Process as a Service (BPaS), is any type of horizontal or vertical business process that helps automate business services. For example, the payroll management process, including payslip entering, the legal filings, the payment process, etc.
The business value of Cloud services
There are three general benefits of using Cloud services:
- Performance — cloud services can be more reliable in building disaster recovery mechanisms that many businesses forget or under-invest in and can provide access to peak capacity that would not be economically viable if built in-house.
- Agility — cloud services can be set up quickly, be scaled as business grows, allow seamless upgrades, and are device diagnostic.
- Cost reduction — cloud services can reduce costs on two fronts. First, they can reduce upfront investments (i.e. CapEx). This is especially important when there is high uncertainty on how much capacity will be needed. Therefore, opting for a pay-per-use model can reduce the risk of over-investing at the start. Secondly, cloud services can reduce the cost of ownership of hardware or software and the human resources needed to run applications.
For a typical company, cloud services can reduce costs by 20–50%. You can find finer details in the articles below.
The challenge: hyper-scalability
Hyperscalability is the key to a successful cloud solution. Hyperscalability is the definition of an architecture that can scale appropriately with increased demand. This is important because clients desire reliability and customizable microservice solutions, and usually, due to network effects factors in the digital economy, demand can grow suddenly overnight. Today, hyper-scalable solutions make up ~20% of the data-center market, and they will grow even further in the years to come.
The amount of stored digital data is increasing exponentially and Big Data can be defined as an enormous amount of unstructured, fast moving data that can be traced, connected, and analyzed to generate business value or transform business models. Earning consumer trust in using their personal data will be the source of a long lasting competitive advantage. Cloud services are growing in demand, driven by fundamental shifts in technology economics. Cloud provides four varying layers of services bringing performance, agility, and cost reduction to firms. Cloud services will need to provide hyper scalable architectures to mitigate demand growth risks.
