WHAT IS BIG DATA?
Big data is a buzzword used to describe structured and unstructured data that is so complex it is difficult to process using traditional database and software techniques.
Big Data can be characterised by 3 Vs:
Volume, Variety and Velocity.
Big data is particularly a problem in business analytics because standard tools and procedures are not designed to search and analyse massive datasets.
However, big data, when captured, formatted, manipulated, stored, and analysed in the correct manner, can help a business to gain useful new insights, increase revenues, get or retain customers, and improve operations.
HOW BIG IS IT?
A typical PC today might have 500gb – 1tb of storage. Today, Facebook ingests 500 terabytes of new data every day; a Boeing 737 will generate 240 terabytes of flight data during a single flight; the proliferation of smart phones, the data they create and consume; sensors embedded into everyday objects will soon result in billions of new, constantly-updated data feeds containing environmental, location, and other information.
THE NEED FOR SPEED
Click streams and ad impressions capture user behaviour at millions of events per second; high-frequency stock trading algorithms reflect market changes within microseconds; machine to machine processes exchange data between billions of
devices; infrastructure and sensors generate massive log data in real-time; on-line gaming systems support millions of concurrent users, each producing multiple inputs per second.
THE SPICE OF LIFE
Big Data data isn’t just numbers, dates, and strings. Big Data is also geospatial data, 3D data, audio and video, and unstructured text, including log files and social media.
Traditional database systems were designed to address smaller volumes of structured data, fewer updates or a predictable, consistent data structure. Traditional database systems are also designed to operate on a single server, making increased capacity expensive and finite.
As applications have evolved to serve large volumes of users, and as application development practices have become agile, the traditional use of the relational database has become a liability for many companies rather than an enabling factor in their business.
HOW DOES BIG DATA WORK?
Although big data doesn’t refer to any specific quantity, the term is often used when speaking about terabytes, petabytes and exabyte’s of data, much of which cannot be integrated easily.
Big Data takes time and money to load into a traditional relational database for analysis, therefore new approaches to storing and analysing need to be utilised.
Pernix offer cloud-based big data analytics to enable analysis of large data sets in real-time to meet the requirements of leading platforms to store large data sets across distributed and high spec’d clusters.