Big Data is a term that describes the large volume of data – both structured and unstructured – that inundates a business on a day-to-day basis.
What is relevant to understanding Big Data is that what is done with the data is what matters. Big Data can be analyzed for insights that lead to better decisions and strategic business moves. Our traditional business systems often bury the data and move on to the next transaction rather than revise the data for intelligence and insights. Big Data is persistent — it does not fade away after it is generated — and it is needed for system evaluation.
The concept of Big Data has progressed from it initial 3 V’s (the first three of the graphic below) to now 6 V’s which are: Volume, Variety, Velocity, Veracity, Value and Variability. The following graphic provides a further explanation of these concepts.
Figure 1: The Six V’s of Big Data
The following are examples of how Big Data is used in manufacturing and assembly processes:
- Predict production equipment health and lifecycle and formulating a maintenance plan
- Preparing next-best-case scenario in case of quality anomalies or failures in real time
- Develop an agile production plan to optimize the schedule dependencies
- Conduct a root cause analysis in case of failure
- Optimize demand planning
All manufacturers concern themselves with their products’ lifecycles. Big Data can support these products’ lifecycle planning processes, through the following:
- Developing a better understanding and management of supplier quality
- Improving product quality and yield
- Improving customer service and satisfaction
- Optimizing raw, semi-finished and finished material usage
- Conducting warranty analytics
- Monetizing after sales service through advanced analytics
As a general comment, Big Data presents a landscape of opportunities and approaches. The following graphic is one representation of this technology landscape, where it can be seen that a significant number of proprietary systems can be combined to perform a variety of tasks in this environment.
Source: Big Data Landscape
Big Data, as the name suggests, is about a lot of data.
System volumes of data can be expected in some cases to escalate from Terabytes, to Petabytes, to Exabytes for hosting requirements of the data which is being generated. This calls for improved and isolated storage as well as data protection. In addition, while this data is being collected, it also needs to be processed, which would be done by High Performance Computers. The common desktop is simply not up to this task and, as a result, either clustered computers containing many cores or purpose-built computers which work primarily with many thousands of Graphics Processing Units are used.
Manitoba’s pursuit of Big Data Development
The University of Manitoba has recently created a Bachelor in Data Science Degree in the Faculty of Science. Follow this link for the announcement. Furthermore, various Faculty are conducting research in this area and are mentoring graduate students.
Red River College, through their student projects, has conducted a Data Analytics Project. The ACE Project Space hosts the Centre for Machine Learning Studies. This is a working lab and educational facility that invites researchers from industry and academia to develop solutions using machine learning techniques.
The University of Winnipeg – Professional, Applied and Continuing Education (PACE) will be launching two new post-degree diplomas in disciplines of Big Data and Machine Learning. The Predictive Analytics Diploma program will be offered for the first time in May 2021 and the first cohort in the Artificial Intelligence Diploma will start in September 2021. Graduates of both programs will be heading into the workforce in 2022.The University of Winnipeg also recently announced a Data Science Program.
A consortium of participants from the academic community (UW, RRC, UofM) participate in hosting annual conferences on High Performance Computing. The intent of this group is to build an HPC Centre in Manitoba which will serve industry, new companies, and academics. The website for this group is at: https://manitobahpc.ca/
EnviroTREC’s role in Big Data
Our role in Big Data is to support local development particularly as focused on the aerospace industry. EnviroTREC is also a member of the Manitoba Aerospace Research and Technology Committee and the Manitoba HPC consortium.
For Further Reading in this area