'What's Best for Enabling Better Project Decisions: Big Data Predictive Analytics or Monte-Carlo Simulation? 23rd June 2016 in Bristol

The Bristol Aerospace Welfare Association (BAWA) and the APM South Wales & West of England (SWWE) branch were host to the thought-provoking presentation ‘What’s Best for Enabling Better Project Decisions: Big Data Predictive Analytics or Monte-Carlo Simulation?’ delivered by one of the UK’s top scientists, Dr Nira Chamberlain.

Dr Chamberlain is a Principal Consultant for Babcock Analytics Solutions and is an Elected Council Member of the Institute of Mathematics and its Application. He has spent the majority of his career in the Defence and Energy Sectors, and his work has entered the US encyclopaedia of mathematical inventions. 

Dr Chamberlain opened by identifying that, whilst Big Data is in currently ‘in vogue’, everyone has a different opinion about what it means. In his view: 

‘Big data is a term that describes the large volume of data – both structured and unstructured – that inundates a business on a day-to-day basis. But it’s not the amount of data that’s important. It’s what organizations do with the data that matters. Big data can be analysed for insights that lead to better decisions and strategic business moves’.

Dr Chamberlain explained that his focus is on the predictive side of Big Data (something he quantified as ‘n-dimensional space’) and ‘using what the data says’ to help modern society to define forward looking strategies. He also acknowledged that there is huge potential within Defence for the exploitation of Big Data Analytics, due to the number of the projects and initiatives that are undertaken by the Defence Sector and the amount of associated data that is generated. 

‘Dark Data’ was also touched upon; the data that is captured by organisations but simply isn’t used or analysed (i.e. timesheets). Dr Chamberlain explained that Big Data Analytics is a way of utilising this data, and cited HR (i.e. identifying trends that predict when an individual will be off with sickness) as the function that could make the most of Dark Data.

The topic ‘Is Monte Carlo Simulation Obsolete’ was then explored, with Dr Chamberlain identifying the main problems with MCS (which is inherent in MOD Projects) as being:

• Tends to underestimate cost, time and risk;
• Risk probabilities are assumed;
• It is only as good as the assumptions that underpin it (gut feel);
• Represent an event from a probability distribution which may be chosen arbitrarily.

Nira then identified the five main uses of MCS, and explored if MCS still had a role to play compared to the value that the application of Big Data Analytics could bring:

• Cost Modelling: Big Data Predictive analytics could challenge Monte-Carlo methods in the way we do cost modelling by interrogating datasets of what actually occurs as opposed to exposure to optimism bias
• Risk Modelling: Big Data Predictive analytics could challenge Monte-Carlo methods in the way we do Risk modelling by interrogating datasets; risks are evidence driven (i.e. probabilities of risks occurring can be determined much more accurately) rather than pure opinions.
• Reliability Modelling: Reliability, traditionally the domain of Monte-Carlo simulation is being challenged by Big Data Predictive Analytics; it can find hidden patterns that can improve spares modelling.
• Discrete Event Simulation: Big Data Predictive analytics can be used to improve event simulation and find new ways of solving problems (i.e. it can demonstrate that using non-traditional ways of moving spares, such as helicopters, can be cost effective).
• Agent Based Simulation: MCS still has a role to play with agent based simulation; it would take sixty different Big Data Predictive Models to produce the same graph as one single Agent Based Simulation.

In conclusion to the above question re: MCS obsolescence, Dr Chamberlain determined that MCS still has a role to play:

• Big Data Predictive Analytics better at predicting near future events and short term decisions. The model’s shelf life is short.
• Monte-Carlo Simulation better at long term strategic events.

Nira followed this conclusion by outlining a combined approach; the Analytical Hybrid Simulation (ASH). ASH makes best use of both MCS and Big Data to improve the accuracy of predictive models. Black Box analytics (i.e. those used on a plane’s Black Box) are 73% accurate; if they were to be enhanced with an ASH algorithm, the accuracy would improve to 96%. 

Dr Chamberlain concluded his presentation by emphasising that ‘Big Data is not the replacement of science. It complements it. However, if Best Practices are not followed, it will not add value to your project’.

Paul Johnson
Newcomers Representative - APM SWWE Committee