Oil and Gas Machine Learning Potential Hinges on Big Data Investment and Training

Oil and Gas Machine Learning Potential Hinges on Big Data Investment and Training

Machine learning is transforming the oil and gas industry, potentially bringing billions of dollars in savings over the next decade. The only thing slowing its maximum potential is the need for big data and the talent and capital to drive the technology forward.

With five generations in the oil and gas industry in my personal history, I understand how slow the oil and gas industry takes on innovative technologies.

While the Shale Revolution was possible for decades, only extreme natural gas shortages beginning in the 1970s and a dire need for increased production spurred federal support and tax incentives for horizonal drilling and hydraulic fracturing technologies.

Now, with prices trading at between $26 to $50 per barrel over the past two years, our industry is again shifting gears - this time to rapidly improve operating efficiencies. Machine learning is proving to be a powerful and necessary tool moving forward.

Machine Learning Automates Operations, Lowers Production Costs

Oil and gas companies, particularly upstream, are experimenting with equipment digitalization and machine learning to accomplish a variety of ends, including:

  • Lower production costs
  • Automated operations
  • Equipment failure prevention
  • Decreased on site decision-making
  • Improved safety and maintenance
  • Mitigation of environmental hazards

Machine learning can save oil and gas companies millions of dollars in physical labor on well stops, repairs and environmental damage. The technique uses software to analyze historical data, identify specific data patterns, and then use these patterns to predict potential equipment problems.

The speed and accuracy of computerized decision-making cuts human error rates and enables operators to better control and maintain equipment.

For example, LNG operators on the U.S. Gulf Coast are using machine learning software to maintain compressors. AspenTech’s Mtell software takes historical data from previous compressor problems, identifies data patterns that occurred prior to equipment problems, and uses these patterns to inform the operator of potential recurrences in real-time.

In 2016, Southwest Research Institute developed a machine learning system to detect liquid hydrocarbon leaks. The Smart Leak Detection system (SLED) uses optical infrared and visual sensors positioned along a pipeline or on drones flying over the pipeline network.

These sensors scan infrastructure and transmit images for real-time computer analysis. While companies have used cameras for leak detection in the past, the SLED system improves accuracy, detecting smaller leaks and specific non-leak conditions, markedly reducing false alarms.

Oil and Gas Lacking Big Data: Limits Machine Learning Potential

Though machine learning is already benefiting a number of oil and gas companies, integration of machine learning into the industry as a whole could be another five to 10 years off due to one major challenge – the need for big data. Machine learning requires significant amounts of historical data to form the patterns required for predictive analysis.

The less data you have to work with, the less effective any machine learning algorithm will be.

As things stand, only 3% to 5% of oil and gas equipment is digitally outfitted, and companies use less than 1% of any data they do collect for decision-making. Most historical maps and drilling surveys are still on paper or digitally stored using a diversity of incompatible formats.

Without a history of digital data to analyze, many oil and gas companies are easing into machine learning using software that blends information from well-proven predictive models of the past with recently collected digital data. Using these combined data sources is already allowing operators to make faster, more accurate decisions about equipment.

Meanwhile, to bring oil and gas machine learning to its full potential, big data is a primary area of focus. In 2016, 68% of large oil and gas companies had invested over $100 million in data analytics over the past two years and nearly 75% planned to set aside between 6% and 10% of their capital budget for digital technology.

Many large oil and gas service firms like General Electric Co and Schlumberger NV are investing in large data processing facilities to help clients extract and prepare vast amounts of historical data for machine learning.

Data Science Training and Investment Expedite Machine Learning Integration

To get a full grasp on how the U.S. oil and gas industry can take advantage of machine learning moving forward, read my new book, Giant Shifts: Energy Trends Reshaping America’s Future. There are a few key moves that need to take place to reap the full rewards of machine learning in oil and gas.

(1) Oil and gas companies must seek out talent with petroleum data science training. Universities and other institutions are already implementing petroleum data science training programs like Stanford University’s Resources Engineering Department’s Smart Field Consortium, University of Southern California’s Center for Interactive Smart Oilfield Technologies and USC’s master’s degree in Smart Oilfield Technologies.

Houston, Texas is proving to be the hub for petroleum data science training.

BP established the Center for High-Performance Computing in Houston to allow global industry leaders close interaction with the Global Petroleum Research Institute, Center for Petroleum and Geosystems Engineering and the Berg-Hughes Center for Petroleum and Sedimentary Systems. In addition, Houston offers impressive K-12 STEM education programs focused on the petroleum industry.

(2) Venture capitalists must invest in petroleum data technology. Venture capitalists and entrepreneurs should seek opportunities in data analytics, nanotechnology, robotics and IoT. Big data analytics is an immediate need, necessary to collect and sort unprecedented amounts of both old and new data.

Storage technology, advanced data capture and sophisticated software development will continue to become increasingly important.

While the expense of technology development and implementation is high, the long-term gain is well worth it. Infrastructure digitalization and machine learning will bring billions of dollars in savings over the next decade through improved operational efficiency and environmental compliance.

Training and investment in big data is key to realizing the full potential of oil and gas machine learning.


Attorney Jimmy Vallee is an energy Mergers & Acquisitions lawyer, oil and gas industry commentator, and frequent resource for media outlets including USA Today, U.S. News & World Report, Oil & Gas Monitor and others. His new book, Giant Shifts: Energy Trends Reshaping America’s Future, released in May, 2017 hit #1 in two Amazon categories the week of its launch.
Connect with Jimmy at [hidden email]
Noted Energy Futurist” – Mensa AG 2016

You might also like:

Visit Jimmy's Law Firm: Paul Hastings