Power and gas trading is undergoing a transformation. The global trading value pool for these commodities is growing by leaps and bounds. It hit nearly $33 billion in 2023, up 50 percent from the year before.
But that’s only part of the story. The rise of renewable energy has introduced greater volatility into gas and power markets, increasing the value of optionality and drawing new market entrants. This is creating unique opportunities and challenges for both new and incumbent market participants.
North American gas and power markets are at the epicenter of this transformation. Those who want to capture the value in these markets can’t just conduct business as usual. In this article, we explore the trends that are already beginning to reshape these markets, the crucial capabilities that industry players could consider investing in, and the specific capability sets that matter most for each type of participant in these markets.
The five trends shaping North American gas and power markets
The North American power and gas trading value pool has tripled since 2018 to an estimated $10 billion of EBIT in 2023, approximately 30 percent of the global total (Exhibit 1). While gas trading has historically accounted for the lion's share of the total value pool, power trading’s share has increased. We expect these value pools to maintain their upward trajectory in the medium to long term and continue to attract more entrants and liquidity (although we anticipate a pullback in value pools in 2024–25 given the gas storage levels allowed by a milder-than-normal winter this past year).
We see five developments transforming these markets.
North American gas has emerged as an interconnector of value chains
North American natural gas has become a highly competitive global fuel and feedstock, driven by the shale revolution and legislation in other regions such as Europe’s Carbon Border Adjustment Mechanism, which favors carbon-efficient products. As a result, North American natural gas has become tightly intertwined with multiple regional and global commodity value chains.
North American gas infrastructure links the price of power across multiple markets, providing traders with new optionality as the region’s power and gas industry grows. For example, changes to gas pricing or flows in Pennsylvania can ripple through Ontario and Alberta to affect California gas prices, influencing power prices along the way.
Similarly, North America—emerging as the largest exporter of liquefied natural gas (LNG)—has intertwined prices at Louisiana’s Henry Hub with prices in other LNG markets, increasing both Henry Hub and global LNG spot-market liquidity. Europe- and Asia-based LNG traders and consumers now rely on the price at Henry Hub as a reference price to capture interregional arbitrage opportunities or hedge physical positions.
North American gas pricing is also linked to other global value chains that use gas as a feedstock, such as power, fertilizers, and petrochemicals. And the entanglements are becoming even more complex with the emergence in North American markets of new products such as e-methane, which is synthesized from recycled carbon dioxide and green hydrogen and is now present alongside natural gas in the North American LNG value chain.
The interconnectedness of the natural gas market presents North American gas traders with new opportunities to maximize returns via their gas and power desks and, consequently, to expand their reach into other commodity trading value chains. Incorporating this complexity into trading strategies can significantly increase the scale and diversification of players’ portfolios and create new arbitrage opportunities, maximizing the risk/return expectations of their invested capital.
Extreme weather events are increasing risk for power and gas market players
A trio of factors related to weather mean that players may need to consider new risk approaches in North American power and natural gas.
First, extreme weather-related events have risen in both frequency and magnitude: heat waves have become more intense, wildfires more damaging, hurricanes stronger, and winter storms more severe. These weather events have direct economic costs: there have been 273 billion-dollar natural disasters in the United States in the last 20 years, compared with only 97 in the 20 years before that; 2023 alone had 25, the most ever for a single year (Exhibit 2). In Canada, droughts have reduced electricity export volumes to the United States by nearly 50 percent because of reduced hydropower.
Second, as these weather patterns encounter North America’s aging infrastructure, the vulnerability of all segments of the power and gas networks is amplified. Extreme weather is further challenging the ability of market operators to forecast supply, demand, and congestion and to plan for contingencies across multiple time horizons. Volatile, weather-driven demand has highlighted the inadequacy of both the capacity and the operational resilience of the existing natural gas pipeline system. The reality of all these forces played out as recent temperature extremes in Texas summers and winters resulted in or threatened rolling blackouts in that region.
Finally, the combined effect of increased renewables (which tend to provide more energy during warm weather) and more volatile winters means some markets are shifting away from summer demand peaks. For instance, New England’s independent system operator (ISO), ISO New England, which oversees electricity markets for the region, predicts that it will no longer be a summer-peaking system within the next five to six years. Such a shift changes the core drivers of volatility. When demand peaks in summer, the challenge is matching demand to supply. Winter peaks, on the other hand, lead to problems with fuel constraints, more volatile fuel costs, and winter storms potentially causing plant outages.
A single weather event can create vastly different risk outcomes depending on the nature of the player’s market participation. For example, during winter storm Uri in Texas in February 2021, participants in the retail market were obliged to make whole on their contractual obligations to deliver power. In contrast, participants positioned upstream could claim the protection offered by “force majeure” clauses. To minimize weather-driven risks, market participants have explored alternative, bespoke risk transfer structures. As a result, the weather derivatives and insurance-linked securities markets have experienced strong recent growth: the average daily trading volumes of weather derivatives on the Chicago Mercantile Exchange increased more than 250 percent from 2022 to 2023 (Exhibit 3). Offloading risks through such mechanisms enables market participants to take greater risks elsewhere in their portfolios.
Growth in renewable energy is pushing volatility to new highs in energy markets
Renewable capacity is expected to account for approximately 50 percent of US energy generation capacity by 2030. In some markets such as the California ISO (CAISO), renewable energy sources (RES) account for 100 percent of energy supply during certain hours, resulting in routinely negative prices in the middle of the day followed by a massive ramp-up in dispatchable generation and imports to meet the evening peak load. This shift has caused volatility to more than double versus 2016–18 levels (Exhibit 4), a trend that other ISOs could expect as they continue to build out their RES capacity. In regulated markets, conventional asset owners’ rates do not compensate for their capacity being displaced by RES. These asset owners are likely facing years of eroded energy revenues, leading to asset retirement without replacement.
As RES penetration increases, existing baseload gas assets, such as newer combined cycle gas turbines, are expected to be used less often and primarily during peak times. At the same time, large-scale energy storage in the United States has grown tenfold since 2019, and this growth is accelerating in most markets. Both gas generation and battery storage assets are now vying to capitalize on the price volatility created by RES, which could lead to reduced profits for both.
One way asset owners could mitigate these threats is via capacity markets, where generators are paid for the ability to produce in the future: capacity prices are likely to rise in the face of this volatility. In markets without capacity payments, such as the Electric Reliability Council of Texas (ERCOT), players may be able to offset some risk through tolling agreements, for example. Without a risk management and portfolio strategy to protect against fluctuating returns, these assets may run into problems covering their debts. (Renewable energy players, too, could consider diversifying their portfolios to account for the high volatility RES introduces.1)
Another consequence of the expansion of RES is increased congestion. Approximately 40 percent of today’s North American transmission and distribution (T&D) infrastructure is near or at the end of its planned lifetime. Building the necessary T&D to accommodate new RES could take a decade or more, at a cost of $200 billion per year. In the interim, grids will face increased congestion. Our initial analysis indicates that congestion rents in ERCOT could double to $5 billion annually by 2030 (Exhibit 5). This forecasted rise in congestion could leave power developers overextended on their debts and will likely incentivize a rise in off-grid power generation projects.
Although merchant traders and hedge funds account for a significant chunk of the activity in the congestion markets in most ISOs, participation in congestion markets—financial transmission right/up to congestion (FTR/UTC) markets—could increase as more market participants understand their congestion exposure and seek to manage and monetize the complex pricing relationships among nodes. An analysis of Pennsylvania–New Jersey–Maryland (PJM)’s FTR auction activity points to more than a 200 percent increase in participation in the shorter-term congestion markets in recent years.
Data centers are driving demand growth
After a period of limited growth stretching back to 2003, power consumption is expected to increase, driven by data centers (DCs) and, to a lesser extent, vehicle electrification and hydrogen production (Exhibit 6). DC power demand is expected to grow at approximately 15 percent per year from 2023 to 2030. DC demand growth has prompted grid and market operators to revise their growth forecasts upward. For example, ERCOT announced in June 2024 that overall capacity will need to increase from 85 gigawatts to 150 gigawatts instead of to 110 gigawatts by 2030 to meet peak loads.
DCs tend to be willing to pay premiums in their power-purchasing agreements (PPAs) to guarantee their supply, hedge against gas-based marginal pricing, and access renewable energy certificates2 where possible, which presents an opportunity for market participants. Even though many companies with high demand for DCs have indicated their intentions to secure carbon-free power by underwriting carbon-free power solutions, new gas plants will be needed to either replace carbon-free capacity that is no longer available to the rest of the grid or to provide backup to new renewable capacity for DCs.
To meet 2030 demand, including replacing retiring coal-generated power, we estimate that the United States will need to increase electric supply by about 1000 terawatt hours, approximately half of which will be new natural gas generation. The planned growth in pipeline and infrastructure capacity, however, is unlikely to be enough to meet this demand, which will add to gas basis volatility.
The combination of increasing power demand, coal and gas plant retirements, and regulatory hurdles to building new capacity create the potential for generation shortfalls, regional price increases, and reliability issues. PJM’s capacity auction prices for 2025–26 increased over 800 percent from the year before, suggesting that players believe a generation shortfall is plausible.
The influx of new entrants is propelling market participation and liquidity to record highs
In the past ten years, several types of entrants have expanded into the North American power and gas markets, including merchant traders, global and North American energy players (utilities, international and national oil companies, midstream companies, and integrated oil and gas players), and hedge funds.
International energy companies have grown their presence in these markets, through both organic and inorganic options, to pursue new avenues of portfolio growth and to capitalize on the increased linkage between North American and global energy markets. Despite market differences, European players have been able to translate their experience, especially with renewables, structuring, and data-driven trading, to the North American context. Some players, on the other hand, have leaned on partnerships and acquisitions to compete.
Hedge funds and asset managers have beefed up their power and gas trading desks in the last two years, allowing them to capture incremental returns or to diversify risk. Some hedge funds expect commodity trading, specifically power and gas trading, to represent a significant source of their value creation.
The variety and depth of energy market constructs in North America invites participants with differing capabilities, scale, and capital access—from small-scale intraday financial players to global multistrategy hedge funds. While hedge funds have traditionally been paper traders (this is part of what has driven an increased participation in North American paper markets), we also see instances of hedge funds venturing into physical markets and acquiring asset positions.
Given expectations of persistent volatility and continued market shifts, all players’ risk exposures are likely to constantly evolve. Those with a merchant mindset might welcome the optionality embedded in their portfolios, positioning them to stay resilient as risks change. Incumbents might need to shift from their traditional project-based mindset toward a merchant mindset. Those who don’t could face pressure to offload lower-return assets, creating merger and acquisition opportunities for new players.
Three capabilities and three archetypes
No player is immune to the changes under way in North American power and gas trading. To successfully navigate the transition ahead, market participants can build three capability sets. How they wield these capabilities will vary depending on the type of participant.
The capabilities that matter most
All players will need to develop key capabilities in short-term markets, advanced analytics, and complex origination. Incumbents and new entrants alike must consider whether organic or inorganic pathways are the best route to build capabilities.
Embrace the migration of value into short-term markets. Asset owners that fail to extract value from the flexibility in their portfolios and the arbitrage opportunities created by short-term volatility could generate lower margins relative to active short-term market participants. Players may need to recognize—and optimize their portfolios for—the enhanced optionality among a spectrum of possibilities: day-ahead and real-time pricing, virtuals, UTCs, inter-ISO arbitrage, and ancillary service markets.
They could also invest in short-term trading capabilities such as weather prediction (including at the hyperlocal level), load and flow forecasting, congestion and basis pricing, production cost estimation, and asset bidding and dispatch. These investments allow players to understand and withstand disruptions at the local level. Operational risk is tied to market risk—an outage can have large market implications—so players may need to remain cognizant of this connection and account for it in their strategies.
Leverage AI and machine learning (ML) as new essentials. The large volumes of complex, structured and unstructured data inherent in power and gas markets make these markets ideal for AI/ML-enabled analysis. This is particularly relevant in the short-term trading space, where these advanced analytics can process complex data sets to provide improved predictions for nonlinear market drivers such as weather, load, congestion, and pricing. The speed with which AI/ML can arrive at these predictions can make the difference between profitability and loss.
Recent entrants into the market—in particular, hedge funds—already have these capabilities. Other players could work to minimize the capability gap versus these tech-savvy new entrants. We have seen some efforts from merchant traders and utilities, but few incumbents have made these investments at a comparative scale.
Ramp up complex and custom origination and structuring. Deal structures previously considered complex are now commoditized, transforming trading into a low-margin, high-volume game. This is particularly true for products that are engineered to be “hedge-able” in paper markets or to have minimal residual risk for the originator.
Faced with the proliferation of low-margin, low-risk deal structures, players can accrue premium origination margins by pursuing bespoke deal structures with physical optionality that is tied to the risk of actually delivering power and gas. In this vein, the growth in DCs can lead to new partnership opportunities with traders to manage the former’s susceptibility to price increases.
New cross-commodity relationships are emerging from evolving regulation, supply chain links, similar capabilities, and portfolio effects. We see growing connectivity among chemical, agricultural, metals, and energy-related commodities markets. Deals that trigger exposure across these markets offer premium margins for similar reasons: they are large and involve illiquid physical assets. Players that are active across multiple commodities markets develop an appreciation for the correlations among them and can bundle commodity exposures to better capture value.
Strong balance sheets and diversified portfolios are crucial for participating in such custom and complex opportunities.
Market archetypes
Over the past five to ten years, business models have converged in the commodity trading space, with merchant players increasing investments in assets, asset-backed players building origination, and third-party business and hedge funds moving more into physical trading. We expect this trend to extend into North American power and gas trading as well—which means market participants may need to invest in specific combinations of new capabilities to ensure that they are poised to capture the greatest value.
The tactical trader-investor: Their access to capital and advanced analytics make private equity and hedge fund players potentially well placed to take on illiquid physical positions created by supply–demand imbalances while factoring in competitive margins of safety to improve their overall risk-adjusted return. However, owning physical positions brings its own challenges: these players may need to invest in both sophisticated origination capabilities that will enable them to find, value, and transact these illiquid deals and short-term market capabilities. This combination will allow them to better capture value from physical positions and manage the operational, regulatory, or environmental constraints that come with flexible power and gas assets.
The asset-light trader: Merchant traders’ historical information advantage from long-term customer and client relationships has been chipped away by newer, data-driven strategies. Supplementing this institutional experience with investment in advanced analytics may prove essential to remain competitive. Players’ new analytics platforms should be structured to remain efficient and agile, protect foundational data via strong data governance, and use secure technology infrastructure built with robust cloud capabilities.
The profitability of previous years has made merchant traders cash rich, making them eager to secure assets, but given the migration of value to short-term trading in power and gas markets, traders may need to invest in short-term capabilities, too, particularly in weather and congestion prediction, asset-bidding and dispatch, and regulatory compliance. These investments could enable merchant traders to extract the most out of the optionality embedded in any potential equity assets, as the definition of short term in power and gas markets is much shorter and more volatile compared with other commodities these players may be more familiar with.
The asset-backed optimizer: Most asset-backed optimizers—such as oil and gas players, utilities, and renewable-asset players—will face the prospect of thinner margins and may need to invest in all three of the capabilities discussed earlier. Utilities and oil and gas players with healthy balance sheets and large portfolios can leverage their strengths to increase the scale and complexity of their origination capabilities and enable the transition toward a merchant strategy involving lower capital expenditure. Investing in short-term trading and optimization capabilities allows these players to reduce risk in their asset margins.
Renewable-asset players and oil and gas players that are responsible for their own exposures may need to create desks to continuously optimize commodity positions against physical and financial market conditions. Alongside these desks, players could benefit from other capabilities: sophisticated risk frameworks with risk definitions and heat maps, portfolio risk–return modeling, and stress testing to improve business resilience. Most incumbents are at a disadvantage relative to tech-savvy entrants into the North American markets, so targeted investments in AI and data analytics are crucial. Asset-heavy players also need to consider questions related to their operating model—questions such as how to create a superior employee value proposition to compete with hedge funds and merchant traders for the right talent, and what governance and risk management structures should be put in place to address new data-driven strategies.
North American power and gas markets are in the middle of a paradigm shift. Given the level of coming disruption, every type of player—whether incumbent or new entrant—will likely need to make assertive moves and develop the capabilities that will equip them to manage risks, withstand and benefit from volatility, and build competitive advantage in the power and gas trading markets of tomorrow.
