Bengali
As data centers aim to boost energy efficiency—especially amid the demands of AI workloads—many are adopting liquid cooling to enhance performance and control energy use. Liquid cooling effectively manages the heat produced by high-performance servers, allowing them to run at maximum capacity without the high energy costs linked to traditional air cooling. Solidigm’s high-density SSDs are well-suited for these settings, delivering excellent terabyte-to-watt efficiency.
YouTube video player
While AI is driving many data center operators to consider liquid cooling, its benefits extend far beyond that. In a previous report, we analyzed how liquid cooling affects a 2U Dell PowerEdge R760. CoolIT’s direct liquid cooling (DLC) cut server energy use by reducing fan speeds, resulting in a 200-watt power savings. That test focused solely on CPU performance; this time, we took a more storage-focused approach to understand how SSDs influence server power consumption.
Dell PowerEdge Solidigm and CDU
What are NVMe Active Power States?
NVMe power states are preconfigured modes that an NVMe device can switch into to regulate power use and performance. The NVMe specification permits up to 32 power states, each defined by maximum power consumption, entry latency (ENLAT), exit latency (EXLAT), and relative performance metrics. These states are split into operational and non-operational categories. Operational power states, or P-States, let the device handle I/O operations. Non-operational states, or F-States, are used when the device is idle and not processing I/O tasks.
Managing these power states is vital for optimizing the energy efficiency of NVMe devices, particularly in environments where power use is a key concern—such as edge devices and specialized applications like the SSDs on the International Space Station (ISS). For example, the NVMe specification includes features like Autonomous Power State Transition (APST), which enables the device to automatically shift between power states based on current usage and thermal conditions. This helps balance performance and power consumption, ensuring reliable operation in remote or constrained settings. Runtime D3 (RTD3) support allows the device to enter a zero-power idle state, further saving energy when it’s not in use.
NVMe power states are especially valuable when energy efficiency and thermal management are top priorities. In edge devices, for instance, the ability to quickly switch to lower power states during idle periods can greatly reduce energy use—critical for devices operating in remote or harsh environments with limited power. This is made possible through features like PCIe Active State Power Management (ASPM) and low-power states such as L1.1 and L1.2, which minimize power consumption. Managing power and thermal output on the ISS is essential due to its limited, controlled environment. NVMe power states can help throttle SSD power use to manage thermal design power (TDP) and optimize the overall energy budget, ensuring SSDs operate efficiently without overheating.
In these specialized environments, NVMe power states offer a flexible, efficient way to manage NVMe device power consumption. By utilizing these states, devices can balance performance and energy efficiency, making them suitable for a range of applications—from edge computing to space missions. The ability to dynamically adjust power states based on real-time conditions ensures NVMe devices can meet the varying demands of different environments while optimizing for energy efficiency and thermal management.
In addition to NVMe power states, Composite Temperature and Touch Temperature are key to managing the thermal performance of NVMe SSDs in modern enterprise models. Touch Temperature refers to the SSD’s external case temperature. Solidigm has been a pioneer in adopting new, higher Touch Temperature standards. For example, the factory-set Touch Temperature for the Solidigm D5-P5336 is 80°C. This higher limit allows SSDs to be cooled with less airflow or operate in higher ambient temperatures. This flexibility enables data centers to refine cooling strategies, improve overall thermal management, potentially lower cooling costs, and extend SSD reliability and lifespan.
Managing NVMe Active Power States
In a Linux testing environment running Ubuntu 22.04, we used the NVMe toolset to poll the drive and view or modify the D5-P5336’s power states. As shown below, the drive supports states 0, 1, and 2—with state 0 being the least restrictive and state 2 the most restrictive.
For the Solidigm 61.44 TB D5-P5336, PS0 is 25W, PS1 is 15W, and PS2 is 10W. The drive idles at approximately 5.5W, so as power modes become more restrictive, the SSD has less power available for NAND read and write operations. Write operations are most affected, as writing to NAND consumes more power than reading from it.
The command to check the current power state of our Solidigm D5-P5336 SSD is shown below. A current value of 00000000 indicates the drive is in PS0, the highest 25W mode.
A similar command is used to change the power state, with the final number representing the desired power mode. For example, the command below sets the Solidigm D5-P5336 SSD to PS0. To use power modes 1 or 2, adjust the —value= figure to match the correct mode.
Impact of Power States on Performance
To measure how power states affect the power use and performance of the Solidigm D5-P5336 61.44TB SSD, we equipped a Dell PowerEdge R760 with 24 SSDs. Running Ubuntu and the FIO workload generator, we could easily run a consistent workload across all SSDs and adjust the power mode in real time.
Dell PowerEdge Solidigm P5336
We used Dell’s onboard power monitoring within the server’s iDRAC9 management system to track power at the system level.
Dell PowerEdge iDRAC Power
We focused on sequential read and write bandwidth workloads, using a 128K block size per drive, then measured aggregate performance across all 24 SSDs. It’s worth noting that this specific Dell PowerEdge R760 configuration—with 24 NVMe bays—uses a PCIe switch instead of direct-attached NVMe bays. As a result, the total bandwidth measured saturates the available PCIe switch lanes before reaching the drives. This impacts the total read performance we recorded compared to the Solidigm P5536 spec sheet, but aggregate write speeds remained below that limit.
| Total Watts | Write Speed | Read GB/s | Watts Over Base |
Watts/Drive (with system overhead) |
|
|---|---|---|---|---|---|
| Idle No Drives | 462 | – | – | – | – |
| Idle Drives Installed | 594 | – | – | 132 | 5.5 |
| 24x Sequential Read PS0 | 858 | – | 109GB/s | 396 | 16.5 |
| 24x Sequential Read PS1 | 858 | – | 105GB/s | 396 | 16.5 |
| 24x Sequential Read PS2 | 759 | – | 79.8GB/s | 297 | 12.375 |
| 24x Sequential Write PS0 | 1089 | 82.5GB/s | – | 627 | 26.125 |
| 24x Sequential Write PS1 | 825 | 34.4GB/s | – | 363 | 15.125 |
| 24x Sequential Write PS2 | 726 | 17.3GB/s | – | 264 | 11 |
Looking back at our article on the advantages of converting an air-cooled platform to direct-liquid cooling, we observed a slight CPU performance improvement along with a 200-watt power savings. Power is a valuable resource in the new generation of AI-focused servers, which often allocate all available resources to GPUs and high-end CPUs. In data centers at or near their air-cooling power budget limit, switching to DLC frees up power that allows the server to accommodate more SSDs for the same power footprint as an air-cooled server.
solidigm liquid cooling coolit CDU
A 200-watt power savings can significantly boost storage density: for read-intensive workloads, this savings lets you double the storage capacity from 12 to 24 SSDs in a liquid-cooled server compared to an air-cooled one. With the Solidigm D5-P5336, this 24-bay server increases storage capacity from 737TB to 1,474TB thanks to the liquid loop. For write-heavy workloads, you could add about eight more SSDs. These figures apply to base power modes, so if you’re willing to reduce top-end write performance, you could easily equip the server with 24 SSDs even for write-heavy tasks.
Conclusion
Our testing of the Solidigm D5-P5336 SSDs shows that managing NVMe power states can significantly improve energy efficiency without drastically impacting performance. Data center operators seeking to maximize energy efficiency can use these power states to increase storage density or cut operational costs—especially in AI-centric environments where power is scarce. Solidigm’s high-density SSDs are well-suited for this, offering strong terabyte-to-watt efficiency, particularly when paired with modern liquid cooling technologies.
Our findings indicate that even small adjustments to power states can yield substantial power savings, which is critical in environments with limited power availability. Optimizing the overall power consumption of servers not only boosts storage density but also supports more sustainable data center operations.
Dell PowerEdge Solidigm P5336 Single
Power management grows increasingly important as modern servers are pushed to their limits—especially in AI-driven workloads. Combining liquid cooling with efficient SSD management provides a path for data centers to scale performance and storage density without exceeding power budgets.
You can see a full demo of these technologies live at OCP 2024, where we’ll showcase how liquid cooling and Solidigm’s SSDs can serve as the foundation of energy efficiency in the modern data center.
Beijing Qianxing Jietong Technology Co., Ltd.
Sandy Yang/Global Strategy Director
WhatsApp / WeChat: +86 13426366826
Email: yangyd@qianxingdata.com
Website: www.qianxingdata.com/www.storagesserver.com
Sandy Yang/Global Strategy Director
WhatsApp / WeChat: +86 13426366826
Email: yangyd@qianxingdata.com
Website: www.qianxingdata.com/www.storagesserver.com
Business Focus:
ICT Product Distribution/System Integration & Services/Infrastructure Solutions
With 20+ years of IT distribution experience, we partner with leading global brands to deliver reliable products and professional services.
“Using Technology to Build an Intelligent World”Your Trusted ICT Product Service Provider!