Optimize DMR to Minimize Performance Impact During Drawer Swaps

On IBM z16 servers, we have observed a significant increase in the cost of accessing data/memory located in a remote drawer. In certain scenarios, this can result in a notable increase in CPI, leading to an LPAR’s CPU utilization doubling -or even tripling- compared to its baseline without a proportional increase in workload.

To address this issue holistically, we are submitting a series of three related ideas. This is the second one of these three and focuses on improving the efficiency of the Dynamic Memory Relocation (DMR) process itself.

In our experience, the performance impact of a DMR operation exhibits an “all-or-nothing” behavior. That is, the CPU utilization increases sharply at the beginning of the drawer swap and remains consistently elevated throughout the entire relocation process. Typically lasting 10 to 15 minutes without degrading gradually over time and then it suddenly gone. This suggests that memory operations continue to access the source drawer until the full relocation completes.

We propose the development of an enhanced DMR mechanism -possibly bitmap based- that can:

• Prioritize frequently or recently accessed pages during relocation 
• Enable early redirection of memory access to the target drawer as soon as those frames are copied.

In typical z/OS workloads, memory access patterns are not uniform, and cache hit ratios tend to be high. We guess, during DMR, only a small subset of memory frames (possibly under 1-2%) are actively accessed. If these pages could be prioritized and made accessible from the target drawer immediately after being copied, the performance impact might be significantly reduced potentially 100 to 1000 times less than what it was today.

These numbers are rough estimates, not based on formal measurements or calculation, but potential for performance improvement is clear. Implementing such optimizations would reduce the operational impact of drawer swaps and enhance workload stability during dynamic memory topology changes.