Adapting Memory Hierarchies for Emerging Datacenter Interconnects
Journal article, 2015
Efficient resource utilization requires that emerging datacenter interconnects support both high performance communication and efficient remote resource sharing. These goals require that the network be more tightly coupled with the CPU chips. Designing a new interconnection technology thus requires considering not only the interconnection itself, but also the design of the processors that will rely on it. In this paper, we study memory hierarchy implications for the design of high-speed datacenter interconnects particularly as they affect remote memory access and we use PCIe as the vehicle for our investigations. To that end, we build three complementary platforms: a PCIe-interconnected prototype server with which we measure and analyze current bottlenecks; a software simulator that lets us model microarchitectural and cache hierarchy changes; and an FPGA prototype system with a streamlined switchless customized protocol Thunder with which we study hardware optimizations outside the processor. We highlight several architectural modifications to better support remote memory access and communication, and quantify their impact and limitations.
datacenter network
time shared memory
memory hierarchy
high-speed interconnect
Author
Tao Jiang
Institute of Computing Technology Chinese Academy of Sciences
Chinese Academy of Sciences
Rui Hou
Institute of Computing Technology Chinese Academy of Sciences
Jianbo Dong
Institute of Computing Technology Chinese Academy of Sciences
L. Chai
Chinese Academy of Sciences
Institute of Computing Technology Chinese Academy of Sciences
Sally A McKee
Chalmers, Computer Science and Engineering (Chalmers), Computer Engineering (Chalmers)
B. Tian
National High Performance Integrated Circuit Design Center (Shanghai)
L. Zhang
Institute of Computing Technology Chinese Academy of Sciences
N. H. Sun
Institute of Computing Technology Chinese Academy of Sciences
Journal of Computer Science and Technology
1000-9000 (ISSN) 1860-4749 (eISSN)
Vol. 30 1 97-109Subject Categories (SSIF 2011)
Computer and Information Science
DOI
10.1007/s11390-015-1507-4