Just Move It! Dynamic Parameter Allocation in Action

We demonstrate Lapse, a parameter server (PS) with dynamic parameter allocation (VLDB '20 paper, source code). Lapse allocates parameters dynamically, i.e., it can re-allocate parameters among cluster nodes during run time. This enables Lapse to exploit techniques that reduce communication overhead. In our experiments, Lapse was 4-203x faster than other PSs (which allocate parameters statically).

We provide a guided tour that takes you through a series of steps to learn about static and dynamic parameter allocation and Lapse.  

In the plots below, these colors show at which node a parameter is allocated:
node 1
node 2
node 3
node 4
node 5
node 6
node 7
node 8

Allocation Replay

This analysis replays parameter allocations. Each square corresponds to one parameter. The color of the square indicates at which node the parameter is currently allocated. You can adjust replay speed and jump to any point in time, using the progress bar at the bottom.

Time:
0 sec

Allocation over Time

This analysis plots parameter allocations over time. Each row corresponds to one parameter, the x-axis depicts time, and colors indicate the current allocation of a parameter.

th selected parameter

Parameter Affinity

This analysis calculates node affinity of parameters. It calculates the time a parameter spent at each node and the number of relocations of the parameter to each node. Click any of the parameters in the two visualizations above to calculate affinity for it.

Parameter Time spent at each node (in %) Number of relocations to each node
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8