Direct nonlinear acceleration
Optimization acceleration techniques such as momentum play a key role 9.5-4 igora vibrance in state-of-the-art machine learning algorithms.Recently, generic vector sequence extrapolation techniques, such as regularized nonlinear acceleration (RNA) of Scieur et al.[22], were proposed and shown to accelerate fixed point iterations.In contrast to RNA