ReLU訓練における特異曲率：微分と勾配流極限の非可換性

arXiv cs.LG ・ 2026-08-31

原題: Singular Curvature in ReLU Training:Differentiation and the Gradient-Flow Limit Need Not Commute

AI による要約

Gradient descent (GD) is explicit Euler for gradient flow, but a state-accurate continuous-time surrogate need not remain accurate after differentiation. At every fixed nonresonant step size, ordinary automatic differentiation exactly differentiates the executed hard-ReLU GD program. We prove that, over a fixed finite horizon, the GD states converge and these exact discrete derivatives approach an event-free regional propagator, whereas the derivative of the limiting flow also contains speed-normalized activation-event transfers. A prepoint Stieltjes representation separates the absolutely continuous regional Hessian from atomic interface curvature; one nonzero gradient jump produces an exactly rank-one endpoint discrepancy, and global convexity prevents complete multi-event cancellation whenever an event is strict. Nevertheless, a standard family of globally 1-strongly convex residual-R

この要約は当サイトの AI が生成したものです。正確な内容は 元記事（arXiv cs.LG）をご確認ください。