Linear Attention w/ Yikang Shen
NeurIPS
OVERVIEW
Join us for our fourth research talk lunch event in San Diego at NeurIPS with Yikang Shen (Pre-Training Lead at xAI). Lunch will be catered for all attendees!
For this session, we'll be discussing Yikang's work in Linear Attention and the DeltaNet, along with their applications:
Gated Linear Attention (link), DeltaNet (link), Gated DeltaNet (link)
Usage in latest foundation models like Qwen3-Next (link) and Kimi Linear (link)
We'll gather ~25-30 folks of different research backgrounds together to go through Yikang's journey and applications of his work to current foundation models.