in | | / \ / \ / \ W(x) V(x) | | | ↓ | ReLU² | | \ / \ / \ / ⊗ | ↓ out


researcher at zyphra
currently working on novel pretraining architectures and optimizers
-Rishi

Feel free to email me, I am almost always interested in meeting new people.

recent work

Compressed Convolutional Attention
'_'