Language Is Not All You Need: Aligning Perception with Language Models - Microsoft Research

https://www.microsoft.com/en-us/research/publication/language-is-not-all-you-need-aligning-perception-with-language-models/