In this reading group, Jeff Schulman presents Emergent representations of program semantics in language models trained on programs by Jin and Rinard. This paper offers interesting results, showing that despite the narrow task of next-token prediction, LLMs can interpret programs and exhibit other emergent properties. For those unfamiliar with the term, it refers to phenomenon like tornadoes, where a bunch of air particles behaving in a certain way together at sufficient scale causes devastation.
Presentation Link: https://psu.mediaspace.kaltura.com/media/Group+Meeting/1_udhql9bo