Most Deep Learning systems are implemented in domain-specific languages (DSLs) often called “frameworks”. Some of these, like Caffe and Darknet, have been custom DSLs. Most, however, have been embedded in other standard general-purpose programming languages (i.e. Torch 7 in Lua, PyTorch and TensorFlow in Python). No widely used Deep Learning framework, however, is embedded in Scheme. This is a missed opportunity. In this talk I will present two specific case studies where the Scheme mindset is particularly conducive to building a powerful and performant Deep Learning framework. The first involves using partial evaluation by way of flow analysis to migrate run-time reflective source-code transformation to compile time to allow extremely fast but convenient computation of gradients. The second involves using CPS transformation to implement “engines” (Haynes & Friedman 1984) to in-turn implement divide-and-conquer checkpointing for computing gradients of extremely deep neural networks. These allow easily performing computations that would be tedious, if not impossible, in traditional frameworks.

Jeffrey Mark Siskind received the B.A. degree in computer science from the Technion, Israel Institute of Technology, Haifa, in 1979, the S.M. degree in computer science from the Massachusetts Institute of Technology (M.I.T.), Cambridge, in 1989, and the Ph.D. degree in computer science from M.I.T. in 1992. He did a postdoctoral fellowship at the University of Pennsylvania Institute for Research in Cognitive Science from 1992 to 1993. He was an assistant professor at the University of Toronto Department of Computer Science from 1993 to 1995, a senior lecturer at the Technion Department of Electrical Engineering in 1996, a visiting assistant professor at the University of Vermont Department of Computer Science and Electrical Engineering from 1996 to 1997, and a research scientist at NEC Research Institute, Inc. from 1997 to 2001. He joined the Purdue University School of Electrical and Computer Engineering in 2002 where he is currently a professor. His research interests include computer vision, robotics, artificial intelligence, neuroscience, cognitive science, computational linguistics, child language acquisition, automatic differentiation, and programming languages and compilers.