CloudNativeCon + KubeCon Europe 2017

OpenAI is a non-profit research company that does cutting-edge AI research. Our mission is to build safe AI, and ensure AI's benefits are as widely and evenly distributed as possible. This means democratizing the technology and releasing our research publicly. As a result, we rely heavily on open-source software. The majority of our experiments run on our Kubernetes cluster that spans Azure, AWS, and our own data center. Kubernetes and Docker have allowed us the flexibility to experiment with various computing frameworks and topologies without paying the infrastructure cost. However, our use cases are distinctly different from the well-supported microservice use case, and we've written custom components on top of Kubernetes to optimize for our work. Some examples include our own autoscaler for batch jobs, a library to deploy distributed Tensorflow jobs, custom scripts to do GPU-scheduling and CPU-affinity, and a variety of internal tools to make Kubernetes friendly to researchers who have no experience in operations. In this talk, we will go over some of the motivations and internals of our customizations, as well as an example of how they all come to work together to accelerate research on the Universe platform.