In the Spring of 2018, The Middleware and Grid Interagency Coordination Team (MAGIC) held a four-part speaker series to examine the impact of containerization and virtualization technologies on computing ecosystems. The presentations and discussions focused on: (1) various packages that are in common use, (2) adoption and usage of container technologies in science communities, (3) adoption and deployment by resource providers, and (4) research and operational deployment challenges.
MAGIC is a public team under NITRD’s Large Scale Networking Interagency Working Group. The team is comprised of technical experts from academia, industry, and government, and is led by co-chairs, Richard Carlson, Department of Energy, and Vipin Chaudhary, National Science Foundation. They meet monthly to discuss current and future activities in the distributed computing environment.
On February 7, MAGIC kicked off the series with presentations on current containerization and virtualization technologies:
- Canon, Shane. Containers for HPC, NERSC.
- Godlove, Dave. Singularity Containers, Singularity.
- Kurtzer, Greg. An Introduction to the Company Behind Singularity, Sylabs, Inc.
- Snavely, Cory. NERSC Spin Project Overview, NERSC.
- Thain, Douglas. Strategic Overview of Docker, University of Notre Dame.
- Wagner, Rick. Kubernetes Overview, Globus.
On March 7, MAGIC hosted presentations on the adoption and use of container technologies in science communities:
- Cinquini, Luca. The Magic of Dreams, NASA Jet Propulsion Laboratory (JPL).
- Clark, James. Containing BayesWave: Deploying LIGO Analyses on the OSG, Laser Interferometer Gravitational-Wave Observatory (LIGO), Georgia Institute of Technology.
- Crusoe, Michael R. Common Workflow Language: Portable Workflow Creation and Deployment with the CWL Standards, a project of the Software Freedom Conservancy.
- Parkinson, Dula. Synchrotrons and Supercomputers, Advanced Light Source (ALS), Berkeley Laboratory.
- Spentzouris, P; Holzman B; Tiradani, A. Container Technology for HEPCloud, DOE Fermi National Accelerator Laboratory (Fermilab).
- Yang, Wei. Containers at US Atlas, SLAC National Accelerator Laboratory, Stanford University.
On April 4, MAGIC featured presentations on the adoption and deployment by resource providers:
- Bockelman, Brian. Moving CMS to a Container-Based Infrastructure, Large Hadron Collider (LHC), University of Nebraska-Lincoln.
- Donenfeld, Yaniv. Amazon ECS, Container Services, Amazon Web Services.
- Hoge, Chris. OpenStack and Container Integrations, OpenStack Foundation.
- Papadopoulos, Philip. Comet Virtual Clusters and Containers, San Diego Supercomputer Center.
- Simpson, A; Jenson, D; Punzel, J. Singularity: DOE LCF HPC Container Usage, Oak Ridge and Argonne Leadership Computing Facilities.
- Wuerthwein, Frank, Containers on OSG, Open Science Grid (OSG), San Diego Supercomputer Center.
On May 2, the speaker series concluded with discussions led by the following experts on the impact of containers on science, system development and standardization activities, the issues involved in building and deploying containers, security concerns, funding issues, and long-term solutions:
- Altintas, Ilkay. Chief Data Science Officer, San Diego Supercomputer Center, Division Director, Cyberinfrastructure Research, Education, and Development Director, Workflows for Data Science Center of Excellence
- Blaiszik, Ben, A Data Ecosystem to Support Machine Learning in Materials Science, University of Chicago, Globus and Argonne National Laboratory, Data Science and Learning Division
- Lockwood, Glenn. Simplifying Data Management Through Storage System Design, Storage Architect. Advanced Technologies Group, National Energy Research Scientific Computing Center (NERSC), Lawrence Berkeley National Laboratory
Meeting minutes are available for the February through May meetings in this MAGIC speaker series.
MAGIC holds public meetings on the first Wednesday of each month (12-2 pm ET).