A Deep Dive into Deep Learning Techniques: A First-of-its-kind Hands-on Workshop

A diverse team of researchers came together to deliver a first-of-its-kind Introduction to Cloud-based Deep Learning, a virtual workshop applying Deep Learning techniques to agricultural datasets. Deep Learning (DL) is a subset of Machine Learning (ML) methods that utilizes neural networks to mimic the ability of a human brain to learn novel patterns from large amounts of data, resulting in trained models which can rapidly identify similar patterns in new datasets. Training these models with large amounts of data requires specialized hardware and software components, making DL-based analysis an ideal candidate for cloud-based workshops which offer both on-demand configurable learning environments and scalable computational resources such as GPUs. The workshop was held on April 9, 2022 and was attended by more than 40 participants.

Image

The workshop team, comprising TrAC, CyVerse, and Jetstream2, shared a common goal of training researchers: for CyVerse, to use cutting-edge computational resources for managing large datasets and complex analyses; for Jetstream2, for computational capacity and GPU support for DL/ML analyses and workflows; for Iowa State University’s Translational AI Research and Education Center (TrAC), to bring together core Iowa State artificial intelligence (AI) researchers across disciplines.

Two TrAC projects, funded by the National Science Foundation (NSF) and the United States Department of Agriculture (USDA), were instrumental in identifying the need for this training. The USDA-funded AI Institute for Resilient Agriculture (AIIRA), a collaborative project involving eight universities and organizations across the U.S., aims to build and utilize AI/ML based techniques; and the NSF-funded COntext-Aware LEarning for Sustainable CybEr-agricultural systems (COALESCE) brings Cyber-Physical Systems principles to sustainable agriculture.   

“Training participants with real scale data and tools, more importantly ensuring access to the same resource so they can continue their learning journey and work with their own data, is essential. Jetstream2 and CyVerse provide that platform for democratizing access to popular data science tools and specialized hardware which otherwise are not accessible or affordable for most researchers and educators,” said Nirav Merchant, director of the Data Science Institute at the University of Arizona and Co-PI for CyVerse and Jetstream.Many big tech firms have commercial solutions for Machine Learning, Artificial Intelligence, and Deep Learning but these products have a steep learning curve and often high cost to access. “Most of the current advancements in AI at scale have been limited to the big organizations which have abundant access to lots of GPUs. Furthermore, these advancements were happening specifically in a very few limited domains of science. Most of the resources available for bringing awareness of AI to other domain experts are very limited,” said Aditya Balu, a Data Scientist at Iowa State University and workshop instructor. 

Image

Commercial and proprietary closed source platforms for Machine Learning, Artificial Intelligence, and Deep Learning have a steep learning curve and often high cost to access. “Most of the current advancements in AI at scale have been limited to the big organizations which have abundant access to lots of GPUs. Furthermore, these advancements were happening specifically in a very few limited domains of science. Most of the resources available for bringing awareness of AI to other domain experts are very limited,” said Balu.

The team collaborated to develop this workshop to meet this need. “This was the first time we were able to easily provide so many students with a GPU-enabled container so quickly. Normally, getting GPU resources on an HPC scheduler, like OnDemand, takes time, and the high demand for GPUs makes finding 40 or more unoccupied resources an impossibility,” said Tyson Swetnam, CyVerse Co-PI and workshop instructor. “Furthermore, the A100 GPUs on Jetstream2 represent the newest generation of tensor processor cores from NVIDIA, which allowed participants to work with the latest CUDA-based analysis code.” 

“This training tutorial on cloud-based DL helped us bring people from several domains such as materials, manufacturing, agricultural science, bioinformatics and computational biology, virtual reality and many more.” Balu goes on to explain, “The tutorial also had a diverse set of participants including postdocs, graduate students, undergraduate students and even a high school student.”

The training workshop’s compute-intensive needs were supported by NSF-funded Jetstream2 which provides cloud-based, on-demand computing resources to communities that have not had previous HPC experience or an easy route to harness the massive compute resources essential for AI/ML through XSEDE, the national Extreme Science and Engineering Discovery Environment.

The CyVerse team helped to create a containerized environment with all of the latest ML/AI software applications used by the TrAC team. CyVerse used their new Cloud Automation and Continuous Analysis Orchestration (CACAO) platform to start and customize the virtual machines with containers across multiple Jetstream2 GPU nodes. 

For the Deep Learning workshop, CyVerse built on the success of its Container Camp and Foundational Open Science Skillsworkshops and extensive training expertise. The workshop used the CyVerse Discovery Environment, a web-based workbench, for container orchestration and data management. Multi-cloud launching of GPU-enabled JupyterHubs that were integrated with GitLab were also available. CACAO provided the means to assign workshop participants to individual JupyterHubs.

“This tutorial made some of the latest software stack built by CyVerse and newest generation hardware tools from Jetstream2 available to the researchers in different denominations and has empowered them to boldly perform research in deep learning/AI”, said Balu.

Demand continues to grow for training on machine learning and deep learning techniques and readily available compute resources to provide that training in a classroom setting. “CyVerse will provide the push-button automation used for the TrAC deep learning workshop for other workshops as part of CACAO. Instructors will be able to apply for GPU allocations through Jetstream2,” noted Edwin Skidmore, CyVerse Director of Infrastructure.

The TrAC and CyVerse teams are also working on future joint training opportunities. CyVerse is offering its highly successful Container Camp in May 2022, where attendees cab learn how to create and maintain their own containers. This year’s Advanced Container Camp will include new training on GPU-enabled JupyterHubs and Kubernetes using the CyVerse Discovery Environment and CACAO. 

More about the Workshop Team: 

TrAC:  Iowa State University’s Translational AI Research and Education Center, or TrAC, aims to break down disciplinary silos by bringing together core Iowa State artificial intelligence researchers and domain experts interested in applying new technologies to their work.

Twitter: @traciastate | YouTube: Translational AI Center - Iowa State University

Jetstream2: Jetstream2, led by the Pervasive Technology Institute at Indiana University, adds cloud-based, on-demand computing and data analysis resources to the national cyberinfrastructure.

Twitter: @jetstream_cloud | YouTube: IU Pervasive Technology Institute

AIIRA: As a USDA-NIFA AI Institute, the AI Institute for Resilient Agriculture has a vision to create new AI-driven, predictive digital twins for modeling plants, and deploy them to increase the resiliency of the nation's agricultural systems. 

Twitter: @AII4RA | Instagram: @aii4ra  | YouTube: (104) AIIRA - AI Institute for Resilient Agriculture - YouTube

COALESCE: COntext-Aware LEarning for Sustainable CybEr-agricultural systems is a trans-disciplinary team seeking to bring Cyber-Physical Systems (CPS) principles to sustainable agriculture. Their mission is to disrupt the current agricultural practices with CPS innovations to enhance efficiency, resiliency, sustainability and autonomy.