Snapshot: Scaling a National Research Platform

Based on community input and on ESnet’s Science DMZ concept, the National Science Foundation (NSF) has funded over 100 campuses to deploy Science DMZs to build local big-data freeways starting in 2011. The Science DMZ is a portion of the network, built at or near the campus or laboratory’s local network perimeter that is designed such that the equipment, configuration, and security policies are optimized for high-performance scientific applications rather than for general-purpose business systems or “enterprise” computing.

In 2015, NSF awarded a five-year, $5 million grant to fund the creation of a regional end-to-end science driven big-data highway system, called the Pacific Research Platform (PRP). Led by researchers at UC San Diego and UC Berkeley, PRP is a partnership of more than 50 institutions that includes all 10 University of California campuses, the National Science Foundation, Department of Energy, and multiple research universities in the U.S. and around the world. The PRP’s data-sharing architecture utilizes FIONAs (Flash I/O Network Appliance): clustered Data Transfer Nodes (DTNs) carefully optimized for storing, forwarding or using large amounts of data (10s to100s of terabytes) on 10, 40, or 100 gigabit per second networks. This regional big-data superhighway system enables researchers to move data between labs and collaborators’ sites, supercomputer centers, or data repositories without performance degradation.

Inspired by the PRP project activities and goals, multi-partner collaborations across the U.S. are underway to provide new and transformative capabilities for high-performance science applications. Collectively, these efforts are serving as pilot testbeds to push toward a National Research Platform or NRP. The NRP’s primary purpose is to accelerate scientific research and discovery through building networks of campus Science DMZs that scale across geographies to create an end-to-end science-driven “big-data superhighway” system through partnerships in science engagement and technology adoption. The vision of the NRP goes beyond simply unifying the nation’s high-speed research networks. It aspires to serve as a cloud-like, distributed supercomputer for the nation and beyond.

Leveraging their positions as trusted collaborators as well as regional network aggregators for their community institutions, Quilt members are actively involved in efforts to scale to a national research platform. Here’s a snapshot of several of these efforts.

Full-length articles for Great Plains Network, NJEdge, and Florida LambdaRail about their research platform initiatives are featured in the 2019 Quilt Circle.

Lonestar Education and Research Network (LEARN) – Texas
The PRP has connected universities and labs in California, Oregon, and Washington, and, more recently spread eastward to Chicago and Colorado and westward to Hawaii and the Pacific Rim. The NRP continues the network’s march south and east across the U.S.

The PRP worked with Texas, Oklahoma and the Great Plains Networks to extend a really fast backbone link further east,” said Akbar Kara, LEARN’s CTO. “Together we received additional funds from NSF to bring in a 100-gigabit facility that came into Houston and Dallas, where we have points of presence. LEARN connected to both Dallas and Houston, so researchers can tap into resources connected to new Pacific Research Network facilities.

LEARN is directly connected to Pacific Wave via a 200 Gbps network that can carry traffic internationally to enable greater collaboration across borders. LEARN plays an important role as a major regional testbed. It will help the NRP develop and test use cases for difficult data transfer needs with hardware like the LEARN-routed FIONA, located at the Texas Advanced Computing Center (TACC), which began operations in February 2019. New collaborations that were previously difficult will emerge because of the improved ability to collaborate, share, and access computing and scientific datasets instruments. [Learn more…]

Eastern Regional Network – CAAREN, Connecticut Education Network, KINBER, NetworkMaine, OARnet, OSHEAN, NJEdge, and NYSERNet
Through a partnership of eight Quilt member regional network providers, numerous higher education institutions, research facilities, and Internet2, the Eastern Regional Network (ERN) provides layered and transparent access to shared data and computing facilities for research projects located at and between partner sites. The resulting regional research and education platform will support a diverse set of science drivers and emergent educational opportunities and offer the educational research community access to a broad range of collaborative multi-institutional resources that are not available on any one campus alone. The goal of this multi-institution, multi-state collaboration is to create a prototype for a policy-based federation of research resources that span participating campuses and commercial cloud resources. The prototype provides the ability to reserve network slices and launch containerized jobs as well as traditional HPC jobs, from any site to any other site including commercial cloud, via Simple Linux Utility for Resource Management (SLURM) schedule. [Learn more…]

New York State Education and Research Network (NYSERNet)
NYSERNet is participating in a pilot to scale the PRP into a national testbed (NRP). The technology consists of Data Transfer Nodes (DTNs), Compute Nodes, and a distributed Ceph storage environment acting in concert to enable inter-institutional high-speed research data transfers on a federated computing platform offering Intel and GPU processing capacity. Not designed to compete with multi-million core Supercomputers, this is an environment based on federated access into Kubernetes Linux containers providing a temporary virtual compute and storage environment, suitable for smaller research workloads. Thus far, NYSERNet has deployed the DTN component of the NRP Pilot in both its Syracuse Data Center and at Stony Brook University. NYSERNet, in collaboration with Stony Brook, is implementing a Science DMZ network and supporting research needs such as improving data transfers to XSEDE via the DTN. NYSERNet has also started collaborating with the American Museum of Natural History on an NSF-funded project to design a Science DMZ which supports NRP Pilot activities including facilitating an astrophysicist’s data transfers with the Palomar Observatory and providing a compute platform. [Learn more…]

Florida LamdaRail
Florida LambdaRail’s (FLR) Regional Science DMZ is the basis for the Atlantic Research Platform (ARP), a science driven, high capacity data exchange and transport facility that links campuses to other regional, national and global science DMZs. The FLR Regional Science DMZ and ARP provide participants the ability to extend and interconnect campus research, high-performance computing, and instructional technology environments to the global research and education networking fabric. Utilizing the capabilities of the Internet2 Advanced Layer 2 Services Network, the FLR Science DMZ now peers with the PRP thereby extending the Science DMZ concept from coast to coast. [Learn more…]

Great Plains Network Research Platform
The Great Plains Network Research Platform (GPN RP) began with a goal to draw together different groups of people to create a seamless research platform that encourages collaboration on a broad range of data-intensive fields and projects. The end goal, like the PRP and other regional efforts, is to create a seamless research platform that encourages collaboration on a broad range of data-intensive fields and projects. GPN is accomplishing this by closing the gap between a number of communities including scientific equipment, high performance computing (HPC), storage, and the networks that connect them. This ultimately increases access to the research cyberinfrastructure across the region. [Learn more…]

NJEdge – New Jersey
NJEdge has strategically advanced its capacity and capability to support and advance research in New Jersey and beyond with a research as a service (RaaS) solution called EdgeDiscovery. NJEdge has been fortunate to partner with Rutgers University and the Rutgers Office of Academic Research Computing (OARC), which has worked collaboratively with other institutions, organizations, and networks in the region to spearhead the Eastern Research Network proof of concept and demonstrate the viability of the new construct. Recognizing the new model’s potential to democratize access to research capacity and capability for less well-resourced institutions, NJEdge regards these developments as an opportunity to bring value to the community with EdgeDiscovery. EdgeDiscovery combines an NJEdge research network DMZ with an optimized technology stack residing on our backbone, and is outfitted with hardware and software components which enable researchers accessing it to streamline their research projects, schedule compute cycles, and access applications, analytics tools, and storage resources. [Learn more…]