ESnet Prepares for U.S. Arrival of Big Bang Machine Data


When the Large Hadron Collider (LHC), a gigantic particle accelerator spanning the French-Swiss boarder, comes online in September 2008, terabytes of data per second will flow out of Europe via fiber optic cables to thousands of researchers spread across the globe. This experiment will significantly increase the amount of data that the international scientific community must transport and manage.

Last week, preparations for the LHC data U.S. arrival continued at a torrid pace as engineers from the Department of Energy’s Energy Sciences Network (ESnet) added bandwidth, updated hardware, and installed a new high-speed hub to the existing science data transport network around Brookhaven National Laboratory (BNL) in Upton, N.Y. BNL is one of only two facilities in the U.S. that will process and store all data collected by the LHC’s ATLAS detector, then make it available to researchers in the U.S.

“The upgrade was extremely tricky,” says Joe Burrescia, general manager of ESnet. “It was like changing the tires on a moving car. This network is a critical component of the work currently being done by thousands of researchers across the country, and we had to install hardware and update the system without disrupting their current work flow.”

According to Evangelos Chaniotakis, an ESnet engineer, coordination for the hardware upgrade took months of planning. ESnet engineers meticulously mapped out multiple “detours” for each failure scenario to ensure all information traveling across the new network infrastructure would flow uninterrupted.

Like a virtual Ellis Island, the new hub installed on 8th Street in Manhattan, N.Y, is a primary U.S. entry point for data collected by the LHC’s ATLAS detector. From here, the data will go to BNL for processing and storage. Meanwhile, the other entry point at Fermi National Accelerator Laboratory (FNAL) in Batavia, Ill., will process and store data collected by the LHC’s CMS detector. ATLAS and CMS are the two detectors inside the LHC.

New high-speed connections were also installed to both the IP and the Science Data Network, which connects the 8th Street hub to BNL. Like an expressway, terabytes of information will be able to flow uncongested through these lines directly to BNL without any interference from other network traffic.

“With only a small but experienced team, we were able to make the enhancements without the end users ever perceiving the significant changes happening across the network,” said Chaniotakis. “This was a complete success.”

Researchers across the country will be able to access the LHC data stored at BNL and FNAL through ESnet 4, the new large-scale science data transport network with enough bandwidth to transport multiple 10 gigabits of information per second – the equivalent of transmitting 500 hours of digital music per second for each 10 gigabit line. The LHC has been nicknamed the “Big Bang Machine” because scientists will use it to recreate the cosmic conditions one trillionth of a second after the big bang, in hopes of finding insights into the origins of matter.