August 2003


ORION enables first practical use of international end-to-end lightpath

700 GBs transport cut from 67 days to 6.5 hours

In an historic data transmission test, ORION’s new optical infrastructure helped reduce the transfer of 700 GBs of data from Ontario to Europe in just over six hours recently, a massive transport that would have normally taken 67 days of continuous transmission.

The test involved the Canadian ATLAS experimental group which needed to transport the data from an experimental test site at CERN in Geneva back to the Physics Departments of Carleton University and the University of Toronto.

A recent test at the CERN laboratory provided data to calibrate a set of four-ton, million-dollar particle detectors that were constructed in Canada and the U.S.

The extensive analysis of this data to be undertaken by the Canadian groups required that the data be shipped back to Canada and accessible on local computer systems. The volume of data made it impractical to do this over existing networks.

To solve this problem, a unique arrangement was orchestrated between CANARIE in Canada, SURFnet in the Netherlands, CERN in Switzerland, StarLight in the United States, ORION in Ontario, Carleton University in Ottawa, and the shared facilities of TransLight.

A set of optical lambdas were configured to provide an end-to-end lightpath to facilitate the transfer of 700 GBs of data in what can be considered the first real practical use of an international end-to-end lightpath.

The 700 GBs of test beam data from an experimental area at the European laboratory CERN to the Department of Physics at Carleton University was successfully transferred in about 6.5 hours overnight between July 16 and July 17. To transfer data of this magnitude over the normal Internet network connection through firewalls and campus networks would have taken 67 days continuously.

The end-to-end lightpath, a point-to-point optical link, between CERN and Carleton University is a realization of an emerging and core networking paradigm of CA*net 4 and other experimental infrastructure networks for enabling data and computationally intensive research. This "optical bypass" provides a fast long wide pipe for high-speed data transport, a fundamental requirement for large-scale scientific experiments and global scale Grids.

Through the months of June and July, the Canadian and U.S detector components were exposed to a test beam at CERN using one of CERN's high-energy proton beam facilities. The detectors are part of an energy measuring system and are referred to as the Forward Calorimeters or FCALs.

The FCALs, which are an integral part of the $500 million ATLAS detector being constructed for the Large Hadron Collider (LHC) at CERN, were designed and built at Carleton University, the University of Toronto and the University of Arizona.

The data will be transferred from Carleton University to the University of Toronto and the University of Arizona as soon as similar lightpaths can be established to these universities. All three groups will be analyzing the data to determine the performance of the detectors and establishing optimum software analysis tools to be used when the ATLAS detector begins acquiring data in 2007. Further FCAL test data will be transmitted over this facility in the coming months.

The lightpath was established with a capacity of one Gbps end to end between the FCAL Linux data acquisition PC at CERN in Geneva and a three Terabyte Linux disk server at Carleton University Physics Department in Ottawa. Due to the small time window to actually transport the data during the running experiment, there was no opportunity to tune the network and I/O performance of the servers. Tsunami, an experimental high-speed network file transfer protocol developed by the Advanced Network Management Laboratory at Indiana University was used as the transfer protocol. An average rate of about 250 Mbps was obtained.

This link was made possible with the contributions and support from Ralph Michaelis and his staff at Carleton University. ORION, Ontario's Optical Regional Advanced Network (ORAN), provided use of the fibre between Carleton University and CA*net 4 for this experiment.

ORION’s Sam Mokbel and Randy Neals were instrumental in enabling this. Damir Pobric and Thomas Tam from CANARIE coordinated the provisioning of the lightpath between CERN and Carleton University, as well as helping to isolate local high dispersion fibres, which were problematic. Dale Theoret and Verna Murray from Telecom Ottawa, and Michael MacKenzie from Group Telecom assisted in identifying and interconnecting fibres at the Ottawa ORION PoP and at the Ottawa CA*net4 PoP, respectively.

This experimental use of an end-to-end lightpath was conducted as part of the Carleton University led CA*net 4 International Grid Testbed, supported under CANARIE's Directed Research Program.

ORANO’s President and CEO Phil Baker is looking forward to more practical research uses of the ORION network. “That’s part of our mandate,” he said. “As one of the world’s largest and most advanced optical research networks, ORION certainly wants to collaborate with the international research community, and remain at the cutting edge of technology and innovation in the field,” he said.


Back to Headlines