by Staff Writers
Pasadena, Calif. (UPI) Oct 22, 2013
NASA says new strategies will be needed to manage the ever-increasing flow of large and complex data streams from the agency's many space missions.
Dozens of missions pour in data every day like rushing rivers -- data that need to be stored, indexed and processed so spacecraft engineers, scientists and people across the globe can use the data to understand Earth and the universe beyond, the agency said.
For NASA missions, hundreds of terabytes -- one terabyte is equivalent to the information printed on 50,000 trees worth of paper -- are gathered every hour, creating what the technology community dubs "big data."
"Scientists use big data for everything from predicting weather on Earth to monitoring ice caps on Mars to searching for distant galaxies," Eric De Jong of NASA's Jet Propulsion Laboratory in Pasadena, Calif., said.
De Jong is the principal investigator for NASA's Solar System Visualization project, which converts NASA mission science into visualization products that researchers can use.
"We are the keepers of the data, and the users are the astronomers and scientists who need images, mosaics, maps and movies to find patterns and verify theories," he said.
Scientists face three challenges in dealing with the huge amounts of data from space missions, he said -- storage, processing and access.
Rather than build more hardware for storage, engineers are developing creative software tools to better store the information, such as "cloud computing" techniques and automated programs for extracting data.
For processing, JPL has been increasingly turning to open-source software, creating improved data processing tools for space missions.
"We don't need to reinvent the wheel," said Chris Mattmann, a principal investigator for JPL's big-data initiative. "We can modify open-source computer codes to create faster, cheaper solutions."
Huge amounts of data, stored and processed, are still of little use if it can't be easily accessed, the researchers said.
"If you have a giant bookcase of books, you still have to know how to find the book you're looking for," said Steve Groom, manager of NASA's Infrared Processing and Analysis Center at the California Institute of Technology in Pasadena.
NASA's "big data" work is intended to make it easy for users to grab what they need from the giant data archives, JPL said.
Space Tourism, Space Transport and Space Exploration News
|The content herein, unless otherwise known to be public domain, are Copyright 1995-2014 - Space Media Network. AFP, UPI and IANS news wire stories are copyright Agence France-Presse, United Press International and Indo-Asia News Service. ESA Portal Reports are copyright European Space Agency. All NASA sourced material is public domain. Additional copyrights may apply in whole or part to other bona fide parties. Advertising does not imply endorsement,agreement or approval of any opinions, statements or information provided by Space Media Network on any Web page published or hosted by Space Media Network. Privacy Statement|