scientific infrastructure

Can Scientific Data be Free?

Jerry Sheehan — Tue, 23 Dec 2008 08:55:15 -0800

Description

With a good deal of scientific fanfare, Google announced last January a new beta program (Google Research Datasets-GRD) to provide free terabytes worth of storage for scientific data. This service was initiated to address

a scientific need for a robust freely available infrastructure that can be used to share the immense amounts of data being created by modern scientific exploration in disciplines as wide ranging as biology and physics. Initial GRD data sets included the 120 terabyte Hubble Space Telescope data and digital images from the text Archimedes Palimpsest.[1]

Less then a year later, with a weakening economy swinging the ax, Google CEO Eric Schmidt announced the company would be cutting back on experimental projects and GRD came to an untimely end. As of December 2008, GRD had over 30 datasets uploaded, all of which now have approximately 1 month to find alternate hosting sites before Google pulls the plug.[2]

The scientific community is deeply disappointed in Google's move. Life scientists have argued that it is another sign that Google isn't interested in helping them on an infrastructure level while Astronomers are hoping they will reconsider their decision as the economic situation improves.[3] But, the bottom line is Google didn't see a business case here and decided to close down the experiment.

Google's decision in in sharp contrast to that taken by one of their up and coming competitors, Amazon. While many may think of Amazon as only the world's largest on-line bookstore, within the last three years they have become a major information infrastructure provider through products such as their Amazon's Web Services (AWS) and Elastic Compute Cloud. Just last Thursday, the company announced that they will be hosting massive amounts of public data including the annotated human genome, US Census data, and 3D renderings of molecules. [4]

The catch? While Amazon is hosting the data for free they will charge users for downloading the data or for any use of it for derivative computation. As they explains, "Previously, large data sets such as the mapping of the Human Genome and the US Census data required hours or days to locate, download, customize, and analyze. Now, anyone can access these data sets from their Amazon Elastic Compute Cloud (Amazon EC2) instances and start computing on the data within minutes. Users can also leverage the entire AWS ecosystem and easily collaborate with other AWS users. For example, users can produce or use prebuilt server images with tools and applications to analyze the data sets. By hosting this important and useful data with cost-efficient services such as Amazon EC2, AWS hopes to provide researchers across a variety of disciplines and industries with tools to enable more innovation, more quickly."[5]

The death of GRD and the rise of the Amazon Web Services for public scientific data is important because it yet another signal that Amazon, not Google, may be the major player in experimentation with cloud computing services and data storage for scientific research.

Science in the United States

Source

[1]"Google to Host Terabytes of Open-Source Science Data", Alexis Madrigal, Wired Science, Jan 18, 2008, http://blog.wired.com/wiredscience/2008/01/google-to-provi.html

[2]"Failure to Launch: Google Research Datasets", Clinton Boulton, Google Watch, December 19, 2008, Google Watch - Failure to Launch - Failure to Launch: Google Research Datasets

[3]"Google Shutters Its Science Data Service", Alexis Madrigal, Wired Science, December 18, 2008, http://blog.wired.com/wiredscience/2008/12/googlescienceda.html

[4]"Amazon Hosting, Crunching Massive Public Databases", Aaron Rowe, Wired Science, December 5, 2008, http://blog.wired.com/wiredscience/2008/12/massive-amounts.html?referer=sphere_related_content

[5] "Public Data Sets on AWS Now Available", Amazon Web Services What's New, http://aws.amazon.com/about-aws/whats-new/2008/12/03/public-data-sets-on-aws-now-available/

Database tools for science -opportunity for collaboration on standards

Patricia Larenas — Mon, 08 Sep 2008 13:49:51 -0700

Description

In the recent Nature issue (Nature 455, 28-29 (4 September 2008) | doi:10.1038/455028a; Published online 3 September 2008) a very interesting special section on "big data" is presented: (http://www.nature.com/news/specials/bigdata/index.html).
This especially caught my attention because of project I'm working on, and we are doing exactly what is discussed in the following article regarding generation of data and providing a sophisticated archive with lots of functionality as a part of the project. Questions arise as to curation, access, and desired functions. It is expected that this is becoming commonplace as a way to work will continue to evolve in the scientific community, as will the need for standards: http://www.nature.com/nature/journal/v455/n7209/full/455028a.html

It is an excellent opportunity for scientific institutions to take the lead.

Excerpts:
"Community standards for data description and exchange are crucial. These facilitate data reuse by making it easier to import, export, compare, combine and understand data. Standards also eliminate the need for each data creator to develop unique descriptive practices. They open the door to development of disciplinary repositories for specific classes of data and specialized software management tools."
"The time is right for scientists to take stock of the institutionalized data services that are available or under development, to understand how these institutions are governed and financed, and to make choices about the best strategies for their disciplines. Can a discipline-oriented solution work? If a university-based system seems more practical, what can be done to expedite the move to university consortia strategies? As the volume of data, and the need to manage it grows, disciplinary consensus leadership will be very powerful factors in addressing the challenges ahead."

Structure, Tools, and Platforms of Science

Source

http://www.nature.com/news/specials/bigdata/index.html
http://www.nature.com/nature/journal/v455/n7209/full/455028a.html

The Information Superhighway Becomes International

Jerry Sheehan — Wed, 03 Sep 2008 09:27:09 -0700

Description

Prior to the last decade most data networks were routed through the United States. It was not uncommon for traffic from one European country to find itself being hauled to the US prior to being delivered to an EU neighbor.
However, we are clearly witnessing a major transition in this position as the exceptional pace of the development of international high speed network creation quickens.

This growth has been fueled by a number of factors. First, nations have begun to look at high speed networking as a necessary infrastructure for economic growth. A number of futurists have argued that today's networks are as importance economically as the sea routes and roads of yesterday.[1] Second, a number of countries in a post-911 world began to become concerned that American intelligence agencies might be "easedropping" on their data communications for either political or economic gain.[2] This de facto collaboration between American telecommunications companies and federal agencies is now an accepted fact.[3]

The globalization of the world's information infrastructure was a natural consequence of the increased adoption of the Internet throughout the world. One of the interesting challenges for the American marketplace is the development of methods to capitalize on the increasing percentage of international traffic coming to US web sites. The Wall Street Journal has reported that most major US sites not draw more then half their audience from international visitors but only generate 5% of their revenue from this traffic.[4]

This data infrastructure globalization will also likely lead to an increase in "dark" budgets used by intelligence agencies for electronic surveillance. J. McConnel, Director of National Intelligence for the United States, has revealed in the course of investigation of the Bush Administration's wireless wiretapping program that the US domestically wiretaps thousand of international calls.[5] As these communications move overseas the US will need to expend more effort and technological prowess to keep up.

Computer & Information Science

Source

1] "Can Optic Cables Predict Economic Shifts?", Om Malik, BusinessWeek, August 18, 2008
[2] "Internet Traffic Begins to Bypass the U.S.", John Markoff, New York Times, August 29, 2008, http://www.nytimes.com/2008/08/30/business/30pipes.html
[3] "Verizon and Government Seek Dismissal of Data-Mining Programs on Secrecy and Free Speech Grounds", Ryan Singel, Wired, August 30, 2007, http://blog.wired.com/27bstroke6/2007/08/verizon-and-gov.html#previouspost
[4]"U.S. Web Sites Draw Traffic From Abroad But Few Ads", Emily Steel and Amol Sharma, Wall Street Journal, July 10, 2008, http://online.wsj.com/article/SB121563492172840249.html?mod=googlenews_wsj
[5]"Spy Chief Sheds Light on Wiretaps", Greg Miller, Los Angeles Times, August 23, 2007, http://articles.latimes.com/2007/aug/23/nation/na-intel23

Robot scientists to explore treacherous ice environments

Alex Soojung-Kim Pang — Wed, 28 May 2008 05:33:49 -0700

Description

EurekAlert reports:

Scientists are diligently working to understand how and why the world’s ice shelves are melting. While most of the data they need (temperatures, wind speed, humidity, radiation) can be obtained by satellite, it isn’t as accurate as good old-fashioned, on-site measurement and static ground-based weather stations don’t allow scientists to collect info from as many locations as they’d like.

And unfortunately, the locations in question are volatile ice sheets, possibly cracking, shifting and filling with water — not exactly a safe environment for scientists.

To help scientists collect the more detailed data they need without risking scientists’ safety, researchers at the Georgia Institute of Technology, working with Pennsylvania State University, have created specially designed robots called SnoMotes to traverse these potentially dangerous ice environments. The SnoMotes work as a team, autonomously collaborating among themselves to cover all the necessary ground to gather assigned scientific measurements. Data gathered by the Snomotes could give scientists a better understanding of the important dynamics that influence the stability of ice sheets.

:In order to say with certainty how climate change affects the world’s ice, scientists need accurate data points to validate their climate models," said Ayanna Howard, lead on the project and an associate professor in the School of Electrical and Computer Engineering at Georgia Tech. "Our goal was to create rovers that could gather more accurate data to help scientists create better climate models. It’s definitely science-driven robotics."

Howard was previously a scientist at NASA, and so you can argue that there's a family resemblance-- and a design one-- between planetary explorers and SnoMotes.

The SnoMote represents two key innovations in rovers: a new method of location and work allocation communication between robots and maneuvering in ice conditions.

Once placed on site, the robots place themselves at strategic locations to make sure all the assigned ground is covered. Howard and her team are testing two different methods that allow the robots to decide amongst themselves which positions they will take to get all the necessary measurements.

The first is an “auction” system that lets the robots “bid” on a desired location, based on their proximity to the location (as they move) and how well their instruments are working or whether they have the necessary instrument (one may have a damaged wind sensor or another may have low battery power).

The second method is more mathematical, fixing the robots to certain positions in a net of sorts that is then stretched to fit the targeted location. Magnus Egerstedt is working with Howard on this work allocation method.

In addition to location assignments, another key innovation of the SnoMote is its ability to find its way in snow conditions. While most rovers can use rocks or other landmarks to guide their movement, snow conditions present an added challenge by restricting topography and color (everything is white) from its guidance systems.

For snow conditions, one of Howard's students discovered that the lines formed by snow banks could serve as markers to help the

SnoMote track distance traveled, speed and direction. The SnoMote could also navigate via GPS if snow bank visuals aren’t available.

Computer & Information Science

Source

http://www.eurekalert.org/pub_releases/2008-05/giot-rgw052708.php

"The cloud" - on-demand distributed computing power

Andrew Walkingshaw — Sat, 10 May 2008 16:25:00 -0700

Description

Simulation scientists are mostly limited by both the number, and the speed, of the computers available to them. Really large simulations need really serious computer resources, but simulations like that are pretty rare; so the resources for them have been concentrated in regional grids or national centres like the UK's HPCx.

Economically this makes a lot of sense, but there's a lot of overhead; for instance, compute time has to be bid for long in advance of when it might actually be used. In that light, commodity on-demand computing services like Amazon's EC2 begin to look promising as an alternative; they have even greater economies of scale than the national infrastructure services, can provide a scientist with more CPU power at essentially no notice, and often provide more flexibility in choice of operating system and software than a centrally-provided system can. At the moment, they don't scale to the massively parallel calculations that the national supercomputers specialize in, but sooner or later they'll be competitive even for those cases.

Computer & Information Science

Source

http://www.grid-support.ac.uk/content/view/239/157/
http://www.hpcx.ac.uk/
http://aws.amazon.com/ec2

JANUS: an FPGA-based System for High Performance Scientific Computing

Matt Daniels — Wed, 09 Apr 2008 08:12:01 -0700

NOTE: This content was aggregated from RSS feed. Original source is http://arxiv.org/abs/0710.3535

This paper describes JANUS, a modular massively parallel and reconfigurable
FPGA-based computing system. Each JANUS module has a computational core and a
host. The computational core is a 4x4 array of FPGA-based processing elements
with nearest-neighbor data links. Processors are also directly connected to an
I/O node attached to the JANUS host, a conventional PC. JANUS is tailored for,
but not limited to, the requirements of a class of hard scientific applications
characterized by regular code structure, unconventional data manipulation
instructions and not too large data-base size. We discuss the architecture of
this configurable machine, and focus on its use on Monte Carlo simulations of
statistical mechanics. On this class of application JANUS achieves impressive
performances: in some cases one JANUS processing element outperfoms high-end
PCs by a factor ~ 1000. We also discuss the role of JANUS on other classes of
scientific applications.

Managing Large Scientific Communities

Sabine Hossenfelder — Sat, 22 Mar 2008 07:49:23 -0700

Description

The community of scientists is increasing, becomes tighter connected, and more global. Under these circumstances, people working together face additional challenges that occur in large groups. They struggle to balance specialization versus diversity, collaboration versus competition, and have to avoid fallacies that large groups of people can run into when left without management. This problem is especially obvious in the academic world where management above the very local level is only slowly emerging in multi-institutional projects (that however come with an enormous bureaucratic effort).

These are points that have long been acknowledged and addressed by company leaders, but embarrassingly the academic world has not yet come to realize the scientific community needs a proper management to function efficiently, a way to provide incentives for researchers and their studies such that it does not hinder progress on the long run. As a consequence, presently lots of time and energy is wasted with self-supporting but progress-hindering behavior like hiring and selection processes that have become inappropriate (e.g. simply the increasing amount of applications is more demanding and tempts superficiality or reliance on personal connections), the time-scales on which funding is provided is in many areas very far off reality (in that writing reports about results all three months, or planning ahead for five years is a requirement that does not fit to the way science actually works) and the occurrence of pseudo-interesting areas that flourish because members of sub-groups provide themselves with positive feedback (as a highly discussed case study, see e.g. Lee Smolin, "The Trouble with Physics" Houghton Mifflin (2006)).

The sociology of science consists so far mostly of analytic studies, but practical advises and their implementation are badly needed. I therefore expect that the interest in this area will further increase in the next decade, and that new strategies to distribute funds and support researchers will have to be established.

Source

Global Diffusion of Science

Alex Soojung-Kim Pang — Tue, 23 Oct 2007 11:10:30 -0700

Description

Over the next 50 years, the long US dominance of a wide range of fields in science and technology is likely to end as the global scientific playing field becomes flatter and more diverse.

For the last 50 years, global science has been dominated by a handful of countries: the US, the UK, Germany, France, Japan, and a few others. Of these, the US has been the top player: it has had the largest, best-funded, and most vibrant scientific community; it has encouraged and rewarded technological innovation; and it has attracted scientists and engineers from around the world. Over the next 50 years, this scientific US hegemony is expected to end. The coming world of global science is likely to be more diffuse, with high mobility among scientists, the emergence of multiple centers of excellence, and a more even (but far from uniform) distribution of scientific and technical talent. It might also be characterised by the movement of a number of small states into world leadership positions in specialised fields, or into dominant positions at the margins between disciplines or between the laboratory and the marketplace. The EU and developed Asian countries (Korea, Japan, India, and possibly China) have the potential to become more attractive and interesting places to conduct basic research.

Great powers: States like Britain and Germany are likely to continue to be leaders in a variety of sciences, but the range of challengers will grow, particularly in interdisciplinary areas and emerging fields of research that do not require massive capital investments. The degree to which these countries maintain their preeminence is likley to depend partly on internal factors, such as domestic investment in basic research and ability to attract (but not necessarily permanently retain) talent from around the world. Just as financial institutions today succeed not by hoarding capital but by circulating it, tomorrow's scientific state could succeed by circulating and enhancing human capital. States with significant antiscience movements (like the Intelligent Design movement in the United States) could face decline.

Small developed nations: Developed nations that have had more modest scientific capabilities could have an opportunity to create world-class expertise in targeted areas. None are likely to challenge the great powers across a variety of fields but instead could focus on building excellence in interdisciplinary fields, in applied sciences, or in areas that draw on a mix of scientific expertise and local culture. Singapore's efforts to become a world power in biotechnology (based largely on the importation of world-class talent from Europe and the US), South Korea's work in stem cell research, and Denmark's emergence as a power in high-tech product design (which builds on a long, world-class reputation in architecture and furniture) are examples of local excellence with global appeal.

Emerging states: Developing countries that are able to participate in international collaboration in ways that allow them to build their domestic communities (as Brazil, for example, is likely to do) have the potential to follow a path similar to the small nations. The great unknown among the emerging states is China. Despite its massive output of scientists and engineers and its recent success at luring home PhDs trained in the West, China is not likely to emerge as a scientific world power in the absence of political liberalisation. Historically, authoritarian states develop world-class expertise in more theoretical rather than practical subjects and do poorly at translating scientific strength into technical and economic innovation."

This will be enabled by:

Greater mobility among scientists and engineers as bureaucratic rules hindering worker mobility are lowering, and rules governing access to national educational systems are relaxing.
Increased international collaboration among EU countries and scientists in developing countries, particularly in large-scale projects that require the combined resources of a number of countries (such as space exploration or high-energy physics) and interdisciplinary projects that require assembling expertise from a variety of places.
Replacement of 'brain drain' by 'brain circulation', whereby talented people spend extended periods of time working outside the countries of their birth but do not stay away forever.

Early indicators include:

"Growing importance of small states like South Korea, Singapore, and Finland in science and technology"

What to watch:

"Spending on basic scientific research in emerging nations approaches US and EU levels of spending.
Small developed and emerging nations increase their scientific productivity and succeed in translating investments in basic research into technical innovations, as shown by scienometric indicators.
Graduate students in science flow into small developed and emerging nations, influencing the countries' long-term connectedness in global scientific communities and exchanges, and their ability to attract new generations of students and scientists.

Signals

Does Globalization of the Scientific/Engineering Workforce Threaten U.S. Economic Leadership?

Foreign science and engineering graduate students returning to U.S., but numbers still below 2001

Reverse Immigration from the US Back to Brazil

Small is Beautiful When It Comes To Being Wired