data mining

Data Mining in the War on Terror: Useful and Worth the Risk?

Jerry Sheehan — Thu, 16 Oct 2008 18:52:09 -0700

Description

In 2002, the Defense Advanced Research Project Agency established the Information Awareness Office (IAO) to unify projects using data mining technologies to counter threats to national security, in particular terrorism. One of the largest of these efforts initially named the Total Information Awareness (TIA) Program, later changed to the Terrorism Information Awareness Program, was focused on integrating various public and private database. Public awareness and concern over the program was raised by William Safire who wrote that TIA had a $200 million dollar budget to create computer dossiers on 300 million americans. After public outcry over the program Congress moved in 2004 to prohibit further funding of TIA.[1]

Deeper analysis of TIA shows that it was never completely defunded and perhaps most troubling various technology components of the effort continue to be developed under the less publicly viewable budget of the National Security Agency (NSA). [2] Regardless of if TIA technologies continue to be developed or not it is clear that the United States Government is engaging in data mining on a large scale effort.

Perhaps most interesting in this regards is a 2004 study conducted by the General Accounting Office that surveyed 128 federal agencies and departments and found that 52 of them were planning to launch nearly 200 new data mining projects. Of the reported projects 122 involved personal information with at least 54 projects focused on integrating data from the private sector. Clear application drivers for these efforts included analyzing intelligence to detect terrorist activities and identifying criminal activities.[3]

Data mining programs that automatically search across personal and private sector data have been sold by proponents as being a necessary evil in the war against terror, but are they effective? A new study conducted by the National Research Council on behalf of the Department of Homeland Security concludes unequivocally no and argues that not only will the technology not work but the inevitable mistakes made by such an effort are inherently un-American and an unnecessary risk in the war on terror.[4]

Most interestingly, the report also recommends that "All U.S. agencies with counterterrorism programs that collect or "mine" personal data -- such as phone records or Web sites visited -- should be required to evaluate the programs' effectiveness, lawfulness, and impacts on privacy."[5]

Computer & Information Science

Source

[1]"Information Awareness Office", Wikipedia, http://en.wikipedia.org/wiki/Information_Awareness_Office
[2]"TIA Lives On", Shane Harris, National Journal, Feb 23, 2006, http://www.nationaljournal.com/about/njweekly/stories/2006/0223nj1.htm
[3]"Data Mining: Federal Efforts Cover a Wide Range of Uses", United States General Accounting Office, May 2004, pg10-12
[4]"Data-Mining for Terrorists Not 'Feasible,' DHS-Funded Study Finds", Ryan Singel, October 7, 2008, WIRED, http://blog.wired.com/27bstroke6/2008/10/data-mining-for.html
[5]Protecting Individual Privacy in the Struggle Against Terrorists:
A Framework for Program Assessment,National Research Council, October 7, 2008.

Database tools for science -opportunity for collaboration on standards

Patricia Larenas — Mon, 08 Sep 2008 13:49:51 -0700

Description

In the recent Nature issue (Nature 455, 28-29 (4 September 2008) | doi:10.1038/455028a; Published online 3 September 2008) a very interesting special section on "big data" is presented: (http://www.nature.com/news/specials/bigdata/index.html).
This especially caught my attention because of project I'm working on, and we are doing exactly what is discussed in the following article regarding generation of data and providing a sophisticated archive with lots of functionality as a part of the project. Questions arise as to curation, access, and desired functions. It is expected that this is becoming commonplace as a way to work will continue to evolve in the scientific community, as will the need for standards: http://www.nature.com/nature/journal/v455/n7209/full/455028a.html

It is an excellent opportunity for scientific institutions to take the lead.

Excerpts:
"Community standards for data description and exchange are crucial. These facilitate data reuse by making it easier to import, export, compare, combine and understand data. Standards also eliminate the need for each data creator to develop unique descriptive practices. They open the door to development of disciplinary repositories for specific classes of data and specialized software management tools."
"The time is right for scientists to take stock of the institutionalized data services that are available or under development, to understand how these institutions are governed and financed, and to make choices about the best strategies for their disciplines. Can a discipline-oriented solution work? If a university-based system seems more practical, what can be done to expedite the move to university consortia strategies? As the volume of data, and the need to manage it grows, disciplinary consensus leadership will be very powerful factors in addressing the challenges ahead."

Structure, Tools, and Platforms of Science

Source

http://www.nature.com/news/specials/bigdata/index.html
http://www.nature.com/nature/journal/v455/n7209/full/455028a.html

Google Contributes Funding to NASA Space Science Technology + Mission

Matt Daniels — Tue, 10 Jun 2008 00:21:21 -0700

Description

Google has contributed funding to the Transiting Exoplanet Survey Satellite (TESS), a NASA spacecraft (smallsat) in cooperation with MIT and the Harvard Smithsonian Center that could potentially be launched in 2012.

Google, the Internet search powerhouse that in recent years has expanded to include mapping of the stars as well as the surfaces of the moon and Mars and which has an ongoing collaboration with NASA's Ames Research Center, provided a small seed grant to fund development of the wide-field digital cameras needed for the satellite. Because of the huge amount of data that will be generated by the satellite, Google has an interest in working on the development of ways of sifting through that data to find useful information.

It's interesting that private companies seem to be increasingly funding large science projects.... (N.B., I've noticed this mostly in the physical sciences...)

Engineering & Design

Source

http://web.mit.edu/newsoffice/2008/google-planets-tt0319.html

Web companies race to social and open information streams

Mike Love — Tue, 30 Oct 2007 14:05:30 -0700

Description

Many large web companies are racing towards a similar goal: owning people's online attention streams by providing the most benefit to those that use their services. For an example of this direction of new products, a San Francisco company, Radar Networks, recently unveiled a new service at the Web 2.0 Summit, Twine, which will assimilate and make sense of the information in users' lives if they feed it with bookmarks and the content of emails.

At the same time, we can see a push towards letting users choose if they want to open their digital identity and network. A new initiative from Google will add a social component to more of their services. Similar to Facebook, it looks like Google will open up a new set of APIs to let developers access the services.

Some details about Maka-Maka have already leaked out, particularly how Google plans to use the feed engine that powers Google Reader (known internally as Reactor) to create “activity streams” for other applications akin to Facebook’s news and mini feeds.

Google is planning to “out open” Facebook with a new set of APIs that developers can use to build apps for its social network Orkut, iGoogle, and eventually other applications as well.

A New York Times article describes Google, Myspace, LinkedIn and some of the most popular social networking sites convening on a commons standard for social network developers, OpenSocial:

MySpace and Bebo, two of the world’s largest social networking sites, on Thursday joined a Google-led alliance that is promoting a common set of standards for software developers to write programs for social networks....

The open standards could create a boom of innovation around social networks as applications reach more users than ever and encourage developers to create more Internet tools.

Data mining on a large scale has been common practice for companies and the government for years now, but these new ambitious data aggregation and data mining projects will potentially open their results to the public. At the least, each individual user will be benefitted with better recommendations by the addition of data by others.

Radar Networks: Twine http://www.twine.com/
Google's Response to Facebook http://www.techcrunch.com/2007/10/29/googles-response-to-facebook-maka-maka/
MySpace Joins Google Alliance
http://www.nytimes.com/2007/11/02/technology/02google.html?_r=2&oref=slogin&oref=slogin
Plaxo: More on social network portability http://blog.plaxo.com/archives/2007/08/more_on_social.html
Twine, Freebase and Powerset http://mikelove.wordpress.com/2007/10/21/twine-freebase-and-powerset/

data mining

Data Mining in the War on Terror: Useful and Worth the Risk?

Description

Source

Database tools for science -opportunity for collaboration on standards

Description

Source

Google Contributes Funding to NASA Space Science Technology + Mission

Description

Source

Web companies race to social and open information streams

Description

Signals