At TED 2009, Tim Berners-Lee called for “raw data now” — for governments, scientists and institutions to make their data openly available on the web. At TED 2010, he revisited this theme and illustrated a few interesting examples of real world linked data.

Some of the examples he references include:

UK Bike incidents – based on data from data.gov.uk

Paul Clarke, in the U.K. government blogged that, “Oh, I’ve just got some raw data. Here it is, it’s about bicycle accidents.” Two days it took the Times Online to make a map, a mashable map, we call these things mash-ups, a mash-up, user interface that allows you to go in there and have a look and find out whether your bicycle route to work was affected.

Zainsville water map – created by Cedar Grove Institute for Sustainable Communities

Look at Zanesville, Ohio. Here is a map a lawyer made, put on at the water plant, seeing which houses are there, which houses have been connected to the water? And he got, from other data sources, information to show which houses are occupied by white people. Well, there was too much of a correlation, he felt, between which houses were occupied by white people and which houses had water, and the judge was not impressed either. The judge was not impressed to the tune of 10.9 million dollars. That’s the power of taking one piece of data, another piece of data, putting it together, and showing the result.

Where Does My Money Go – Open Knowledge Foundation

Let’s look at some data from the U.K. now. This is U.K. government data, completely independent site, Where Does My Money Go, allows anybody to go there and burrow down. You can burrow down by a particular type of spending or you can go through all the different regions and compare them. So, that’s happening in the U.K. with U.K. government data.

California stimulus spending - California Economic Recovery Portal

Here’s a site which allows you to look at recovery spending in California. And take an arbitrary example, Long Beach, California, you can go and have a look at what recovery money they’ve been spending on different things such as energy. (more at recovery.gov)

Postcode Newspaper – Newspaper club

If you have lots of data about places you can take, from a postcode, which is like a zip plus four, for a specific group of houses, you can make paper, print off a paper which has got very very specific things about the bus stops, the things specifically near you.

Afganistan Election data – National Democratic Institute

This is a mash-up of the data which was released about the Afghan elections. It allows you to set your own criteria for what sort of things you want to look at. The red circles are polling stations, selected by your criteria. And then you can select also other things on the map to see what other factors like the threat level. So, that was government data.

OpenStreetMap Haiti edits – Port-au-Prince before and after

Here focusing in on Haiti. The map of Port au-Prince at the end of 2009 was not all it could be, not as good as the map of California. Fortunately, just after the earthquake, GeoEye, a commercial company, released satellite imagery with a license which allowed the open-source community to use it.

This is January time lapse, people editing, that’s the earthquake. After the earthquake immediately people all over the world, mappers who wanted to help, and could, looked at that imagery, built the map, quickly building up.

Haiti OpenStreetMap

We’re focusing now, Port au-Prince. The blue is refugee camps these volunteers had spotted from the air. So, now we have, immediately, a real-time map showing where there are refugee camps, rapidly became the best map to use if you’re doing relief work in Port au-Prince.

Witness the fact that it’s here on this Garmin device being used by rescue team. And Haiti, there is the map showing on the left-hand side there, that hospital, actually that’s a hospital ship.

In finishing Tim Berners-Lee thanks all those involved the the projects above and similar initiatives worldwide, and notes how the open data movement has just got started:

So, if you’ve been involved in that at all, I just wanted to say whatever you’ve been doing, whether you’ve just been charting raw data now, or you’ve been putting government or scientific data online, I just wanted to take this opportunity to say thank you very much, and we have only just started.

Related:

More on data.gov.uk launch

• Tim Berners-Lee on the launch of data.gov.uk
• Making Data Public: Presentation by Sir Tim Berners-Lee and Prof. Nigel Shadbolt -
• Data.gov.uk: the Power of Apps

Government data around the world

• Gov data around the world
• World Government Data

TargetPoint‘s VP and Director of Research, Alex Lundry, was recently a featured speaker at DC Ignite, where he gave an enlightening talk on the Political Power of Data Visualization.

The talk is only 5 minutes long and consists of insights into how data visualisations can be used for political ends. It also includes this reconfiguration of the famous H.G. Wells quote:

“Statistical Visual thinking will one day be as necessary for efficient citizenship as the ability to read and write.”

He begins with a discussion of House Republican John Boehner’s infographic describing the Democratic Health Plan proposal. Lundry calls it “messy, complicated and confusing” while others have described it as “a nightmare of bad color choices, poor layout, and overall obfuscated design”.

He notes how this then set off a series of Chart Wars. The Democrats quickly responded to Boehner’s chart with one of their own:

Meanwhile, the New Republic made a chart of the current health-care system:

Robert Palmer then redesigned Boehner’s chart and posted it to flickr under the title “Do not fuck with graphic designers“. Lundry describes this as “elegant, easy on the eye and brings order to the previously chaotic chart”.

Lundry does not proclaim any of the charts as “right”, rather he explains how they illustrate the Political Power of Data visualization. He notes how vision is our most dominant sense and takes up 50% of our brains’ resources. “Pictures are a superior and more efficient delivery mechanism for information”; The Pictorial Superiority Effect.

He goes on to discuss the release of data in open, machine readable formats; where anyone with a computer can create easy data visualizations using tools such as Swivel, Wordle, ManyEyes or Verifiable. As such, he explains how we can expect to see more visualizations used for messaging purposes.

The Sunlight Foundation has been a prominent proponent of data visualizations to demonstrate patterns and correlations based on government data. They believe, however, that government should concentrate on providing open, machine readable data, rather than visualizations. This is because visualizations – like we’ve seen above – can actually ‘hurt transparency’:

Visualizations, like any other form of news product, can be editorial– even inadvertently. If government puts more of a priority on producing great visualizations and user experience than on providing quality accurate data with a great feedback loop, then it runs a pretty good chance of not adhering to the goal of being actually transparent.

If Chart Wars have taught us anything it’s that visualizations can tell powerful stories; they’re quickly becoming the medium with the most powerful message.

Further reading

For more information, Lundry recommends books by:

• Edward Tufte (‘Godfather of the Data viz movement’) – The Visual Display of Quantitative Information
• Stephen Few – Show me the Numbers
• Connie Malamed – Visual Language for Designers
• John Medina – Brain Rules

Online check out:

• FlowingData
• Datavisualization.ch

Peter Corbett, founder of iStrategyLabs, recently released a step-by-step guide on how to create a civic application development competition, to harness the potential of citizens to develop apps based on Open Government data.

The methods outlined in the guide can apply to local, state, and federal government, as well as non-profit and for-profit entities.

Create an Apps for Democracy – Open Government Data Meets Citizen Innovation

The guide is based on the successful Apps for Democracy competitions held over the past few years in Washington DC. The document defines 9 core components, and outlines a checklist of tasks to complete for each step of the framework.

The guide provides details of how to structure each component of the framework. This includes expected time-lines, stakeholders to involve, technology requirements and how to market the competition. As such, it provides a useful resource for others in helping to develop similar competitions.

The Apps for Democracy model has already been successfully utilised by other cities and countries for their own Open innovation contests. Competitions such as NYC Big Apps, Mashup Australia, Apps for America and INCA have all incorporated aspects of the Apps for Democracy initiative, as a means of engaging citizen developers to create apps based on Open government data.

Indeed, the recent Online Engagement review for Australia’s Government 2.0 Taskforce recommended:

Employ Apps for Democracy “Community Edition” model: solicit application ideas before the contest, judge submissions based on responsiveness to these ideas, and provide a development path through which the best entries can be integrated with government operations.

City Camp

Peter presented the guide at this weekend’s CityCamp, where he explained the background to the Apps for Democracy contest and the cost savings and innovations it produced.

Innovation prize best practice

A competition/prize is a familiar and easily understood concept that has a long history of inspiring beneficial change. Prizes, as offered in the Apps for Democracy contest, are primarily centered on recognizing excellence in the area of innovation. The prizes are used to attract innovators and recognize that – as Vivek Kundra (Federal CIO) says “Government does not have a monopoly on the best ideas”.

McKinsey note how the “most successful prize competitions place an equal emphasis on other elements, such as the broader change strategy, the competition itself, and post-award activities designed to enhance the impact of the prize.”

The guide expands on these points including the importance of collaboration/feedback before the competition closes:

iStrategyLabs has now screened 60+ applications submitted to our own contests, and another 45 during our involvement as a judge of the Apps for America contest. Upon review, we typically provide entrants with feedback so they can further iterate on their submission if the contest deadline has yet to elapse. We’ve found that this ensures a higher quality of applications, which better meet the needs of the city and citizen. Constructive feedback also gives technology developers a greater chance of winning an award.

This adheres to McKinsey’s research on best practice for Innovation prizes:

Much as prize sponsors can exploit the power of competition to drive innovation, they should also recognize the benefits of collaboration. A great deal of research suggests that collaboration can promote innovation substantially and some prizes actively encourage it. Changemakers, for example, encourages publication of submissions during the competition, generating conversations that often inspire participants to improve their entries before the competion closes.

The Apps for Democracy Community edition included the provision of a grant as part of the prize:

OCTO will have the option to award a Community Grant administered by iStrategyLabs for a total of $14,000 over a 9 month period for further development and support of the winning application.

Again, this adheres to McKinsey’s best practice on post-prize activities:

Finally, much of the impact of a prize occurs after it is awarded. Prize sponsors who devote significant effort to post-prize activities consistently impressed us. Sponsors, for instance, can make their prizes part of a broader change strategy that also includes grants, contracts, or infrastructure investments to help institutionalize benefits or scale up innovations.

By sharing his experiences Peter Corbett has provided a valuable insight into how Innovation competitions can be constructed. The guide’s recommendations adhere to best practice in the area of Innovation prizes, and thus it should be recognized as an important tool for developing Open innovation competitions.

Nearly 200 people attended the Africa Gathering event in London today. The event included presentations and seminars aimed at highlighting and progressing issues related to IT, social networking and technology in Africa.

There isn’t often many success stories relating to technology and Africa, but the story of Ushahidi (meaning “testimony in Swahili”) is an example of one such initiative that has inspired others in developing countries throughout the world. It utilizes the power of the mobile phone and the internet to help spread eyewitness reports and news during crisis events.

Erik Hershman talks about the origin of the initiative in the video above.  The idea came about during the 2008 Kenyan elections as a means of crowdsourcing problems relating to the vote. He explains how a small team  prototyped and built, in three days, a system that allowed anyone with a mobile phone to report what was happening during the election. Reports of violence (such as that below) and intimidation were submitted to the site from all areas of the country, using various different communication mediums.

These user generated accounts were then mashed up using Google maps to display an interactive visualization and timeline of election incidents. Realising the power of the platform the team decided to make the project code opensource so that it could be used during other crises and elections throughout the world. The application has since been deployed in the Congo (DRC), by NGOs in east Africa and in Gaza.

Vote Report India

Vote Report India is based on the Ushahidi software and serves as an election monitoring platform for the current India elections. Users can contribute eyewitness accounts of election incidents through SMS, email, twitter or via the website. The platform then aggregates user content with news reports, blogs, tweets and an interactive map. The site relies on India’s civil society to monitor and report the election, rather than on international observers.

Ellen Miller notes how one of the inspirations for Vote Report India was Twitter Vote Report. It was used to document and report on problems during last years US presidential elections. This allowed users to submit election reports using Twitter, SMS, Telephone and smartphones, and was instrumental in creating a user driven narrative of election incidents.

Crowdsourced filters

Erik notes how the Ushahidi team are currently working on a crowdsourced filter mechanism to improve the veracity and accuracy of information reported through the platform. Incidents such as the Mumbai bombings demonstrated the difficulties in relying on first hand reports relayed through twitter and SMS. Determining the accuracy of these ‘first drafts of history’ is a difficult task, but the power of the crowd – especially on platforms such as twitter – usually weeds out any inaccurate reports fairly quickly. Nevertheless, during the first few hours of a crisis there is often so much information that it’s difficult for distanced observers to decide the veracity of different accounts. Because of this, a crowdsourced filter that’s based on factors including the trust level apportioned to the user submitting the information, has the potential to probabilistically determine those reports most likely to be accurate.

The interesting point is that this innovation is coming from Africa. The Ushahidi project was a runner up in the recent USAID Development 2.0 Challenge, which sought to highlight the power of mobile technologies to address global development problems. Many of the entries to the competition came from Africa, with the winning project an open source mobile phone based platform to transmit data from clinics to government and UNICEF databases.

Today’s Africa Gathering is judging its success as whether it leaves people ‘inspired, enthused and better connected.’ The opensource nature of Ushahidi, and the other USAID Development Challenge projects, means that many other countries can benefit from these innovative solutions created in developing countries. It also serves to exemplify how Africa can contribute back public service solutions, to help with crises throughout the world. Certainly the nature of these projects, and all those outlined at Africa Gathering, has left me inspired and enthused by how the continent is helping itself, and others, through opensource development.

The Sunlight Foundation announced a new competition today called Apps for America. This mashup contest follows in the footsteps of recent competitions from the District of Columbia and the UK Government’s Power of Information Taskforce.

The contest is run a little differently in that the mashup must be based on one of the APIs below, and be released under an open source license.

• Sunlight Labs API
• OpenSecrets.org API
• FollowTheMoney.org API
• Capitol Words API
• Sunlight Labs API Lawmaker Dataset
• Sunlight Labs API Lobbyist Dataset
• Sunlight Partytime Database Dump

The first prize is $15,000 which is slightly less that the Showusabetterway or Apps for Democracy contests. Nevertheless, if the results are anything like those achieved by the District of Columbia – 47 Applications being built in 30 days, and an estimated 4,000% return on investment – it’ll be well worth the money.

Open Information

Jimmy Wales recently outlined one of the ‘core components of a structurally sound, technologically savvy federal government’ as openness of information. These kind of contests, along with the fantastic work of the Sunlight Foundation provide an opportunity to – as Craig Newmark recently commented – ‘Free the nerds‘. Giving others the opportunity to mashup and remix data using publicly available APIs can create – in Ellen Miller’s words – ‘magical‘ things.

Tom Steinberg’s recent post on the Top 5 Internet Priorities for the Next Government identified the Freeing of data as a priority:

Free your data, especially maps and other geographic information, plus the non-personal data that drives the police, health and social services, for starters. Introduce a ‘presumption of innovation’ – if someone has asked for something costly to free up, give them what they want: it’s probably a sign that they understand the value of your data when you don’t.

Liz Azyan wrote a good post recently about Mashups in Government. This covers work from mySociety and other parties, in the area of exploiting Government data to great a more transparent and innovative  Government/Citizen relationship.

Such competitions and the resulting applications can only serve to advance the prioritization of free and open access to Government data across a wide spectrum of public life.