TargetPoint‘s VP and Director of Research, Alex Lundry, was recently a featured speaker at DC Ignite, where he gave an enlightening talk on the Political Power of Data Visualization.

The talk is only 5 minutes long and consists of insights into how data visualisations can be used for political ends. It also includes this reconfiguration of the famous H.G. Wells quote:

“Statistical Visual thinking will one day be as necessary for efficient citizenship as the ability to read and write.”

He begins with a discussion of House Republican John Boehner’s infographic describing the Democratic Health Plan proposal. Lundry calls it “messy, complicated and confusing” while others have described it as “a nightmare of bad color choices, poor layout, and overall obfuscated design”.

He notes how this then set off a series of Chart Wars. The Democrats quickly responded to Boehner’s chart with one of their own:

Meanwhile, the New Republic made a chart of the current health-care system:

Robert Palmer then redesigned Boehner’s chart and posted it to flickr under the title “Do not fuck with graphic designers“. Lundry describes this as “elegant, easy on the eye and brings order to the previously chaotic chart”.

Lundry does not proclaim any of the charts as “right”, rather he explains how they illustrate the Political Power of Data visualization. He notes how vision is our most dominant sense and takes up 50% of our brains’ resources. “Pictures are a superior and more efficient delivery mechanism for information”; The Pictorial Superiority Effect.

He goes on to discuss the release of data in open, machine readable formats; where anyone with a computer can create easy data visualizations using tools such as Swivel, Wordle, ManyEyes or Verifiable. As such, he explains how we can expect to see more visualizations used for messaging purposes.

The Sunlight Foundation has been a prominent proponent of data visualizations to demonstrate patterns and correlations based on government data. They believe, however, that government should concentrate on providing open, machine readable data, rather than visualizations. This is because visualizations – like we’ve seen above – can actually ‘hurt transparency’:

Visualizations, like any other form of news product, can be editorial– even inadvertently. If government puts more of a priority on producing great visualizations and user experience than on providing quality accurate data with a great feedback loop, then it runs a pretty good chance of not adhering to the goal of being actually transparent.

If Chart Wars have taught us anything it’s that visualizations can tell powerful stories; they’re quickly becoming the medium with the most powerful message.

Further reading

For more information, Lundry recommends books by:

• Edward Tufte (‘Godfather of the Data viz movement’) – The Visual Display of Quantitative Information
• Stephen Few – Show me the Numbers
• Connie Malamed – Visual Language for Designers
• John Medina – Brain Rules

Online check out:

• FlowingData
• Datavisualization.ch

Peter Corbett, founder of iStrategyLabs, recently released a step-by-step guide on how to create a civic application development competition, to harness the potential of citizens to develop apps based on Open Government data.

The methods outlined in the guide can apply to local, state, and federal government, as well as non-profit and for-profit entities.

Create an Apps for Democracy – Open Government Data Meets Citizen Innovation

The guide is based on the successful Apps for Democracy competitions held over the past few years in Washington DC. The document defines 9 core components, and outlines a checklist of tasks to complete for each step of the framework.

The guide provides details of how to structure each component of the framework. This includes expected time-lines, stakeholders to involve, technology requirements and how to market the competition. As such, it provides a useful resource for others in helping to develop similar competitions.

The Apps for Democracy model has already been successfully utilised by other cities and countries for their own Open innovation contests. Competitions such as NYC Big Apps, Mashup Australia, Apps for America and INCA have all incorporated aspects of the Apps for Democracy initiative, as a means of engaging citizen developers to create apps based on Open government data.

Indeed, the recent Online Engagement review for Australia’s Government 2.0 Taskforce recommended:

Employ Apps for Democracy “Community Edition” model: solicit application ideas before the contest, judge submissions based on responsiveness to these ideas, and provide a development path through which the best entries can be integrated with government operations.

City Camp

Peter presented the guide at this weekend’s CityCamp, where he explained the background to the Apps for Democracy contest and the cost savings and innovations it produced.

Innovation prize best practice

A competition/prize is a familiar and easily understood concept that has a long history of inspiring beneficial change. Prizes, as offered in the Apps for Democracy contest, are primarily centered on recognizing excellence in the area of innovation. The prizes are used to attract innovators and recognize that – as Vivek Kundra (Federal CIO) says “Government does not have a monopoly on the best ideas”.

McKinsey note how the “most successful prize competitions place an equal emphasis on other elements, such as the broader change strategy, the competition itself, and post-award activities designed to enhance the impact of the prize.”

The guide expands on these points including the importance of collaboration/feedback before the competition closes:

iStrategyLabs has now screened 60+ applications submitted to our own contests, and another 45 during our involvement as a judge of the Apps for America contest. Upon review, we typically provide entrants with feedback so they can further iterate on their submission if the contest deadline has yet to elapse. We’ve found that this ensures a higher quality of applications, which better meet the needs of the city and citizen. Constructive feedback also gives technology developers a greater chance of winning an award.

This adheres to McKinsey’s research on best practice for Innovation prizes:

Much as prize sponsors can exploit the power of competition to drive innovation, they should also recognize the benefits of collaboration. A great deal of research suggests that collaboration can promote innovation substantially and some prizes actively encourage it. Changemakers, for example, encourages publication of submissions during the competition, generating conversations that often inspire participants to improve their entries before the competion closes.

The Apps for Democracy Community edition included the provision of a grant as part of the prize:

OCTO will have the option to award a Community Grant administered by iStrategyLabs for a total of $14,000 over a 9 month period for further development and support of the winning application.

Again, this adheres to McKinsey’s best practice on post-prize activities:

Finally, much of the impact of a prize occurs after it is awarded. Prize sponsors who devote significant effort to post-prize activities consistently impressed us. Sponsors, for instance, can make their prizes part of a broader change strategy that also includes grants, contracts, or infrastructure investments to help institutionalize benefits or scale up innovations.

By sharing his experiences Peter Corbett has provided a valuable insight into how Innovation competitions can be constructed. The guide’s recommendations adhere to best practice in the area of Innovation prizes, and thus it should be recognized as an important tool for developing Open innovation competitions.

Kildarestreet.com was launched in May of this year as a means of allowing the public to keep a closer eye on their parliamentary representatives. It allows the public to track the activities of their elected TDs (think Congresspeople/MPs) on an easily accessible and searchable website. The site records all their speeches, remarks and votes in the Dáil and also provides a platform in which citizens can comment and discuss these debates and remarks.

One of the rationals for the site was to make the activities of the Dáil more Open, Transparent and Accessible. The accessibility issue was a major factor in the development of the site. The full text of the Dáil and Seanad, including written answers to parliamentary questions were already published on the Oireachtas website. Searching this site, however, and contextualizing and linking the comments to individual representatives was difficult. Kildarestreet.com solves these issues by linking together TD’s Dáil activity in a transparent and accessible format, thus providing greater accountability to the people for all their actions in parliament.

Site contents

KildareStreet.com contains information on the profile and performance of TDs not currently available on any Government sites. It has profile pages for each TD which lists their voting record, constituency details, topics of interest, Dáil activities (including Debates and written answers) and a numerical comparison of their parliamentary participation versus others e.g. how many debates they have spoken in compared to other TDs.

The site was developed by John Handelaar and is based on, and uses the same Open source software as, the UK website TheyWorkForYou.com. He notes some of the site features as:

• A dramatically-more-legible version of the Dáil Record going back to January 2004,
• Searches restricted to speeches or written questions, by speaker, or by date or date range,
• Sign up for email alerts for when a search query you’re interested changes, or whenever a TD of your choosing says something or asks a question which generates a written reply, and
• Subscribe to RSS feeds for individual TDs or for search queries.

Inconsistent data

The site relies on the ‘canonical’ version of the Official Parliamentary Record provided by the Oireachtas.  Debates and questions need to be extracted from raw XML files provided by a third party working for the Oireachtas.

Handelaar explains this process:

The trick, however, is to get all the actual debates and questions into the site’s database. You need to build a parser program to convert all that data from one format into another — in Ireland’s case, from the raw XML published by the Houses of the Oireachtas which is primarily intended as a print publishing format for the official record, but which also powers debates.oireachtas.ie.

Soon after the site was up and running, however, these data feeds started to cause issues. The problems encountered are all outlined on the KildareStreet.com blog:

• 8th May -

For reasons I cannot hope to explain, the Oireachtas has a production process for the Official Record which is bound by no formal timeframes, nor any published targets, nor apparently any guarantee that the material produced will be correct within a timeframe useful to anyone but historians.

Today, for the second time in a week, the official data from which we generate this web site has been released on the official website in a severely mangled manner. You can see at the time of writing that yesterday, the adjournment debate apparently contained over three hundred and fifty contributions, there were no written answers to questions, and the phrase “as of 01 May 2009″ has been inexplicably promoted to the rank of “major heading”.

• 13th May -

The Oireachtas official report printing company has once again mangled all the written answers from yesterday…They’ve now blown this three times in eight sitting days – not an impressive way for a recipient of public funds to deliver value, is it?

• 11th June -

Our apologies — the official report is once again in a state of utter nonsensical garbage because someone’s published a debate XML file which can’t possibly even validate under their own rules…Stepping into an overly-specific technical domain for a moment, and to ask a question which only they and we understand: why –

1. …do you use a DTD which permits you to enter Question elements under a ‘General Debate’ heading?
2. Conversely, if the DTD isn’t broken and in fact doesn’t permit that, why aren’t you USING IT to validate your output?
3. And since when was <table> a valid element to start a Debate section with?

We ask because these three points, if understood properly, would cover almost every single cockup you’ve made since we launched.

• 26th June -

You won’t get your email alert today because the Oireachtas debates production system is garbage operated by people who don’t care about the quality of their work.

[June 26] You know what? I’m done being polite about this.

Every single day this week, the official report has been dripping with errors and stupidity. Just now they updated the file from yesterday and instead of simply adding the remaining debates, the new file destroys yesterday’s written questions in the exact same way that we keep pointing out.

Tuesday’s debate record was unparseably-broken until yesterday morning. Wednesday of this week is currently complete garbage (update: this was finally fixed on July 1st, a week after it was originally broken), and now Thursday’s stuffed as well.

There are three sitting days each week. This week they screwed up with all of them.

The comments above highlight the frustrations involved with the release of poorly formatted government  data. These data feeds are provided by a private company, but as Handelaar notes “the only thing which makes the official site work properly is having us shout at them loudly and in public.”

One of the Open Government Data principles requires data to be Machine Processable: Data is reasonably structured to allow automated processing. Unfortunately, the frequent formatting errors contained in the Oireachtas files make this machine processing very difficult. It therefore makes the Dáil less accountable for its actions, and more open to cynicism by those disenfranchised with the political process.

Reconnecting the Dáil with the people

Last month, the Dáil opened its doors to thousands of people as part of a family day. 8,000 tickets were made available to members of the public and these were snapped up within 24 hours. The Ceann Comhairle (Chairman of the Dáil) was quoted as saying the level of interest in the family day made a clear statement that people do wish to connect with their parliament, their politicians and the political process.

In order to facilitate this connection, however, the Oireachtas needs to ensure more competent processes are followed in the release of its data. The Irish Government should embrace the tenets of the Transparency and Open Government directive as outlined by President Obama. The principle of Transparency is outlined as:

Government should be transparent.  Transparency promotes accountability and provides information for citizens about what their Government is doing.  Information maintained by the Federal Government is a national asset. My Administration will take appropriate action, consistent with law and policy, to disclose information rapidly in forms that the public can readily find and use.

The success of Kildarestreet.com – an average of 3,000 pages viewed each day and hundreds of people signed up for email alerts – highlights the Irish public’s appetite for information about the activities of their elected representatives. Satisfying this demand through the timely release of data, can help citizens reengage in the democratic process, and hold their TDs more accountable for their actions. It has already uncovered some interesting trends in answers to questions, and can serve as a valuable resource for those investigating their TD’s performance come election time.

The movement towards Openness and Transparency in relation to government data has been steadily advancing throughout the developed world. The Whitehouse Open Government initiative demonstrates how governments are realizing the benefits of a more open and engaged public. As a corollary, Tim Berners-Lee says “2009 is the year for putting government data online”.

Kildarestreet.com is just one of a number of worldwide sites including Opencongress, TheyWorkForYou.co.nz and OpenAustralia, that seek to make the activities of their Parliament more accessible and transparent. It’s up to Government to ensure they provide the platform upon which to enable these important civic endeavors. The Government’s role should be to at least provide the raw data upon which others can create contextualized applications and mashups. Without this data, the public’s engagement with the political process will be less informed, thus reducing the accountability of those elected to serve us.

The eagerly awaited data.gov site launched this week, ushering in a new area of openness and transparency throughout the federal government. Federal CIO, Vivek Kundra announced the site as a platform to democratize government data by stimulating the creation of innovative applications and visualizations.

Kundra’s aim is for the government to tap into the ingenuity and ideas of the public to exploit information held by government agencies and institutions. He wants others to create new applications, conduct research and come up with new ideas to transform the way government works.

Data.gov is going to democratize data that the taxpayer has already paid for…It is fundamental to the President’s commitment to a transparent and open government. It lifts the veil on how your government works… and is going to be fundamental to driving transparency, innovation and lowering the cost of government operations.

Datasets

The site launched with dozens of datsets consisting of raw data in machine readable formats e.g. XML, Text/CSV, KML/KMZ, Feeds and XML. Along with this the public is invited to suggest datasets to be added to the site. The current data catalogs include:

• Environmentally-relevant data (copper smelters, energy usages, brownfields, soil geochemistries, clean air statuses, weather trends, earthquakes, etc.)
• Demographic data (earnings, ages, etc.)
• National income and accounts (Gross Domestic Products, income levels, etc.)
• Regulatory alerts
• Patent applications and grant information

Vivek Kundra’s interview on Federal news radio explains how more and more information will be added to the site over the coming months and years. Indeed, he expects over 240,000 data sets to be added from a wide range of agencies and departments across the federal government.

The majority of the data sets on the site (see Sunlight labs chart) are from the US Geological Survey, but Kundra expects more data to be added in relation to healthcare and energy.

He mentions how the Sunlight Foundation has launched a competition, called Apps for America 2, to see what applications/visualizations citizens can develop from the information on Data.gov. This crowdsourcing of ideas and inventions from the public is the exactly the kind of activity Kundra is trying to promote through the site. Public participation, analysis and exploitation of the data is encouraged, and re-reinforces Kundra’s view that the government does not have a monopoly on how it’s/our data is best represented.

Open Government needs you

Data.gov is a major milestone in the Obama administration’s Open Government Initiative. Indeed, its launch coincided with the White House’s other initiative for a more open government. Whitehouse.gov now includes an Open Government section highlighting process on this directive. It includes a link for the public to submit ideas, a blog and an innovations gallery that features other federal transparency projects.

The Office of Science and Technology Policy has launched a three phase approach to develop recommendations on how to make the government more transparent. Public participation is integral to the open government roadmap, and the solicitation of ideas is a cornerstone for progress in this area.

• Phase 1 seeks ideas and suggestions from the public on how to make government more open. The public can submit and vote on ideas at http://opengov.ideascale.com. Hundreds of ideas have already been submitted in the areas of Transparency, Participation and Collaboration.
  
• Phase 2 is a discussion and in-depth analysis of the ideas and suggestions submitted during the brainstorm stage. The feasibility and value of ideas will be discussed, with a view on how best they contribute to the President’s objective of a more transparent and open government.
  
• Phase 3 is concerned with drafting proposals to implement the most relevant ideas from phase 2.

The plan outlined above is similar in scope to the National Dialog on how to improve Recovery.gov. Seeking the public’s input and ideas for improving government is a central tenent of Vivek Kundra and Aneesh Chopera’s mission towards fulfilling the goals of President Barack Obama’s memorandum on transparency and open government.

The confirmation of Aneesh Chopra this week to the role of federal CTO will help develop and embed this directive throughout the federal government. Data.gov and the Open Government Initiative are an exciting start to this transformation. Their success, and the mashups/ideas they generate, will be a measure of how the public wants to participate in helping to improve government. It’s now over to us – the public – to engage and demonstrate what can be done with open date.

The first applications built on this data has already arrived (see FBI Fugitive Concentration). It’s exciting to see how an ecology of innovation will develop based on this data. What’s certain is government data, and its visualization are about to get a lot more interesting.

Joshua-Michéle Ross has great blog post related to an interview with Mike Mathieu of Frontseat.org. Frond Seat is a civic software company and incubator, whose objective is to build software for civic life. The interview is enlightening in how Mike sees the tech community providing innovative services based on freely available raw data from Government institutions.

I’ve embedded the two videos below, and have outlined some of Mike’s comments. The points outlined below are paraphrased from the video conversations.

Software for Civic life – Part 1

• Historically software has been really expensive to develop, but now software is getting cheap

• The focus of innovation is more around data and communities of users

• Front Seat are experimenting with ways of taking advantage of cheap software, and data to make the world a better place

Front Seat has created Walkscore as a means to measure the walkability of neighborhoods. It plugs into the Google local search API to rate neighborhoods based on 13 different categories. It is available as a web service and it’s walkability ratings can be easily used by third-party real estate sites.

Countmore.org is a another site created by Front Seat which is targeted towards voter registration for college students in swing states. It’s based on a supreme court ruling in the 70s that says college students can either register to vote in their home or college state. The site was built in a couple of days and shows students if the electoral race is closer in their home or college state. It then gives them a way to register online to vote.

• When we think software for civic life – we think how can you be a better citizen, how do I make better choices.

• We have hundreds of ideas in the idea bank and it’s a trade off between how much investment would it take, the importance of the issue and the opportunity for change. Front Seat is like a social hedge fund – creating software to where there is an opportunity to make a disproportional positive impact for social change.

Software for civic life – Part 2

• With the Sunlight foundation there is the idea of more and more open data from Government, but don’t hold your breath on the Government releasing more data.

• Mysociety has a lot of projects which are similar in vein (to Front Seat). The objective is to create simple ways for citizens to connect with Government. The famous project is Fixmystreet.

• Tom Steinburg’s advice was don’t do any projects that rely on you being able to get the Government to open the data, because either budget issues, political issues etc. will keep that from happening.

• Walkscore, Rockthevote and the Sunlight Foundation are groups that are doing things that the Government is not.

• The Sunlight foundation is a great example, where one of the projects is to create a unique ID for all the senior decision makers in Government. The rational is that you can connect up different databases e.g. the Federal Elections Commission donor database, and you can then map that against voter records and how people voted in Congress. They’ve shown some very interesting correlations in how the votes of Congress people change when a certain industry quadruples their donations to that candidate in the week before a vote.

• It’s interesting to see some of the data types and services that Government typically held onto, or where in control of, are passing into public domain in terms of infrastructure, mapping etc.

• The focus in the for-profit world in technology has shifted from software as competitive advantage, to utilising data and communities. On the other hand Governments, civic organizations, non-profits etc. are sitting on piles of data. They have active communities of interest which provide for a huge opportunity in data mining to improve the efficiency of their operations.

• I know of one case a social service group in Seattle was dealing with public inebriation. They were spending about 2 million dollars annually dealing with that issue. They went and tracked down the data and found about half a dozen individuals caused 80% of those expenses. They then actively cared for these 6 people through intensive interactions with social workers. In the end they saved half their budget just by looking, and having transparency around that data they collected.

• There is an inhibition within Government around making data more transparent and freely available. Most parts of Government are not paid to take risks. They’re paid to be efficient and maintain the status-quo. I think the future is the Government opening up the data, but not doing the innovation themselves. The tech community can do the innovated based on the raw data feeds from Government.

• There is a lot of talk about the Obama administration naming a CTO. I think the single biggest thing they can do there is make the mandate for different agencies – including Congress – to make data feeds publicly available to people who want to make the world a better place.

Summary

There is lots of data currently available that can be used improve civic life and increase transparency in Government. The Apps for Democracy competition demonstrates that individuals can make use of this data to create usable applications to serve the public good.  This trend looks to continue with initiatives all over the world including the UK’s recent Social Innovation Camp. The most interesting developments, however, will be in relation to the Obama administration and how they’ll approach this subject. If the commitments to public data within their Technology white paper are enacted, the future for transparency and innovation in Government to citizen interaction looks bright.