Please visit od4d.com for the latest articles.

Towards common methods for assessing open data: workshop report

Workshop in New YorkGovernments across the world are adopting open data policies and practices. From national portals, to municipal open data initiatives, and sector-specific efforts in transport, health and international aid to name just a few, open data has been adopted as an important governance innovation.

In line with the growth of open data, a range of different efforts have emerged to measure various aspects of open data readiness, implementation, outcomes and impacts. In order to explore the state of the art in measurement of open data activities, and to explore opportunities for future collaboration in their development, just over a a month ago on May 8th and 9th 2014, The World Wide Web Foundation and The Governance Lab at NYU convened a two-day workshop in New York, bringing together open data assessment experts to explore the development of common methods and frameworks for the study of open data.

This two day workshop started by reviewing existing research and projects on open data measurement and looking at use cases assessment, before identifying key questions concerning open data and looking at conceptual frameworks that could connect questions to use cases. Building on this foundation, the second part of the workshop focussed on identifying common categories and indicators within an overarching framework. Meeting participants split into sub-groups to consider specific questions, and to build out the larger framework with which we can study open data.

Today we're publishing the full workshop report (PDF), and the draft framework (Google Doc) this resulted in for comments and feedback.

The framework at a glance:

Screenshot of the frameworkIn assessing open data activities, a project may look at:

  • Context/Environment - The context within which open data is being provided. This might be the national context in the case of central Open Government Data, or might be the context in a particular sector. Important aspects of the environment to assess include the legal and regulatory environment; organisational context; political will & leadership; technical capacity; the wider social environment, in terms of civil society and political freedoms; and the commercial environment and capacity of firms to engage with open data.

  • Data - the nature and qualities of open datasets. Including the legal, technical, practical and social openness of data, and issues of data relevance and quality. The framework also looks to identify core categories of data which might be evaluated in assessments.

  • Use - the context of use of the open dataset. Includes the category of users accessing (or providing) the dataset, the purposes for which the data will be used, and the activities being undertaken. This part of the framework addresses the who, what and why of open data programmes.

  • Impact - the benefits to be gained from using the open dataset. Potential benefits can be studied according to social, environmental, political/governance, and economic/commercial dimensions.

In the framework draft we provide suggested questions and indicators for each of these components, and look at the existing projects that have piloted relevant methods. You can add your comments to the online version of the framework on Google Docs.

Building on a common framework

As we explored in our workshop in New York, there are many diverse reasons to assess open data activities (from checking implementation of a plan, through to critically researching what kinds of approaches to open data yield the most equitable social outcomes), and open data activities are operating at many different levels (from national open data policies, to sectoral programmes in health or education, to individual small NGOs adopting open data practices) and different assessment projects will need to pick-and-mix from the overall framework to select the kinds of questions and indicators of most relevance in any particular instance.

However, having a common framework to draw on allows researchers, policy makers and practitioners to share definitions, to create comparable data, to collaborate on the design of measurement instruments and training and ultimately to join efforts in building up a rich picture of the evolving open data landscape across the world.

In the report you will find details of the next steps we have planned for this work, including creating space for ongoing sharing and discussion amongst those working on open data measurement.

Related documents: