Please visit od4d.com for the latest articles.

Open government data for regulation of energy resources in India

TitleOpen government data for regulation of energy resources in India
Publication TypeReport
Year of Publication2014
AuthorsSrivastava, Nidhi, Agarwal Veena, Bhattacharjya Souvik, Gopalakrishnan Tarun, Meenawat Harsha, Nayak Bibhu Prasad, and Soni Anmol
InstitutionThe Energy and Resources Institute
CityDelhi, India
Report Number2012IA07

The last decade in India has been the most important and eventful decade for access to information in India. Right to Information (RTI) Act in 2005 can be seen as a landmark in the history of access to information in India. The last few years also witnessed the emergence of the National Data Sharing and Accessibility Policy (NDSAP) and an Open Government Data (OGD) portal in India. This study, undertaken by the Energy and Resources Institute (TERI) examines how better availability and access to data can help in governance of resource development, especially energy resources.

India is a resource-rich economy, with strong involvement of the public sector in energy industries. In 2009-10, the public sector accounted for 91% of total coal production, 86 % of crude petroleum and 77% of utilized natural gas. History shows that resource rich states are particularly susceptible to governance failures. While companies and governments benefit, the poor are left worse off despite the resources. As a result, OGD is particularly relevant for the energy industry in India, with the potential for data to be used in addressing a wide range of issues, from resource allocation, funding flows and corruption, through to environmental issues, health and safety, and the more effective and efficient use of natural resources in the country.

Effective natural resource governance also becomes crucial as the country looks towards more private sector and foreign investments. For instance, if natural resources are being allotted through a bidding process, the criteria of selection, information on bidders who submitted bid and who was finally selected should be put out for public scrutiny. In India, concerns have been raised regarding insufficient information on winning bids and final contracts for bid contracts (Sreenivas & Sant, 2009) under the New Exploration and Licensing Policy (NELP). Similarly in the coal sector in India, major concerns have been raised on irregularities in coal block allocations. A CAG report of 2012 confirmed that very little progress has been made in production from the coal blocks awarded and this has resulted in a loss to the public exchequer.  Questions have been raised over lack of transparency in the process of allocation of coal blocks.

Although some energy industry datasets have been made available on India’s OGD portal, there is a much wider, and complex, landscape of relevant data.  Many different agencies are involved in collecting and collating data, with and without statutory basis, and often drawing on data requested from private firms. Whilst many sector specific legal requirements for data collection exist these laws are mostly silent on the dissemination of that data. Agencies often lack technical capacity to manage data effectively, and data is often neither timely, comprehensive nor sufficiently disaggregated. Similar datasets may exist from different agencies, with differences in method or accuracy leading to inconsistencies: making it hard for analyses to work out which data to trust. Users often turn to private sector data sources (from intermediaries who gather and clean government and non-government data), and data from international agencies. The growing usage of private supplier’s data and international agencies is a result of users’ dissatisfaction with the way data is organized and provided by government agencies.

Whilst the RTI Act in India has led to a considerable number of requests for data from energy-related government agencies, replies are generally not available as data in the public domain - being sent only to the requester. Ad-hoc data may also become available through annual reports from government departments, reports of committees and parliamentary Q&As, leading to a scattered data environment. A cultural of secrecy in the working of the government from the pre-independence days also persists, and in the energy industries particular challenges of data-sharing arise due to concerns about commercial confidentiality of data gathered from the private sector. Data is often only provided in PDF, rather then in ‘machine-readable’ formats, and data can be expressed in ways that make it difficult to understand.

In order to address the challenges of making open data available to support governance in energy industries in India this study recommends:


  • Fostering a culture of openness -  While there is a need for stronger enforcement of existing laws and policies, it is important to create a ‘culture of accountability and transparency’ in the government, further sensitising government officials and staff around issues of accountability to the public and not only to political heads.
  • Making open data useful for improved governance and service delivery - In a situation where many citizens may not be able to make sense of the data, one must ensure that data does not result in widening the gap between public and governments. There is need for ‘intermediaries’ to effectively utilise data, develop visualisations and applications that make sense of data and is easily comprehensible by public at large.
  • Clarifying confidentiality - The ‘need for confidentiality’ emerges as a major impediment in getting access to government data. There is however a lot of ambiguity on what data is confidential and what can be shared, and greater clarity on this is needed.
  • Independent verification of data - Independent verification of data and/or appropriate quality checks needs to be in place before putting out data to ensure quality. Where data needs to be collected and compiled from different agencies, formats and processes should be such so as to require minimum human intervention.
  • Streamlining data - Data sharing formats need to be made uniform across government departments in order to ensure inter-operability. Data users suggest that having a single source of data for Energy as against multiple sources for different fuels would streamline data and make analysis and comparison more easy. Greater consistency and collaboration on definitions and methodologies for generating data is also required.
  • Inter-linking data to avoid multiplicity - More data does not necessarily mean more openness in government and its data. Data deluge is something that the OGD portal has to stay away from. It is important that there is a system in place that links to different datasets that provide the same information, either on the data portal or elsewhere. Reducing multiplicity is a key step in avoiding confusion and lack of clarity about which is the most reliable and best quality data source.
  • Usefulness as important as usability - The data portal in its short span has put a variety of data in a readily accessible and machine readable format. While the focus on a machine readable format is immensely helpful for usability of data, it may not always be sufficient to make data useful for improved transparency and governance.
  • Collaboration around open datasets - NDSAP and the data portal depend on various government agencies to upload data. The portal allows citizens and users to place a request for a certain dataset online. While this is a good practice where citizens can directly influence data availability, it is important for the data portal and it’s implementing agencies to seek data in a structured manner. Towards this, a multi-stakeholder group or the communities can be engaged with to help in proposing a clear list of data that can be uploaded for different sectors.
  • Awareness about data portal - Awareness about the portal needs to be increased, especially amongst government agencies to facilitate better contribution of data in this portal and ensure that government uses this data in its decision-making. With respect to awareness amongst non-government users, many users are still relying on the data sources used earlier when improved sources may be available.