Methodology

The assessment carried out within the framework of the Barometer 2020 is based on the collection of data from three instruments:

  1. Expert survey: It is an instrument that asks researchers to answer a series of detailed questions about the situation of open data in their country. Each question is answered on a scale from 0 to 10, according to a scoring guide that details ranges and conditions. Each score is accompanied by a justification that contains evidence and sources to support the assigned score. Then, each score and justification is reviewed by a researcher pair and by the project coordination to guarantee the consistency and quality of the work. Survey responses come and go until quality is guaranteed.
  1. Dataset assessment survey: Researchers evaluate the availability of 15 datasets that are recognized as key, according to 10 points that represent the characteristics of open data. The questions are answered Yes or No, and are accompanied by qualitative data that allow understanding the context and justifying the assigned answer. All responses are reviewed by a research peer and by the project coordination; and they are also subject to the same quality control process.
  1. Secondary data: To complement the data collected from the previous instruments, five secondary sources are used, each selected on the basis of theory and its ability to measure important aspects that are not covered in the primary data collection instruments. These are:
    1. Global Information Technology Report 2016 of the World Economic Forum
    2. United Nations E-Government Survey 2020
    3. Global Competitiveness Index 2019 from the World Economic Forum
    4. Political Freedoms and Civil Liberties Index 2020 from Freedom House
    5. Internet penetration 2017-18 of the International Telecommunication Union.

The ODB structure is made up of a 3-level hierarchy. At the highest level are the subscripts, at the middle level are the components and at the bottom level are the variables. A set of variables make up a component and a set of components make up a subscript. The weights of the variables within a component are equal to each other, the weights of the components within the subscripts are equal to each other, and the weights of the subscripts are also equal to each other. The table below presents and briefly describes this structure.

  1. Readiness Subindex: It assesses the willingness of governments, citizens and businessmen to ensure the benefits of open data. This subscript is evaluated from primary and secondary data. Its components are:
    1. Government policies: It assesses the existence of policies and protocols to ensure that open data can be available in the long term.
    2. Government action: It assesses the basis for the benefits of open data to be available at all levels of government.
    3. Citizens and Civil Rights: It assesses the empowerment of citizens and civil society to participate in government decision-making using open data.
    4. Entrepreneurs and companies: It assesses the degree to which companies and entrepreneurs can take advantage of the economic opportunities offered by open data.
  2. Implementation Subindex: Assesses the degree to which governments publish key data sets in an accessible, timely and open way. It is evaluated from primary data and its components are:
    1. Innovation Cluster: Evaluates data commonly used by entrepreneurs in open data applications, or data that provides significant value to the business sector.
    2. Social Policy Cluster: Evaluates data useful for planning, delivering and evaluating social policies, as well as data with the potential to contribute to greater inclusion and empowerment.
    3. Accountability Cluster: Evaluates data that hold governments and their institutions to account.
  3. Impact Subindex: Evaluates the extent to which there is evidence that the publication of open government data has had a positive impact on a variety of sectors in the country. It is evaluated from primary data and its components are:
    1. Political Impact: Evaluates the impact of open government data on transparency and accountability, as well as on the efficiency and effectiveness of government.
    2. Social impact: Evaluates the impact of open government data on the environmental sector and its contributions to a greater inclusion of marginalized groups.
    3. Economic impact: Evaluates the impact of open government data on entrepreneurs, as well as new and existing businesses.

To learn more about the methodology, process, and research method followed in the development of the Open Data Barometer for Latin America and the Caribbean, you can check the research methodology and manual documents. Feel free to contact us if you want to continue the conversation at barometro@idatosabiertos.org 

The data for this edition and comparable historical data are available on our website.