Webinar on the evaluation of policy research

Next Wednesday, September 21st at 1:oo pm GMT I will be presenting the approaches and methodologies on the evaluation of policy research documented in “Who Used Our Findings?” at a dedicated webinar through On Think Tanks:

Can policy research be evaluated and how? The case of American foundations and think tanks

The webinar is free and there are still plenty of tickets for participation in the discussion that will follow the presentation.

Incidentally, the On Think Tank School is offering a long course on M&E for think tanks starting on September 26!

Webinar outline

The webinar will focus on the approaches and methods used by US foundations and think tanks to evaluate policy research –and its impact.

To do so, it will introduce the general practices of think tanks, but also the most innovative practices identified in the expert interviews.  On the one hand, think tanks tend to promote their presence in terms of how many products they put out, how many times they are quoted in the media, or how often do they appear before congress. Slowly but surely, foundations are attracting them towards more ambitious attempts to understand influence that usually include an increased interest in the research uptake of core constituencies, but also the gravitation towards more explicit advocacy efforts.

Master’s Thesis: How do Foundations evaluate Think Tanks?

The interest in telling apart what works from what doesn’t is widespread in the philanthropic community, particularly in organizations that devote their efforts to policy change in the United States and around the globe. With the goal of maximizing the efficiency of their grants, prominent foundations, like the Bill and Melinda Gates Foundation or the Annie E. Casey Foundation, have confronted the need for specific methods in monitoring and evaluation that can be applied to the work of the nonprofits they support.

Credit: W. K. Kellogg Foundation. Logic Development Guide (2004)

However, little attention has been paid so far to the relationships between research-oriented organizations and the foundations that support them –with good reasons: it is extremely hard to prove the impact and utilization of a policy report.

Continue reading

Puntualizaciones al reportaje de El Confidencial sobre la financiación de think tanks por empresas del IBEX 35

This post is entirely written in Spanish. It contains some clarifications with regards to an article published in the Spanish online newspaper El Confidencial . The article features original research published in this blog on July 2015 about the links of corporations to think tanks in Spain.

Durante las últimas semanas he estado en contacto con Daniele Grasso, periodista de datos de El Confidencial, para preparar un reportaje que incluyera mis resultados sobre las relaciones de mecenazgo entre las empresas del IBEX 35 y los think tanks españoles. La idea era incluir una visualización interactiva que mejorara lo que yo había podido publicar por mis propios medios en este blog hace unas semanas. El reportaje ha salido hoy con el siguiente titular: ¿Cómo influir sobre el debate público? 22 empresas del Ibex financian 11 ‘think tanks’

Captura de pantalla (48)

Es un orgullo que tu trabajo aparezca como pieza central de un diario tan leído y agradezco a Daniele que se interesara por el proyecto. En este post voy a aprovechar para hacer algunas aclaraciones que, por cuestiones de espacio y tiempo, no se coligen del reportaje.

  • La visualización está basada en esta hoja de cálculo que he generado visitando las webs de todas las organizaciones pertinentes, añadiendo indicadores que permitían hacer un análisis de clusters, etc.
  • Por lo que respecta a los datos, lo más importante es que no son 11 think tanks, sino 12. Y es que falta Ecodes, el think tank que obtuvo la calificación máxima de 5 estrellas en el rating de transparencia que Francesc Ponsa y yo publicamos en el blog On Think Tanks en enero de este año. Ecodes es una fundación especializada en problemas de desarrollo y agua. Su buen trabajo les hace merecedores de importantes apoyos financieros de La Caixa, Ferrovial o Abertis, entre otros. Lo que es más importante: su caso es un recordatorio de que la presencia de financiación de grandes empresas no es condición suficiente para la falta de transparencia ni la presencia de intereses espurios.
  • Según los datos de mi investigación, las empresas del IBEX están presentes en el 80% de los think tanks españoles. 12 de los 15 think tanks analizados tenían mecenas del IBEX 35. Hay muchos más think tanks, pero ninguno de los restantes publica sus fuentes de financiación.
  • Con respecto a la metodología, hay que tener en cuenta un componente interpretativo no banal. La presencia en las webs de los think tanks de logos de empresas, fundaciones y universidades se ha interpretado como prueba de que esas organizaciones aportan fondos al think tank. En el caso de las universidades y las fundaciones, es fácil que se trate, más bien, de colaboraciones para proyectos específicos. En el caso de las grandes empresas la relación de patrocinio es la más probable. Pero puede haber errores.

La entrevista

Publico íntegramente las respuestas a las preguntas que me envió Daniele, demasiado largas para incluirlas en el reportaje.

– ¿Cuánto peso dirías que tienen, a día de hoy, los TT en España? ¿Consiguen tener impacto sobre las políticas? ¿Tienes algún caso práctico de ello?

En general, los think tanks en España no están muy desarrollados, aunque se trata de un campo en crecimiento. Hay algunas instituciones muy bien valoradas y relacionadas, como FRIDE y el Real Instituto Elcano, que trabajan en el ámbito de las Relaciones Internacionales y la política exterior. Sin embargo, son excepciones al panorama general.

La pregunta sobre el impacto es muy complicada. Es lo más difícil de medir y prácticamente imposible de probar, tanto en España como en Estados Unidos. Para que la propuesta de un think tank sea adoptada a corto plazo por un equipo de gobierno y aprobada por el poder legislativo se tienen que dar muchísimas circunstancias, es un milagro más que el funcionamiento general. Hay cierto consenso sobre la necesidad de considerar el impacto de los think tanks en términos de contribuciones, más que de aportaciones específicas. Además, a la hora de preguntarse si un think tank tiene influencia una de las cosas que hay que mirar independientemente de los resultados políticos, que son impredecibles, es si ha conseguido abrir un debate que antes no existía. Hay unos cuantos casos en España. Por ejemplo, el Círculo de Empresarios ha contribuido a poner en el mapa cuestiones como la privatización de monopolios estatales, que eran casi impensables antes de que ellos empezaran a estudiarlas. Y FEDEA, por poner otro ejemplo, lleva tiempo abogando por la implantación de un contrato laboral único. No están solos, pero son una de las voces más fuertes junto a Politikon, que yo empiezo a considerar un claro modelo de think tank digital independiente.

Por el momento, casi nadie les hace caso, pero es que el ciclo temporal estándar que se considera para que una nueva propuesta sea adoptada es de unos quince años. Es un plazo muy largo.

– Resulta claro que las grandes corporaciones quieren influir a través de los Think Tanks. ¿Cómo se manifiesta este tipo de influencia?

Las empresas españolas tienen una presencia estable y orgánica en varias fundaciones que funcionan como su brazo investigador. Algunas de ellas funcionan como asociaciones empresariales, realizan propuestas y estudios destinados a servir a sus asociados, representando sus intereses ante el aparato político. No hay nada malo en ello. Cuando vemos que se lanzan nuevas propuestas para el crecimiento de las pymes o el apoyo a la internacionalización, nos encontramos ante un trabajo de carácter altamente técnico que debe ser tomado en serio. Muchas grandes corporaciones forman parte de estas asociaciones, como es el caso de COTEC.

El caso de FEDEA es parecido, pero con matices. Aunque naciera bajo el auspicio del Banco de España hace ya 30 años, tenga un perfil más generalista y reclute a académicos de prestigio para sus investigaciones. Fedea no es “el think tank del IBEX35”, como se lee a veces en los periódicos; pero es cierto que está financiado por algunas de las empresas del IBEX y que aspira a convertirse en el think tank económico de referencia en España. De hecho, su página web publicita que se ha convertido en el interlocutor privilegiado con el FMI y la OCDE. Parece que su objetivo sea convertirse en los representantes de la ortodoxia técnica.

Más en general, aunque el mapa parece constatar que existe un interés por parte de muchas grandes corporaciones en aprovechar la persuasión blanda de los think tanks para ejercer su influencia sobre el mundo de la política hay que reparar en que muchas de estas empresas, como los antiguos monopolios estatales, han tenido siempre comunicación directa con la alta política y no necesitan a los think tanks. Su presencia en el mapa se puede deber a su peso incomparable en el mecenazgo privado, especialmente en el caso de los bancos. Haría falta investigar más para poder determinar la motivación detrás de los patrocinios que hemos observado.

– Sobre el papel, los Think Tanks debería ser fuentes de conocimiento y ‘expertise’ independientes (a menos que se trate de TT vinculados a partidos, claro). ¿Crees que, en España, la presencia de grandes empresas quita credibilidad al trabajo de los Think Tanks? ¿O quizás pese más la “mala imagen” que dan la relación entre determinados TT y partidos políticos?

El hecho de que muchas de las más grandes empresas españolas estén involucradas en la financiación de think tanks no solo es inevitable, sino que no debería ser negativo de por sí. De hecho, en el mapa se ve que las empresas del IBEX35 financian a las instituciones que les son más afines, pero también a think tanks más pequeños e independientes en una labor de mecenazgo que tiene el potencial de soportar la diversidad en el mercado de las ideas, aunque ahora mismo esté poco desarrollada en nuestro país.

Lo que sí resulta preocupante es que los dos modelos de think tanks predominantes en España, es decir, las fundaciones de partidos políticos y los think tanks empresariales como el Círculo de Empresarios, FEDEA o el Cercle d’Economia en Cataluña, estén afiliados a intereses específicos que suelen tender a la conservación y al elitismo, y no cuenten con un contrapeso suficiente en el espectro progresista. En este sentido la clave, más que en la credibilidad, está en la competencia. Yo no creo en la imparcialidad de los think tanks, algo que todavía se defiende en Estados Unidos, porque todo think tank tiene una ideología, unos valores y unos intereses, motivados tanto por su misión como por compromisos adquiridos con los patrocinadores, aunque luego puede funcionar autónomamente en la gestión diaria y en el trabajo de investigación. El problema es que las instituciones más conservadoras tienden a ser más populares entre quien tiene dinero disponible para financiar. Por eso es importante promover una mayor transparencia que permita conocer cuánto dinero se está moviendo en este ámbito, además de apoyar a instituciones más pequeñas para que puedan aumentar su impacto.

Actualización: Algunas de las aclaraciones se han eliminado tras haberse introducido modificaciones en el texto del reportaje.

How influential are corporations in Spanish think tanks?

Update: The results presented in this piece have been featured in the online newspaper El Confidencial, including an interactive visualization with d3. You can read the article (in Spanish) here, and some clarifications to the report here

Access the dataset

This research question quickly stumbles against two main barriers. One is the opacity of the field, with only 21 out of 48 institutions disclosing some kind of information regarding their revenue sources; the second is the cumbersome task of operationalizing corporate influence in an effective way.

The first barrier can be overcome with some stubbornness determination. Let’s measure what we can, and conform the project to the subset of ‘somewhat open’ organizations, in the hope that results will be representative or may be improved in the future. The second barrier is much more substantial, and I fear that my response is very partial at best. My operational definition of ‘corporate influence’ is the existence of funding provided by one of the companies in the IBEX35 (the index of the 35 most important companies in Madrid’s stock exchange).

1. Mapping methodology

  1. Check the websites of the think tanks with at least one star in our DIY Transparify like rating of Spanish think tanks. I also excluded party foundations, as I know that they never identify their sources of private funding;
  2. List all supporting entities. This generates some noise: it’s common practice to recognize sponsors with the inclusion of their logos in the projects they support; but quite often there is no way of distinguishing true sponsors and donors from partners, collaborators and clients;
  3. Cross-reference supporting entities with IBEX-35 companies;
  4. Generate an affiliation matrix;
  5. Upload the affiliation matrix to NodeXL;
  6. Code vertices by
    • type of institution (donor(circle)/grantee (pink square));
    • node degree (number of ties);
    • industry. I used banking (purple), energy (green) and infrastructures (orange), as they are the most important with the results at hand.

Ibex35 dinero

2. Structural similarity

A second step I took was to classify the grantees by similarity of their revenue structure. I chose to do a cluster analysis on SPSS with the following metrics:

  • Total IBEX35 funders
  • Ratio of IBEX35 to non-IBEX35 funders
  • Total public funders

To be honest, I played with the different possible measures until I saw clusters forming, so this is more of a heuristical approach than a solid taxonomy. However, even given the small amount of information on which this analysis is based, it’s interesting to see, at the bottom, a cluster of corporate think tanks clearly bound together and distinguished from outliers like Fundación Sociedad y Educación or Fundación Ortega y Gasset, which has as many ties to IBEX35 as COTEC or FEDEA, but a lot more ties to public institutions both in Spain and in the Americas; at the top, a cluster of independent organizations with no relation to IBEX35 companies, and a middle group of more diversified entities which includes ECODES (which was rated 5 stars); and Fundación Alternativas, related but formally independent from the Spanish socialist party.

taxonomy

The research question can be answered with a grain of salt. The network mapped is quite dense in spite of the fact that a lot of institutions are missing. However, since many think tanks don’t disclose their total budget or the amounts provided by each donor, an accurate measurement of the weight of corporations as a ratio of overall support is impeded. In any case, this exercise has identified a number of central actors in the interaction between corporate interest and policy recommendations and suggests the interest of many of the biggest Spanish companies in using corporate think tanks as their proxies.

Table 1. IBEX35 companies and number of funding ties to transparent think tanks

IBEX35 COMPANIES TIES
ABENGOA 1
ABERTIS 6
ACERINOX 0
ACS 3
AENA 0
AMADEUS 0
ARCELOR MITTAL 0
BANCO POPULAR 2
BANCO SABADELL 2
BANCO SANTANDER 7
BANKIA 2
BANKINTER 0
BBVA 5
CAIXABANK 7
DIA 0
ENAGÁS 1
ENDESA 4
FCC 1
FERROVIAL 3
GAMESA 0
GAS NATURAL 4
GRIFOLS 0
IAG 2
IBERDROLA 5
INDITEX 1
INDRA 4
MAPFRE 1
MEDIASET 1
OHL 5
RED ELÉCTRICA 0
REPSOL 8
SACYR 0
TÉCNICAS REUNIDAS 1
TELEFÓNICA 7

Table 2. Think tanks with at least some degree of financial transparency and number of ties to Ibex-35 companies

THINK TANKS TIES
CATALUNYA I EUROPA 4
CATPE 1
CIRCLE D’ECONOMIA 13
CIRCULO DE EMPRESARIOS 14
CITPAX 7
CLUB DE MADRID 0
COTEC 13
ECODES 5
FEDEA 12
FRIDE 0
FUNDACION ALTERNATIVAS 5
FUNDACION CIUDADANIA Y VALORES 1
FUNDACION ORTEGA I GASSET 12
FUNDACION SOCIEDAD Y EDUCACION 2
REAL INSTITUTO ELCANO 0

New paper on the standards of transparency for think tanks

Conference Milan

Next Saturday, July 4 I will be presenting at the II International Conference on Public Policy in Milan, Italy the results of my research on the accountability of think tanks. The paper, entitled “Towards new standards in the ethics of policy research organizations” explores the following questions:

  • How can we speak of the transparency of think tanks beyond the disclosure of funding?

Financial transparency is an important tool to uncover the influence of special interest in the work of think tanks; however, it’s clearly not sufficient if it’s not followed by more insightful research on the content of policy recommendations that can identify who final beneficiaries are and what is the ideological framing of a certain organization. In other words, the mere acceptance of financial transparency as a sufficient index of openness can reinforce the perception of compliant organizations as sources of non-aligned expertise, instead of revealing the specific forms of influence brokerage and power-knowledge interactions. But can advocates push their demands further and identify new typologies of critical information for proactive disclosure?

  • Should think tanks be accountable in spite of their formal independence from both the public and the private for-profit sectors?

The notion of accountability is a very strong topic of debate among public administration scholars and practitioners. Without a commonly accepted definition, it stands for the ability of public servants and elected officials to be responsive towards citizens, but it’s a heavily context-dependent notion with severe operational difficulties. Can this notion be applied in the context of (usually ) private institutions like think tanks? What adaptations and requirements should be met for this translation to happen?

  • Is current investigative journalism efficient in promoting an informed debate about the role of policy research organizations in advanced democracies?

Information covered by the media is naturally skewed towards the ‘newsworthy’. Can this feature be supportive of a nuanced discussion about the role of think tanks, or does it naturally prevent the public from distinguishing between more and less accountable think tanks?

You can read the abstract below and access the complete paper here. Also, feel free to browse the rest of the panels of this first-tier conference.

The traditional understanding of American Think Tanks as neutral sources of technical expertise free of ideological orientation has been severely damaged both by the advent and dominance of advocacy focused institutions since the nineteen eighties (Rich, 2005) and by radical critiques according to which the field of Think Tanks has relegated social and political scientists working in university departments to the margins of public debate, a move that has especially damaged leftist thinkers (Medvetz, 2012). Departing from the acknowledgement that in the twenty first century bias and ideology are not to be considered obscure weaknesses, but much rather strategic elements and definers of the set of values that underlie the work of any credible Think Tank, this paper will analyze the complex feedback between investigative journalism and financial transparency as a collaborative enterprise aimed at exposing the relation between research and interest. Both unintended consequences and positive outcomes of the investigation/disclosure alliance will be highlighted using, among other evidence, recent pieces appeared in the major American newspapers and interviews with scholars of different Think Tanks based in Washington, DC in view of attaining a more nuanced notion of accountability. Different institutional notions of credibility and creation of value will then be compared with the most advanced standards derived from the public sector (accountability) and from the business and marketing fields (non-profit branding), but specifically applied to the field of Think Tanks. The results will be highly valuable for both scholars in the field, transparency advocates and practitioners, especially if involved with new policy research enterprises.

Key words: Think Tanks, accountability, transparency, ideology, bias

Interview about the Spanish rating in an online newspaper

Ibercampus, an online newspaper specialized in Spanish and Latin American Higher Education, sent me some questions regarding the Transparify rating of Spanish Think Tanks that I did with Francesc Ponsa for On Think Tanks.

Here is the link to the full interview:

http://www.ibercampus.es/los-think-tanks-espanoles-los-menos-transparentes-del-mundo-29602.htm

Transparency and reputation of Spanish Think Tanks

Complete transparency rating of Spanish Think Tanks

This has been an exciting week. On Wednesday my article with Francesc Ponsa, Director of the Spanish Office of the Observatory of Think Tanks, was published on Onthinktanks.org. Today, January 22, McGann’s 2014 Global Go To Think Tank Index Report, the most renowned independent evaluation in the field of Think Tanks, came out. Our article, entitled ‘How transparent are Spanish Think Tanks’, is a complete transparency rating of all Spanish Think Tanks using Transparify’s framework, and a powerful diagnostic of the development of the field, the types of organizations that fall under the ‘Think Tank’ tag, and the current and still marginal culture of disclosure. Out of 48 institutions, 27 disclose no financial information whatsoever, 19 were given a 1 or 2 stars rating, and only 2 had a rating higher than two, out of five possible stars. Fundación Ecología y Desarrollo (ECODES), a small environmental Think Tank, and the foreign policy Fundación para las Relaciones Internacionales y el Desarrollo Exterior (FRIDE) lead the rating with 5 and 3 stars respectively.

The reputation of Spanish Think Tanks

A number of the institutions we reviewed appear in McGann´s Index, showing that, in spite of the marginality of the sector and the still low profiles these organizations have in Spanish media, they are well known internationally and, in some cases, are even important partners and members of international networks. This is the case of Real Instituto Elcano, that hosted the co-authors of two of the top three reports profiled in the index. Here is the complete list of Spanish institutions in GTTTI and the rating we assigned them:

The reason why the last bullet does not link to an external website is simple: IDEAS, formerly the party tank of the Spanish Socialist party, was discontinued in 2013 following a scandal of nepotism, and its staff relocated in Fundación Pablo Iglesias, the official non-profit arm of the party. This is an illustration of the lack of rigour of the GTTTI report, which seems to have defective procedures for factual check and quality assurance. The rating is not applicable to Fundación Carolina because it is an endowment Foundation, not a Think Tank, and to Bankinter’s because this is a corporate Think Tank, which means that the transparency standards for independent institutions do not apply. In any case, it´s somewhat encouraging to see that the average transparency rating of better known institutions is higher than that of the complete sector: 1.4 rather than 0.7, including GRAIN. This  doubles the general average, and I consider it a reasonable indicator that the consolidation of the sector in the years to come may also result in more disclosure and stronger reputations. It should be pointed out the FRIDE, a reasonably new player, responded directly to Transparify’s first evaluation one year ago and has disclosed their funding in different brackets as a direct result of Hans Gutbrod’s team advocacy. A fantastic example of why this work matters.