Reflections on Impact: PIA 2018 Data Assessment

March 15, 2019

As PIA moves forward to a new year of activities and learning in 2019, we at MIT D-Lab have also paused to reflect on the past four years and assess the program’s impact to date. 2018 was a year of introspection and analysis for PIA, learning from the program’s activity data and member input in order to enhance the program and inform PIA’s future development. In this post, we share out several findings from this analysis and the process through which we evaluated each facet of the program. We are also excited to showcase several data highlights and PIA’s new theory of change.

PIA Data

Since PIA’s inception in 2014, the program has gathered a rich collection of qualitative member data from biannual, 1-1-member engagement calls, annual surveys (baseline and end line), and PIA activity-specific surveys.

Who is a part of PIA?

As an institutional membership-based network, current and former PIA members represent a variety of institutional sectors as well as a diverse group of individuals. PIA’s organizational members include corporations, international NGOs, social enterprises, and governmental organizations. Within those organizations, individual members across a number of functions from operations to innovation and more (see charts below). The diverse PIA network is a key resource that provides unique opportunities for PIA members.

In 2018, PIA introduced a number of additional milestones including the 2018 Annual Survey and the Metrics Taskforce in order to better evaluate overall program performance:

2018 Annual Survey

In 2018, PIA’s Annual Survey served to merge elements of its data gathering processes into one more comprehensive survey. Baseline data for the survey was captured during the 2018 PIA Annual Meeting which took place at MIT D-Lab April 18-20 and end line data captured in November of the same year. See several of the survey highlights below.

Members saw the value of PIA as “a unique organization, offering a hub of information and access to broad networks of organizations that have similar goals regarding addressing issues in the developing world.” Members emphasized that the working group activities and “tools developed by D-Lab under PIA are so useful and make innovation accessible and easy to implement in countries in dire need for it.” Members also saw the PIA network as a crucial player in organizational collaborations.

When asked about their individual and organizational outcomes, PIA members report to have significantly increased their focus on engaging BoP users and beneficiaries in a more participatory way. In fact, 50% of members report, “improving practices in their current projects" as a result of being a part of PIA. Members also report that participation in PIA influenced and improved their organization through their impact measurement systems, the way they pursue partnerships, and their design process.

PIA Metrics Taskforce

As part of our efforts to reflect on PIA’s trajectory to date and imagine its future, in 2018 PIA convened the PIA Metrics Taskforce. Comprised of a smaller subset of the PIA member network and MIT D-Lab staff, this taskforce convened over a period of six months to co-design an impact framework and measurement strategy for the program. The Metrics Taskforce reflected on PIA member input, desired outcomes and experiences to articulate the program’s efforts thus far as well as its desired vision for the future. We are very thankful for the valuable input and time of the members who participated in these efforts.

The Theory of Change below is the culmination of conversations with the wider member network as well as the Metrics Taskforce, analysis of the data PIA has collected, and our aspirations as an organization. PIA’s Theory of Change is a document that outlines how PIA intends to affect its members and organizations as a whole through its work. Please find our new Theory of Change below and we welcome your comments.

 

 

--Amanda Epting, Manager, Practical Impact Alliance