An approach for emerging Asia
Credits
Analysts
Ms Khor Qianyi, Senior Analyst
Mr Zhou Yang, YLP Analyst
Research
Mr James Tan
Overview
Impact investing is an area that is quickly gaining traction all over the world, with more than 450 investors allocating US$1.3 trillion to impact investing worldwide1. Seeking the dual goals of profit as well as social impact, it is a market based approach to resolving critical problems faced by communities worldwide – from access to education to poverty alleviation. By leveraging the power of the market, impact investments achieve outcomes that would have been enormously costly if attempted by traditional methods of philanthropy. Take d.light design for example, a startup that only had a solar lantern prototype when the Acumen Fund first invested in them, which has to date brought lighting to over 100 million people2.
While there have been many frameworks already crafted to guide the process, it is tenuous to apply existing frameworks directly to early stage investing due to the diversity of portfolio as well as limitations on quantitative data available. Moreover, there are many iterations of different frameworks, so impact measurement and reporting remains fragmented across different organisations. This report seeks to survey existing methods available and piece relevant metrics to guide impact investing in the different phases of early stage venture capital.
Foreword
Mr James Tan
Managing Partner
Quest Ventures
With more than US$1.3 trillion allocated to impact investing worldwide, the world is seeing an influx of capital that is increasingly comfortable with measuring successes and returns not by financial metrics alone but by a combination of that and other factors. The dual goals of profit and social impact are now broadly termed the double bottom line. While the financial element – profit – has clear measurement guidelines such as the IFRS or GAAP, the social impact element does not. In fact, many impact measurement frameworks are available and the world has not settled on a definitive one to use. To the detriment of meaningful projects, this means that when they fund raise across the globe, their definition of social impact success has to be explained ad nauseum.
This report aims to change that.
We fundamentally believe that if the measurement of success is correct, then the inputs will be correctly deployed. Conversely, if the measurement of success is wrong, then the inputs will be wrong. When outcomes are measured in lives and the betterment of lives, then getting the measurements correct could not be more important.
We are therefore “open sourcing” our investment measurement framework with this publication and welcome inputs from all interested parties to refine this. We are excited that we now have a measurement framework tailored to emerging Asia based on our experience in this region. We look forward to its implementation and continued refinement in many investments from here on now.
Overview of the Impact Measurement Process
The three main reasons to measure impact are firstly to guide investment decisions, secondly to determine and monitor how impactful the investments actually are and lastly to report to stakeholders in the impact fund. Different methods are used to serve each of these purposes at different phases of impact measurement. In the report “Measuring the “impact” in impact investing”, Ivy So & Alina Staskevicius of Harvard Business School accurately divided the process into 4 phases. Starting with Estimating Impact, followed by Planning Impact and Monitoring Impact, then finally Evaluating Impact3.
Firstly, estimating impact involves performing a thorough due diligence on the startup that the fund is investing in. The due diligence process is similar to traditional investing, but with an additional responsibility of analysing the potential impact of the investment. For early stage investing, there will not be an abundance of quantitative data that can be provided in the due diligence process so other objective standards have to be referenced. One standard that we have found to be useful is the Nesta Standards of Evidence4, which will be analysed in further detail in a later part of the report. This is the phase of impact measurement that will guide investment decisions.
Secondly, planning impact involves devising specific metrics for data gathering. This will depend on the portfolio of the fund, and its investment thesis. Once again for early stage investing there will not be much data initially, so this process is rarely used to guide investment decisions. However for startups that the fund has made an investment in, this phase will be relevant in monitoring its progress towards its impact goals, as well as in reporting to key stakeholders. While there are many different metrics available, we have found the IRIS catalogue of metrics by Global Impact Investment Network (GIIN)5 particularly useful in determining the areas of impact to monitor at a portfolio level. The IRIS standards provide a wide range of metrics, of which the most relevant ones to individual funds can be picked to form its own metrics. For example, a fund that specifically targets impact in the bottom of the pyramid sector can choose relevant metrics as such.
Thirdly, monitoring impact will involve data collection based on the metrics that were determined in the previous phase. Data collection would involve enforcing periodic impact reporting based on a template as provided by the fund. This phase is important for the fund to keep track of the progress of its investments, to ensure that its portfolio is in line with its initial mission. This data would also be useful in impact reporting to key stakeholders in the fund, and to justify the investment decisions that were made.
Lastly, evaluating impact involves a deeper analysis of the data collected. The purpose of this phase is to evaluate whether there has been real impact from the investment, taking into account factors including whether the impact outcomes would have been achieved with or without the investment into the startup.
These four phases will be further discussed in section 5.
Importance of measuring impact
Impact measurement and management (IMM) is an integral part of impact investing. If an impact fund has no protocol to track the impact of investments made then it is akin to a traditional fund not tracking their profit/loss. IMM allows impact investors to keep track of the progress towards its impact objectives. Consistent management of impact tracking directly shows what kind of investments work and what does not. Over time it allows the fund manager greater ability to discern the investments with better impact value6, and this is particularly important for early stage investing, when fund managers have to make decisions even when quantitative data is scarce.
Another purpose of IMM is to allow clear reporting of the progress of the fund. Investors into impact funds usually have their own impact targets as well, and when clear reporting standards are observed, it builds trust with investors. If this is communicated effectively, through yearly reports or other forms of outreach, it demonstrates the competency of the Impact Fund, which would naturally come in handy during the next capital raise phase.
Survey of existing frameworks
IRIS by Global Impact Investing Network (GIIN)
The IRIS catalogue of metrics provides a wide range of metrics which can be used to quantify the level of impact achieved in environmental and social factors. It originated from the GIIN, and is largely built upon existing industry accepted metrics, compiled together for fund managers to access7. It aims to increase the consistency of impact reporting methods, and to bolster the credibility of the impact investing sector as a whole.
In general, such methods involving assigning metrics and scoring the investee based on it can be referred to as Mission alignment methods8. Such methods track how the execution of a project measures against its mission goals over a period of time. They are relevant in the planning impact and monitoring impact phase of the impact process.
The IRIS framework is the closest we have towards a widely accepted standard for impact investing. According to a study published by GIIN in 2020, 65% of impact investors surveyed use the IRIS metrics in the impact measurement process9. One reason for its widespread adoption is that the diversity of metrics catalogued allows it to be adapted and used by a wide variety of funds with different impact objectives. A curated set of metrics also allows the impact fund to view, at a portfolio level its progress towards impact goals. It also allows the early stage investor to compare the growth of its portfolio companies using standardised yardsticks. For instance, a fund makes an investment of USD 500,000 into two different companies, and receives their impact report after a period of 1 year. One company reports that it created 100 jobs, while the other reports that it played a role in supporting 300 jobs. Due to different understanding of the terminology, it is difficult to compare the results of the 2 companies. It is also imprecise to add the 2 values together to gauge overall portfolio performance. This is where the IRIS metrics come into relevance, with its catalogue of metrics with standardised accounting methods.
Logical Framework Approach (LFA)
The logical framework approach is a general project management framework that is widely adopted internationally. It covers all phases of the project cycle, from design to implementation and monitoring and finally to evaluation, which makes it useful for impact measurement and management.
The Logical framework is able to communicate the essential elements of a complex project clearly and in simple terms for anyone to pick up10. Usage of such a framework by early stage startups is extremely beneficial in the growth stage. While using the LFA gives companies an efficient tool to clarify their goals and track progress, the measurables and outcomes are often unique to each startup, and cannot be collated with the results from other portfolio companies to give a portfolio level measurement of impact.
Thus while the LFA is a viable tool to track impact at the company level, additional frameworks are required to support such data in order to provide a portfolio overview of impact performance.
Nesta Standards of Evidence
The Nesta Standards of Evidence is a rough framework developed to gauge the levels of confidence in the evidence of an impact initiative, on a scale from 1 to 5. It was modified from the standards used in Greater London Authority’s Project Oracle11 to better fit the demands of venture capital.
In general the Nesta framework supports the strengthening of evidence to allow investors to be more certain that the intended impact outcome of investment is achieved. As a company moves up along the levels, more extensive data collection as well as external validation is expected. When a company reaches level 5, there is sufficient evidence that the business model is able to deliver strong impact at multiple locations – this means that it has achieved impact scalability, a key signal that venture capitalists look for13.
The Nesta Standards is useful for early stage venture capital in two phases: the estimating impact phase, as well as planning impact phase. Before committing an investment into a company that claims significant impact value, the investor can assess the evidence based on the Nesta scale to ascertain the strength of its claims. For a company to which an investor has already committed capital, the Nesta standards can guide the next steps in scaling impact planning.
However, at higher levels on the Nesta scale, more complex testing requiring external specialist assessment would be required. This is costly, and in many early stage startups not financially viable. Investors should consider whether to fund such assessments based on their priorities and cheque sizes.
Phase 1: Estimating Impact
Phase 1 of the impact investing process is where thorough due diligence is done on the potential investee to assess profitability and impact. As for the profit dimension we are looking for companies that are able to provide strong potential of generating risk adjusted market rate returns. We do not consider at all companies that have no promise of that, firstly to be responsible to our investors and secondly we do not intend to support ineffective business models that could bring more harm than good to the markets we want to grow. As for the impact dimension, we have found the Logical Framework Approach and the Nesta Standards of evidence particularly relevant, and have designed a simple matrix that draws on both frameworks to aid in decision making.
Companies that fall under the Impact Stars group have achieved a relatively high score on the Nesta scale, at 3-4. Based on their logframe, they have also demonstrated a potential to deliver high levels of impact in their respective areas. These are the prime targets of many venture capitalists, as they boast both high potential impact as well as the evidence to achieve it. However, few early stage startups will have the bandwidth to provide such robust standards of evidence, and we expect deals coming in from group A to be relatively scarce.
Safe bets on the other hand, while able to provide robust evidence for its impact, does not have an impact plan that can reasonably scale. We will still consider investments in this group, as they do have some degree of impact but we will have to look deeper into how closely linked it is to the fund thesis. The ability to administer such standards of evidence is also reflective of the financial ability of the company, which is a positive indicator for the profit dimension of the investment.
We expect most deals to fall under the Potential Impact Stars group where the company has fulfilled at least a level 1 on the Nesta scale, but provides an ambitious and realistic business plan with enormous impact upside. As with all early stage investments, it is understandable that there is not much quantitative evidence of either impact or profitability, which is why we are open to companies which have only achieved level 1 or 2 on the Nesta Scale. An example of a company that falls under this group would be a company with pioneering business models that have the potential to deliver sector level change. Due to the unprecedented nature of their business model, they usually have difficulty gathering evidence of their impact at the early stage. The company could be the first movers in a market with inadequate regulatory frameworks, consumers that are suspicious of the intentions and value of their product and a lack of industry specific talent. As the company ventures down this path, its efforts to de-risk the business pave the way for many future companies to follow, and while its own impact may be limited, the eventual scale of their impact brought about by future companies can be monumental. A prominent firm that has achieved this would be Grameen Bank with their pioneering microfinance model. These kinds of businesses are identified by Omidyar Network as “Market Innovators”14, and due to its high-risk nature, is where a huge funding gap lies. However we do expect them to move up the Nesta scale at an appropriate timeframe with the aid of our investment.
Common deals consist of companies with no significant impact plan, as well as weak evidence to prove they can achieve what little they have planned. We generally do not pursue investments in this group, save in exceptional situations where we are confident that the company is able to generate significant levels of profit.
Phase 2: Planning Impact
In the planning impact phase, the fund needs to devise the metrics which will guide the data gathering process for the firms. In this phase, it is important to ensure that the reporting criteria is not unnecessarily tedious, which would exhaust significant time and manpower resources and could prove to be a distraction from running the business. The fund also has to work on the appropriate reporting period, which can differ from startup to startup depending on the stage that the startup is at. There are two main frameworks which come into play in this phase – the Logframe and the IRIS metrics. It is important to note the distinction in the purposes served by the two frameworks – while Logframe is more applicable to planning impact at a company level, the IRIS metrics are more suited to providing a portfolio level overview of the fund performance.
Portfolio Level Planning – Usage of the IRIS Metrics
As the IRIS catalogue provides a comprehensive list of industry appropriate metrics, it is very helpful in guiding fund managers to come up with their own fund specific set of metrics, modified to suit the investment thesis of the fund. As our fund seeks to address problems faced by the Base of Pyramid in Southeast Asia, we have curated a set of metrics that we have found to provide the most accurate picture of our progress towards our goals. Our selection of cross-sector metrics, or metrics which can be applicable to any industry, is listed below. A mixture of IRIS metrics and Quest designed metrics are included, with metric IDs starting with “Q” denoting a Quest designed metric. The sector specific metrics covering education, healthcare, housing, agriculture, environment and financial services can be found in Annex A. These sectors are selected as we consider them most relevant to our impact mission – solving the problems faced by the Base Of Pyramid population in Southeast Asia. It is our hope that they can serve as a guide for funds with similar goals.
Company Level Planning – Logical Framework Approach
We rely on the Logical Framework Approach to plan impact in a more nuanced manner, at the individual company level. The company is expected to plan out their own logframe (Figure 2), and cover their impact plan at different levels, from the overarching goal, to the immediate purpose, then to the outputs required for the purpose and subsequently the activities required to produce the outputs. This allows the fund to have a more complete picture of the impact objectives, as a list of metrics may not be able to fully capture the level of real impact that is being delivered by the portfolio companies.
Specifically, at the planning impact phase we are looking at the second column “Indicators of Achievement”.
Phase 3: Monitoring Impact
Monitoring impact is a tricky task – we need a clear and standardised reporting framework which ensures accountability to our investors as well as guide us in future investments. Yet at the same time, excessive reliance on a checklist metric system can result in an overly narrow perspective on the performance of the company. To overcome this dilemma, we adopt two tiers of monitoring, one at the company level, another at the Portfolio level. At the company level we use the Logical Framework Approach to arrive at a more in-depth monitoring of the challenges and successes of each company, while at the portfolio level we use the IRIS metrics to provide a standardised overview of the performance of our portfolio.
Portfolio Level Monitoring
The metrics that decide the focus of monitoring has been curated in the previous phase, planning impact, and this phase will involve the collection of data based on those metrics. Typically this is done by providing the portfolio companies with a reporting template based on the metrics. While the cross-sector metrics (Table 1) are applicable across industries, the rest of the metrics are industry specific. Companies are only required to fill in a report on the industries relevant to their company.
In general, very early stage startups can take years to have a significant impact, so the timeline for reporting has to be adjusted as appropriate to the company. This is particularly so for our potential impact stars, which endeavour to bring brand new business models to the market.
As the reporting metrics are standardised, the individual reports can be simply added up to form the Portfolio level report, which can then be analysed to provide an overview of impact performance.
Company Level Monitoring
At the company level, we continue to monitor the impact progress of the company towards the goals, outcomes, and its indicators of achievement as outlined in the Logframe. This provides us with a more precise outlook on the company, covering areas which are unable to be captured by a general checklist. At this phase, it is also important to carefully analyse the sources and means of verification as outlined in the third column of the logframe.
Phase 4: Evaluating Impact
In this final phase of the impact investing process, we attempt to evaluate whether our investment has made a real change – whether the impact would have been achieved without our investment. Such evaluations involve experimental or quasi-experimental methods, which provide stronger justification of evidence of impact, and at the same time moves the company up on the Nesta Scale.
Experimental methods involve the use of Randomised Control Trials (RCTs), whereby a randomised control group, insulated from the effects of our investment is used as a counterfactual to assess the real impact of our investment. Other methods which utilise different counterfactuals are considered quasi-experimental, and are generally used when isolating the conditions for a randomised control group is not feasible, too costly or unethical, such as when random groups of people are denied access to healthcare15.
Both experimental and quasi-experimental methods are costly and manpower intensive – with experimental methods being the costlier of the two. It can seem hard to justify the relevance of this final phase in the impact investing process, especially for early stage startups which forms the focus of our thesis, but we maintain the stance that this phase is necessary, and will be beneficial to the fund in the long run.
Such methods build up our evidence of impact, bringing us higher up on the Nesta Scale, which is the evidence framework that we use to guide our companies’ impact assessment. Due to the high rigor of the experimental and quasi-experimental methods, it allows both ourselves, as well as our investors a greater level of confidence in the good work that we are doing. This would also serve us well in securing future funding to expand our impact.
That being said, investing in early stage startups can sometimes require up to years to see a degree of impact that justifies such a costly evaluation by external assessors. There is no need to rush to this phase, and the company should be allowed space to grow before we arrive here.
Conclusion
This framework has been crafted based on extensive research on existing Impact Measurement and Management (IMM) frameworks, as well as drawing on 9 years of experience in growing along with the startups we invest in. We are grateful for the resources provided by the Harvard Business School, the Global Impact Investing Network, the Omidyar Network as well the Acumen Fund from which we gleaned valuable insights that shaped the development of this framework. This framework is by no means a definitive rulebook for early stage impact investing, but we do hope that it demystifies the whole process, and lowered the barrier to entry for future early stage impact investors.
Citations
- PRI, Impact investing market map. Retrieved May 13, 2019, from https://www.unpri.org/thematic-and-impact-investing/impact-investing-market-map/3537.article
- D.light, About Us. Retrieved February 14, 2020, from https://www.dlight.com/about/
- Ivy So & Alina Staskevicius, Measuring the “impact” in impact investing, Harvard Business School, 2015.
- Nesta, Standards of Evidence for Impact Investing, 2012
- IRIS+ System | Standards, Retrieved from https://iris.thegiin.org/metrics/
- GIIN, A guide for impact investment fund managers. Retrieved from https://thegiin.org/integrating-impact-measurement-and-management
- IRIS, Getting started with IRIS, 2013.
- I. Soh, A. Staskevicius, Measuring the “Impact” in Impact Investing, Harvard Business School, 2015.
- Rachel Bass et al., State of Impact Measurement and Management, GIIN, 2020.
- World Bank, The Logframe Handbook, 2013
- Investment & Performance Board, Project Oracle, 2013. Retrieved from https://www.london.gov.uk/moderngov/documents/s24085/11%20Project%20Oracle%20-%20Cover%20Report.pdf
- R. Puttick, J. Ludlow, Standards of Evidence: An approach that balances the need for evidence with innovation, Nesta, 2013
- R. Puttick, J. Ludlow, Standards of Evidence for Impact Investing, Nesta, 2012
- M. Bannick & P. Goldman, Priming the Pump: The Case for a Sector Based Approach to Impact Investing, Omidyar Network, 2012.
- Ivy So & Alina Staskevicius, Measuring the “impact” in impact investing, Harvard Business School, 2015.
Annexes
1. Sector Specific Metrics (Education)
2. Sector Specific Metrics (Healthcare)
3. Sector Specific Metrics (Housing)
4. Sector Specific Metrics (Agriculture)
5. Sector Specific Metrics (Environment)
6. Sector Specific Metrics (Financial Services)
7. Sector Specific Metrics (Energy)