This site requires Javascript to be turned on. Please enable Javascript and reload the page.

Balanced Value Impact Model

Participatory Methods - Comparative Chart

This page is referenced by:

Glossary of impact methods to use with the BVI Model 114 This page provides a glossary list of useful methods and terms to use in conjunction with the BVI Model. plain 2019-12-19T15:38:16+00:00
Balanced Scorecard approach
The Balanced Scorecard is useful for this purpose as a performance measurement and guidance framework that fully engages with intangible value and strategic change. Robert Kaplan and David Norton originated the Balanced Scorecard as a response to the overwhelmingly financially weighted measures that dominated the annual reports of major corporations (Kaplan and Norton, 1996). As financial measures by their nature lag behind or look backwards at past performance, Kaplan and Norton suggest that such measures are becoming obsolete and do not reflect the full complexity of competitive organisational environments. Thus, they proposed the Balanced Scorecard, which, although initially developed to fit the needs of for-profit corporations, has found a wide range of applications in governmental, non-profit and – especially – memory institutions.
The Balanced Scorecard creates a holistic set of viewpoints on the central strategic focal point in order to deliver a framework of performance measurement. The use of perspectives is very beneficial for the richly context-driven work of impact assessment. By enabling four core ways of looking at and assessing any strategic opportunity or activity, the Balanced Scorecard provides a much-desired control and nuance to measurement.
The four perspectives in the Balanced Scorecard are financial, customer, internal business processes, and learning and growth. These perspectives are intended to balance each other, including the financial and non-financial; inward-looking processes and outward-facing customer satisfaction; current and future plans; or drivers towards performance and measures of success. These are usually represented in a graphic or tabular format to allow each to include sub-sections of objectives, means of measurement, targets and initiatives.
The Balanced Scorecard has much to offer, but there are problems in applying it without modification to the BVI Model. The language of the Balanced Scorecard is too corporate and either mystifying or excluding for those working in memory institutions. The four perspectives of the Scorecard are useful, but they provide focal points for measurement that do not align closely enough with practice in the cultural, media, academic or heritage sectors. Concepts of digital strategy and digital innovation are introduced in the BVI Model to allow for the framing of thinking provided by the Strategic Perspectives. The BVI Model therefore adopts aspects of the Balanced Scorecard framework, but with adaptations to align closely to the needs and expectations of practitioners from memory institutions.
Thus, to translate the Balanced Scorecard into the BVI Model,
- ‘financial’ becomes an economic impact;
- ‘customer’ becomes a social impact;
- ‘internal business processes’ becomes operational impacts;
- ‘learning and growth’ becomes innovation impact.
The Balanced Value Impact Model (BVI Model)
The BVI Model is a cohesive model developed by Professor Simon Tanner that will provide a definition and model of impact for any memory institution with a pragmatic implementable framework for delivery (BVI Framework). It is specially designed for digital resources and memory institutions. The BVI Model is a measurement model focused on identifying a change in a community arising from the existence of digital resources that are of proven value to the community.
The conceptual model underlying the BVI Model follows a process that stresses the importance of distinguishing between inputs, actions, the output and the outcomes. The BVI Model encourages a continuous assessment to look beyond the immediately measurable ‘output’ towards the demonstrable outcome, which leads to defining the real impact on people.
The BVI Model is a five-stage iterative process:
- Stage 1: Set the context
- Stage 2: Design the framework
- Stage 3: Implement the framework
- Stage 4: Narrate the outcomes and results
- Stage 5: Review and respond.
The BVI Framework
Use of the BVI Model produces an impact framework as an immediate output, known as the BVI Framework. This Framework structures and guides the process to manage the gathering of impact evidence and control a process that usually has many moving parts occurring over differing timescales, with possibly different teams, resources or points of focus. The Framework itself will mainly be of use to those coordinating the activity to track activity and measures against impact indicators and objectives.
The advantage of a framework nested within an overarching model is that it allows flexibility for those tasked with its practical implementation. As long as the conceptual model is recognised, then actual delivery of any one aspect is not prescribed or required. Anyone using the BVI Model should feel free to rename or revise the functional parts in the Framework to suit local needs or to aid cognition or acceptance in their organisation or community.
The model-to-framework approach also accounts for the range of fuzzy elements within an assessment of impact. A high-level conceptual model allows for the thinking, context and perception to be represented, while the Framework allows the products of this thinking to be controlled, contained and then utilised.
Choice modelling
Choice modelling is a family of survey-based methods to derive a stated preference from stakeholders. Surveys ask respondents to make a series of clear choices of their preferences when presented with various alternate descriptions of the object of valuation. These are offered as a set of attributions to be selected and levels of preference assigned to those choices. The levels and attributes are combined, using statistical design, into alternate scenarios which respondents can rank or choose. Discrete choice modelling provides an analytical framework to examine individuals' choices and their drivers.
Choice modelling is a highly sophisticated combination of economics, mathematics and statistics. More tools are becoming available, but choice models remain complex to design and costly to apply. Choice modelling is thought to be a superior predictor of human behaviour and to make forecasts of future consumer demand. For instance, the outputs from choice modelling are routinely used in cost-benefit analyses of major new infrastructure developments. It also provides a valuable method for measuring the values people place on non-market goods and services.
See the Choice Modelling Centre at the University of Leeds https://cmc.leeds.ac.uk/
Content analysis
Content analysis is a tool that does not require the direct collection of data from people. Content analysis refers to a general set of techniques useful for analysing and understanding collections of recorded information in texts, media, or physical items. This includes Internet content and social media.
Content analysis extracts the key themes of large bodies of online documents, derived from the presence of certain words or concepts within texts or sets of texts. Collecting information from these bodies of text is a form of data mining. The tools needed for content analysis operate at two levels: the tools needed to find the content, and the tools needed to analyse the content. The methods are a mixture of linguistics and statistics, and there are free tools available as well as commercial ones. To conduct a content analysis text is coded and categorised on a variety of levels: word, word sense, phrase, sentence, or theme. Relational analysis (or semantic analysis) can explore the connections between concepts or themes or categories. Conceptual analysis quantifies or counts the presence of a concept or category.
See the Iowa State University Library research method guide to content analysis https://instr.iastate.libguides.com/c.php?g=49332&p=318069
Consumer surplus
When there is a difference between the price that a stakeholder pays in the market and the value that they place on the product/service, then the concept of consumer surplus becomes a useful one to investigate. Consumer surplus is an economic measure of the welfare that people gain from consuming goods and services above a purely financial rate as set by the market.
The definition of consumer surplus is the difference between the total amount that stakeholders are willing and able to pay for a product, good or service and the total amount that they do pay (i.e. the market price). This is the value that consumers place on accessing resources over and above any costs that may be incurred to obtain them.
In simple terms, if you would be willing to pay £1.60 for a cup of tea but can buy it for £0.90 – the consumer surplus for that person is £0.70. For example, a study might show that for every $1 of public money spent, users received more than $4 of direct benefit in consumer surplus terms.
See Case study 3.5: The value of the British Library in Delivering Impact with Digital Resources.
See also: Tessler, A. (2013) Economic Valuation of the British Library. Available at: https://www.oxfordeconomics.com/my-oxford/projects/245662
Contingent valuation
Contingent valuation assesses the values associated with the user and non-user’s willingness to pay to continue accessing a service, or the willingness to accept if the service were to cease. Contingent valuation is based on understanding what people would be willing to pay for a particular good or service, for example library provision or visiting a museum exhibition. The techniques are based on constructing a hypothetical market for the non-market goods to be valued and then attaching prices to them by asking people directly about their willingness to pay or willingness to accept compensation for it.
This might prove to be a useful method for the cultural sector, if it is possible for people to estimate the ‘worth’ of such intangible benefits. This method relies heavily on interview and if asking a person to estimate worth in monetary terms will require well developed and detailed questions to avoid gaining meaningless or invalid answers. It is likely that to run a contingent valuation will require external expertise and may be costly.
See Case study 3.5: The value of the British Library in Delivering Impact with Digital Resources.
See also: Tessler, A. (2013) Economic Valuation of the British Library. Available at: https://www.oxfordeconomics.com/my-oxford/projects/245662

Cost benefit analysis
A cost-benefit analysis is a process to sum the benefits of a strategy or action and then subtract the costs associated with taking that action. It is possible to build models to assign a fiscal value on intangible items.
A cost-benefit analysis (CBA) begins by compiling a comprehensive list of all costs and benefits associated with the proposed action or strategic decision. A monetary value is applied to all the cost-benefit list items. Lastly, the results of the aggregate costs and benefits are compared quantitatively to determine if the benefits outweigh the costs. CBA is used as a mechanism to demonstrate that the benefits are greater than the costs and thus the proposal is economically advantageous.

Critical incident technique
A qualitative research method that aims to collect data from users based on the observation and recounting of critical incidents. Individuals are asked to describe behaviours or incidents that are critical in that they contribute to the success or the failure of an endeavour. A critical incident can have a positive or negative connotation. The observations drawn from the incidents recounted are kept track of and can be used to solve practical problems and to develop a set of general principles.
A critical incident is a separable event that significantly influences an activity or task, and thus has an impact on the person’s attitudes, success or failure of achievement. Despite the name critical incidents do not have to be dramatic or unusual just separable from other events. In digital terms the action of visiting the online store is separable from the overall institution website browsing experience and could be observed as a critical incident.

See also:
Zanibellato, F., Rosin, U., & Casarin, F. (2018). How the attributes of a museum experience influence electronic word-of-mouth valence: An analysis of online museum reviews. International Journal of Arts Management, 21(1), 76-90. https://search.proquest.com/docview/2151120106?

Digital ecosystem and digital resources
A digital ecosystem is the set of interdependent relationships among the resources, technologies, organisation hosting, creators and consumers, mapped and clearly described to enunciate the ecosystem of a digital resource. Without understanding the digital ecosystem, it is hard to contemplate the full capabilities of the digital resource and its relationship to stakeholders.
Contemplating the digital ecosystem enables a deeper understanding of the driving forces that have brought the organisation to this point and the challenges that are intrinsic to the context of the organisation. Every organisation has a unique digital ecosystem – a set of attributes that are, so to speak, ‘baked into the cake’. Some organisations will have enterprise-grade systems, some outsource extensively and others have the in-house skills to allow them to rely on open source solutions. These differences will be formative in considering what is measured and how to measure it.
To understand the context of a digital resource the BVI Model scopes it against a set of parameters. These parameters consider the set of interdependent relationships among the resources, content, stakeholders, technologies, infrastructures, organisation hosting, legal and payment structures, creators and consumers. The BVI Model provides an opportunity to map and describe the ecosystem of the digital resource to provide a baseline for measurement and to establish some accepted ground truths.
Parameters to scope what a digital resource is, include the following.
- There is a defined resource that is made up of a describable, cohesive set of primary and secondary materials, services, products and activities.
- The resource is accessed primarily through a digital platform (web, mobile, virtual reality, exhibition console or other means).
- The nature of the content within the resource is digital – either achieved through digitisation or as born-digital content.
- There is a definable group of users that the resource is intended to reach by digital means.
- The resource does not have to stand alone; it could be part of a broader set of activities, products or services.
DPSIR: Driving forces, Pressures, States, Impacts and Responses
The DPSIR (drivers, pressures, state, impact and responses) environmental impact assessment model strongly influences the BVI Model. As articulated by Danish scientist Peter Kristensen the DPSIR model establishes causal relationships across drivers, pressures, state, impact and responses that in turn structure the interactions between society and a defined ecosystem. The DPSIR model itself is an adaptation of a broader framework of relationships (pressure-state-response) adopted by the Organisation for Economic Cooperation and Development, which can be seen applied in the Better Life Index (www.oecdbetterlifeindex.org/topics/life-satisfaction).
DPSIR assumes a causal chain starting with D:driving forces (BVI Model: Value Lenses, stakeholders) affected through P:pressures (BVI Model: Economic, Social, Innovation, Operational Strategic Perspectives) related to an S:state (BVI Model: ecosystem) that records an I:impact (BVI Model: data on changes recorded), eventually leading to strategic and evidence-based decisions R:responses (BVI Model: review and respond).
Economic Impact Assessment
Economic impact assessment estimates changes in employment, income or levels of business activity that may result from a proposed project or intervention. There are an enormous range of mechanisms to measure these outcomes. It is helpful to consider direct, indirect and induced economic impacts. Economic impacts accrue when it is possible to show that there is a financial return on investment that is specific and directly or indirectly measurable. The cultural field has used this approach successfully.
Economic impact can provide a focus to consider the wealth or level of economic activity in a given geographic area or zone of influence. It is usually feasible to identify baselines and significant indicators to measure improvement in the economic well-being of an area, such as via increased:
- business output
- value added
- wealth (including property values)
- personal income (including wages) or
- jobs.
See Case Study 3.6 The economic impact of Canadian libraries in Delivering Impact with Digital Resources.
See also: The Economic and Community Impacts of the Museum of Fine Arts, Boston
Ethnographic research
Ethnographic research is a qualitative research method that is both participatory and observational. Traditionally, it is carried out ‘in the field’. That is, the researcher studies a community by being both a participant in the life of that community and an observer of it. Data collection is often done through participant observation, interviews, questionnaires, etc.
In the past, ethnographic research was more often concerned with remote cultures, but its reach has extended to studying cultural practices closer to home. There is now a considerable body of work by ethnographers and anthropologists on the effects of digital media on culture and society (see Coleman 2010 for a summary of these). There is also a new subject area, digital ethnography, which is the study of online communities and human-technology interactions through the use of qualitative research methods.
Ethnographers study changes brought about by many kinds of interventions and developments in all societies and ethnographic methods can be helpful in assessing change in a community through the introduction of a digital resource.
See: E. Gabriella Coleman (2010), Ethnographic Approaches to Digital Media. Annual Review of Anthropology, 39, 487–505.
Focus Groups
Focus groups are a useful method for obtaining detailed information about a web resource from stakeholders. A focus group is essentially a conversation around a topic or a resource. Focus groups can be mediated and moderated in different ways, generally using a mixture of structured and unstructured methods of eliciting responses from participants. They may begin with a clear set of questions to be answered, but may deviate from these as participants ask questions, respond to points raised, relate anecdotes and experiences, and provoke debate. The aim is to gather qualitative data from these interactions, including data that can be hard to get at in conventional interviews or other data gathering methods.
If an organization has created a resource, focus groups can be a useful way to find out how it is being used, how the target audience responds to it, what problems they encounter, how they find the interface, etc. The discussion is moderated, and usually focus groups comprise 6-12 participants, rarely more. Focus groups can be conducted online, but more usually they are face-to-face encounters.
Focus groups are a useful tool for finding out the specific benefits gained by a representative group, so as to indicate a broader beneficial group. Leading a focus group needs high skill levels to gain a meaningful outcome and rigorous identification and selection of representative participants. Focus groups work very well for social and cultural impacts, especially for Community values.

See: Natasha Mack, Cynthia Woodsong, Kathleen M. MacQueen, Greg Guest, and Emily Namey, “Focus Groups” in Qualitative Research Methods: A Data Collector’s Field Guide (Family Health International, 2005): 51-82.
Impact assessment
The International Association for Impact Assessment defines impact assessment as:
the process of identifying the future consequences of a current or proposed action. Impact assessment is a generic term that can mean either an integrated approach or the composite/totality of all forms of impact assessment such as environmental impact assessment (EIA), social impact assessment (SIA), health impact assessment (HIA), etc.
International Association for Impact Assessment
Impact is a broadly used term with multiple different contexts. These are described in Chapter 1 of Delivering Impact with Digital Resources.
Impact as defined in the BVI Model
The BVI Model defines impact as the measurable outcomes arising from the existence of a digital resource that demonstrate a change in the life or life opportunities of the community.
Indicators
An indicator is a piece of information that indicates something useful. The essential thing to bear in mind in the selection and use of indicators is that they are a measure of a tendency in certain conditions, a measure of a state of being, and never an absolute value. Thus, indicators provide clues to answering the questions posited by the objectives within each Perspective–Value pairing in the BVI Model.
There are many possible measures, but some are not convincing enough, while others may be tangential to the purpose of measurement. The digital domain can be a challenging area in which to find suitable indicators. Impact indicators generally should focus on showing a change for those in the community that the resource serves. Change can be on many dimensions, usually defined as behavioural, knowledge based, competence based or attitudes. If people do things differently, if they know more, are better able to do something, or if they think or feel differently about it, then the impact can be claimed.
See also SMART indicators.

Interviews
An interview is a conversation in which the interviewer questions the interviewee in order to gain information and understanding on a defined topic. Interviews can be formal or informal, structured or unstructured. They can be conducted one-to-one or in groups, face to face or by telephone or online. Interviews are time-intensive for the researcher, who may need to carry out a number of one-to-one interviews and would usually record and transcribe the contents. They permit the gathering of a wide range of qualitative data, and they can provide information about people’s motivations, feelings, attitudes, and what they remember. To be most useful, interviews should be well-structured in advance, but should allow the discussion to be relatively free to get the best out of the subject.
The structured interview holds a kind of halfway position between surveys and focus groups, as it involves a one-to-one interview where the questions are pre-set in a survey style. The interview can range beyond the pre-set questions. All questions must be asked to allow for comparison of results, but the order can vary. While they are time consuming, they are an especially effective method for gaining opinions and providing comparable data from a range of interviewees, especially experts, policy makers, decision makers or funders. These can be most useful to Education and Inheritance values. The data collected is qualitative but may also include some quantitative information.
Life satisfaction assessment
According to the OECD Better Life Index, ‘life satisfaction measures how people evaluate their life as a whole rather than their current feelings. It captures a reflective assessment of which life circumstances and conditions are important for subjective well-being’. This is difficult to measure and highly subjective, however governments are increasingly looking at life satisfaction alongside economic factors to monitor well-being and to drive policy.
See the Organisation for Economic Co-operation and Development (OECD), Better Life Index
Logic models
A logic model links outcomes with activities/processes and the theoretical assumptions/principles of the activity. The model facilitates thinking, planning, and communications about objectives and actual accomplishments. A logic model is a systematic and visual way of presenting and sharing an understanding of the relationships among the resources operating a programme, the planned activities, and the anticipated changes or result.
The W.K. Kellogg Foundation Logic Model Development Guide provides a nice overview of the transition from data gathering, through outputs to outcomes and impact.
The Logic Model example provides a neat way to visualise the transition from activity, through outputs and data gathering, to outcomes and impact. Outcomes are the specific changes and consequences evaluated, such as behaviours, knowledge, skills, status, wealth, well-being or effectiveness, to mention a few instances. Impacts are the fundamental changes that occur because the outcomes demonstrate a set of benefits to a defined group in a specific timeframe. Such impacts can be intended or unintended and, most importantly, may be either positive or negative to some or all of the stakeholders.
Longitudinal study
Longitudinal studies study change over extended periods in the lives of individual people. They can also track changes in organisations and institutions. In a typical longitudinal study, the focus of observation is the same (or a comparable) group of people over an extended period. Data is collected at the outset of the study, then periodically throughout the study. Some longitudinal studies can last for several decades. The benefit is that researchers can look at changes over time, which is an essential concept within impact assessment. The downside is that such studies require significant investments, and it is hard to maintain the same people in the study over extended periods.
There are three main types of longitudinal study:
- Panel Study: this involves sampling a cross-section of individuals.
- Cohort Study: Involves selecting a group based on a specific event such as birth date, geographic location or historical experience.
- Retrospective Study: Involves looking to the past by looking at historical information, such as medical records.
See the UK Longitudinal Studies Centre

The UK Data Service is a good source of longitudinal data that may be useful. Available here are studies such as the Community Life Survey tracking the trends and developments across areas key to encouraging social action and empowering communities.
https://www.ukdataservice.ac.uk/

The Mass Observation Archive specialises in material about everyday life in Britain. It contains papers generated by the original Mass Observation social research organisation (1937 to early 1950s), and newer material collected continuously since 1981 (Mass Observation Project). http://www.massobs.org.uk/

Memory institutions
In the BVI Model, the use of ‘memory institutions’ is a collective phrase for libraries, museums and archives. In using the phrase ‘memory institution’, this assumes a common aspiration across multiple sectors in preserving, organising and making available the cultural and intellectual records of their societies. It also reflects the confluence with the growth in digital.
The BVI Model uses the phrase as a convenience to refer collectively to archives, museums and libraries but does not assume primacy in their role as ‘memory institutions’. As Robinson states, ‘a wider variety of organisations, such as schools, universities, media corporations, government or religious bodies could also legitimately be ascribed this title’ (Robinson, 2012). The BVI Model’s conception of memory institutions references and includes this wider definition. For instance, these organisations could be included: the BBC (British Broadcasting Corporation); US Public Broadcasting, as epitomised by companies like WGBH in Boston; a university press; or the Wayback Machine and the Internet Archive.

Multi-criteria analysis
A Multi-Criteria Analysis (MCA) assesses a range of monetary and non-monetary benefits and impacts. In addition to monetary impacts, MCA measures variable such as material costs, time savings, sustainability as well as social and environmental impacts. It is often used as an alternative to cost-benefit analysis or stated preference and revealed preference techniques. MCA is usually used by decision-makes when weighing up the possible impacts of a policy. In short, the MCA framework permits the integration of monetary, quantitative and qualitative data.
To apply MCA, a performance matrix (or consequence table) is constructed, with each row representing a policy option, and various objectives and criteria in the columns. A number of scoring methods are used: these can be numerical scales, colour coding, high/medium/low assessments. The decision-makers decide the criteria for the performance matrix, but it is also recommended that the views of broader stakeholder groups be solicited in the process.

Department for Communities and Local Government (2009), Multi-criteria analysis: a manual. Communities and Local Government Publications, London. http://www.communities.gov.uk/documents/corporate/pdf/1132618.pdf

Participatory approaches
Participatory approaches stress the importance of taking a defined “public” people’s perspectives into account and giving them a greater say in planning and of an evaluation process. The “public” can be average local people, the specific stakeholders of a particular project or activity, NGO’s and even members of government agencies or commercial entities. All stakeholders decide together how to measure results and what actions should follow once this information has been collected and analysed.
There are a number of participatory methods as illustrated in this comparative chart from the Participatory Methods Toolkit.
Participatory Methods Toolkit - A practitioner’s manual
http://archive.unu.edu/hq/library/Collection/PDF_files/CRIS/PMT.pdf

Public value assessment and framework
Public value assessment is a performance measurement tool for quality assurance of public agencies within he UK. It has three aspects to it: delivering actual services; achieving social outcomes; maintaining trust and legitimacy of the agency. It might prove to be a useful method for cultural organizations as it defines the activities that are required to turn public money in to policy outcomes. The criteria created can be used to assess the extent to which activities are taking place and thus the likelihood that value is being maximised.
The main criteria that contribute to public value are grouped into four sections or ‘pillars’ that structure the framework:
- pillar one: pursuing goals focuses on what overarching goals the public body is aiming to achieve and how it is monitoring the delivery of these
- pillar two: managing inputs tests the public body’s basic financial management
- pillar three: engaging citizens and users highlights the need to convince taxpayers of the value being delivered by spending and importance of engaging service users
- pillar four: developing system capacity emphasises the long-term sustainability of the system and the importance of stewardship
Each pillar is then considered from 13 areas across the framework and within these areas there are a series of headline questions (35 in total) designed to explore a specific aspect of performance.

See the HM Treasury guidance document: Public Value Framework and supplementary guidance - A practical tool for understanding how well public money is turned in to policy outcomes. March 2019.
https://www.gov.uk/government/publications/public-value-framework-and-supplementary-guidance

Quality Adjusted Life Years (QALYs)
QALYs are a specific mechanism, developed within health economics, to understand the cost-effectiveness of policy options, usually based on medical interventions. NICE, the National Institute for Health and Care Excellence, defines QALYs as:
A measure of the state of health of a person or group in which the benefits, in terms of length of life, are adjusted to reflect the quality of life. One QALY is equal to 1 year of life in perfect health.
QALYs are calculated by estimating the years of life remaining for a patient following a particular treatment or intervention and weighting each year with a quality-of-life score (on a 0 to 1 scale). It is often measured in terms of the person’s ability to carry out the activities of daily life, and freedom from pain and mental disturbance.

NICE glossary

Referrer analytics
Referrer analytics are methods for identifying third-party resources from which traffic to a web site originates. Referrals may indicate that a third party considers a web site to be relevant to their user community. Referrer analytics may prove useful for identifying use of a web site and makes use of several webometric methods, including web log analysis and link analysis.
Referrer analysis and link analysis give a snapshot of who is linking to the site and from where, and using the results of this it is possible to better understand the audience for a site. It is useful to consider not only how much total traffic other sites are driving but also how effectively that traffic is adding to engagement. For instance, evaluate duration to see how long users are spending on the site and Pages/Session for how much content they’re viewing.

Revealed preference methods
These methods are based on an analysis of what people actually do in real-world situations rather than them making choices or valuations in hypothetical situations. Some resources/products (even if not traded) are related in consumption to goods that are traded. This relationship is known as revealed preference and can be used to estimate use-values.
The two main categories of revealed preference methods are hedonic pricing and travel cost.
Hedonic pricing is rarely used in the cultural sector. It is used to determine the extent that environmental or ecosystem factors affect the price of a good - usually a home – via variables such as the nature of the property, its location and inhabitants.
Travel cost estimates the amount of time that people are prepared to travel to consume goods or services. The travel cost method is typically used to value the benefits of recreation trips and sites, among other things. It begins by assuming that the travel and time cost is the implicit price of a recreation trip. The benefit of recreation sites is equal to the consumer surplus. The benefit of recreation trips is equal to the consumer surplus for the site divided by the number of trips.
How travel cost methods work with digital is to make a comparison between travel costs previously incurred versus additional services availed digitally that remove the need to travel.
SMART indicators
SMART is an acronym for Specific, Measurable, Attainable, Relevant and Timebound much used in project planning and performance management.
Indicators are the points of evidence to demonstrate that the conditions sought in objectives are met. All indicators should adhere to the SMART criteria: specific, measurable, attainable, relevant, timebound. See also Indicators.
The following are essential considerations in establishing useful indicators for the BVI Framework.
1. Ensure that the indicator will, as directly as feasible, provide evidence for the objective set in the BVI Framework.
2. Focus the indicator on measuring change. It may be necessary to define a benchmark or a baseline from which change is measured.
3. Choose as few indicators as possible to achieve the objective. For the BVI Framework, no more than two per Perspective–Value pairing is desirable.
4. Establish SMART indicators, considering whether the data:
  - is available or attainable
  - comes from a credible source
  - is large enough
  - is gatherable in a suitable timeframe.
5. Do not over commit to quantitative indicators. Remember that a mixed model of quantitative and qualitative approaches generally provides more powerfully persuasive evidence.
6. Test the validity of the indicator. Consider the question: if all these factors were true, would that satisfy that the measure of success is achieved?
Social Return on Investment (SROI)
Social enterprises are required to demonstrate the social and economic value they generate. Social return on investment (SROI) is a performance measurement tool currently being encouraged to capture this impact. SROI as an analytic tool for measuring and accounting for this broader concept of value. It incorporates social, environmental and economic costs and benefits into decision making, providing a fuller picture of how value is created or destroyed.
SROI provides a means of giving a monetary figure to social and cultural value deriving from an investment. For instance, the value created by a training programme for ex-offenders might reveal that for every £1 invested, £10.50 of social value was created.
SROI is a framework based on social generally accepted accounting principles that can be used to help manage and understand the social, economic and environmental outcomes created by an activity or organisation.

SROI is based on seven principles:
1. Involve stakeholders: Understand the way in which the organisation creates change through a dialogue with stakeholders
2. Understand what changes: Acknowledge and articulate all the values, objectives and stakeholders of the organisation before agreeing which aspects of the organisation are to be included in the scope; and determine what must be included in the account in order that stakeholders can make reasonable decisions
3. Value the things that matter: Use financial proxies for indicators in order to include the values of those excluded from markets in same terms as used in markets
4. Only include what is material: Articulate clearly how activities create change and evaluate this through the evidence gathered
5. Do not over-claim: Make comparisons of performance and impact using appropriate benchmarks, targets and external standards.
6. Be transparent: Demonstrate the basis on which the findings may be considered accurate and honest; and showing that they will be reported to and discussed with stakeholders
7. Verify the result: Ensure appropriate independent verification of the account
See also Social Value UK - the national network for social impact and social value.

Stakeholders
In the BVI Model, stakeholders are defined as a person, group, community or organisation who affects or can be affected by the ecosystem of the digital resource assessed.
Stakeholder discovery and categorisation is an important part of the BVI Model. This step establishs as complete a categorised list of stakeholders as feasible. In later stages, the information gathered here will be further segmented and analysed to identify different groupings for investigation specifically in response to the needs of the impact assessment.
Actions to take in establishing and identifying stakeholders include the following.
1. List all primary stakeholders.
2. List all secondary stakeholders.
3. Identify and list all potential supporters and opponents.
4. Identify the interests of vulnerable or minority groups.
5. List all new primary or secondary stakeholders likely to emerge as the impact assessment progresses.
Working from the list of stakeholders previously identified, in their primary and secondary groupings these can now be segmented into defined groups that can serve impact assessment purposes through a stakeholder analysis. Identify by this process the major stakeholders who can have a significant influence on or will be essential to the perceived success of the digital resource. Note that ‘essential’ in this sense refers to those stakeholders whose needs and interests are the priority of the digital resource. It could refer to direct users of the resource or direct beneficiaries of use of the resource. Refer to these as the core beneficial stakeholders.

Strategic Perspectives in the BVI Model
Strategic Perspectives provide for a holistic application of four perspectives to contextualise organisational strategy. Mapping current strategies or actions against the Strategic Perspectives ensures that the outcomes can be clearly narrated. The understanding gained enables the organisation to review and respond to the results.
The BVI Model always applies the following four Strategic Perspectives, defined as:
- Social impacts: stakeholders and wider society have been affected and changed in a beneficial fashion;
- Economic impacts: the activity is demonstrating economic benefits to society, stakeholders or the organisation;
- Innovation impacts: the digital resource represents or enables innovation which is supporting the social, economic or operational benefits accrued;
- Operational impacts: the organisation creating/delivering digital resources has been benefited within its internal processes by the innovation demonstrated.
The key benefits of this approach are to align, clarify and gain consensus about strategy in a simple framing that works in many contexts. It further allows for a clear narrative of benefit to be built into the process from the beginning, with two outward-facing perspectives (Social and Economic) and two more internalised perspectives (Innovation and Operational). A holistic strategic context narrates the outcomes and results powerfully as ‘our digital resources are benefitting society via these social impacts, leading to economic benefits based in our innovation; while becoming a more effective and efficient organisation’.

SWOT analysis
SWOT is an acronym for Strengths, Weaknesses, Opportunities and Threats. A SWOT analysis is a useful planning tool used near the beginning of a project or process. It is relatively simple to carry out, but can offer some powerful insights into a new venture. A SWOT analysis is often a significant part of a situation analysis.

Strengths are those attributes or activities related to the digital resource that are expected to perform better than most in the organisation’s environment. At this stage, building the selection of strengths is based on the organisation’s normal, expected requirements for success.

Weaknesses are those attributes or activities related to the digital resource that could be improved to increase the probability of success. Identify the predictable weaknesses that would affect the probability of ‘success’ for the digital resource. These may be known if the assessment starts after the resource is established, or may be predictable in the context of an upcoming implementation.

Opportunities are those externally focused attributes or activities that may affect the future success of the endeavour. These are likely to be an area of great interest for impact assessment and are often likely to be the product of underlying trends or conditions developing or appearing within the ecosystem for that digital resource. Two suggested activities to help with this process include maintaining a continuous review of the literature and benchmarking inside and outside the organisation to identify and evaluate potential opportunities.

Threats are those attributes or activities related to the digital resource that are obstacles to the accomplishment of goals. What will or is getting in the way of success? Threats differ from weaknesses, as they may be beyond the scope of an easy solution or change in reaction to the threat, which is usually external.

Theory of Change
Theory of Change (TOC) is an outcomes-based, participatory method for planning the desired impact. TOC defines all building blocks required to bring about a given long-term goal. The TOC process hinges upon defining all of the necessary and sufficient conditions required to bring about a given long term impact. TOC uses backwards mapping requiring planners to think in steps leading backwards from the impact goal to the outcomes and then the outputs and near-term changes that would be required to cause the desired change. This set of connected building blocks–interchangeably referred to as outcomes, results, accomplishments, or preconditions is depicted on a map known as a “pathway of change” or a “change framework”, which is a graphic representation of the change process.

See also the Center for Theory of Change.

Triple Bottom Line
The triple bottom line (abbreviated as TBL or 3BL, and also known as people, planet, profit) captures an expanded spectrum of values and criteria for measuring organisational (and societal) success: economic, ecological, and social. It has been used extensively by the United Nations as a standard for public sector full cost accounting and measures of human capital.
According to TBL theory, companies should be working simultaneously on these three bottom lines:
- Profit: The traditional measure of corporate profit—the profit and loss account.
- People: Measures how socially responsible an organisation has been throughout its operations.
- The Planet: Measures how environmentally responsible a corporation has been.
By considering these interlocking factors, triple-bottom-line reporting can support a corporation's sustainability and social responsibility goals.
Triple Task method
The Triple Task (TT) is a methodology developed by Simon Bell and Stephen Morse for systemic action research. Bell and Morse describe it as arising from the authors' previous work with the soft systems approach, the Imagine method for sustainable development assessment and action research in a variety of global locations. The Triple Task is a unique form of participatory action research and stakeholder analysis. TT provides a means for groups to engage together in purposive work and, at the same time, for facilitators to understand how the dynamic of the group influences the group's output. TT is an approach for gaining a deeper understanding of how groups of stakeholders work, and therefore providing a means to understand how multistakeholder groups plan and think collaboratively. Not only does it attempt to arrive at answers to research questions but also tries to understand what factors may have been at play in arriving at those answers.

S. Bell and S. Morse, (2012) Sustainability Indicators: Measuring the Immeasurable? 2nd Edition, Earthscan, 2012.

Value Lenses
In the BVI Model, five Value Lenses focus attention on specific assessment activities reflecting core values measured for their impact. Assign one or more Value Lenses as drivers to any measurement made. The five Value Lenses are:
- Utility value: the value expressed and the benefits directly gained by people through active use of the digital resource now or sometime in the future;
- Education value: the value expressed and the benefits directly gained by people from their own or others’ ability to learn and gain knowledge (formally or informally) from a digital resource;
- Community value: the value expressed and the benefits directly gained by people from the experience of being part of a community engaging with or afforded by, a digital resource;
- Existence and/or Prestige value: the value expressed and the benefits people derive from knowing that a digital resource exists and is cherished by a community, regardless of use or non-use of the resource;
- Inheritance/Legacy value: the value expressed and the benefits derived by people from the ability to pass forward or receive digital resources between generations and communities, such as the satisfaction that their descendants and other members of the community will in the future be able to enjoy a digital resource if they so choose.
Web analytics
Web analytics is the measurement, collection, analysis and reporting of Internet data to understand and optimise Web usage. Analytics are statistics about the traffic and visitors attracted to the website and are widely used to assess its success.
Many sorts of data are gathered in analytics, and there are many tools (paid-for and free) that can analyse and display usage statistics. The analytic data can be used for purposes such as audience research, assessment of return on investment, assessment of whether the site is working properly, among others. Information can be collected by analysing web server log files, by page tagging, by using geolocation software to track where users originate, and by click analysis. These methods can give information on:
- the number of visits
- the number of page views
- how long users spend on a site
- most viewed pages
- other sites referring traffic to the site
- where users originate
- number of clicks
A common method of analyzing web site traffic is using log file analysis. Log files are text files maintained by web servers to record downloads of web pages and other files.
Participatory approaches Participatory approaches stress the importance of taking a defined “public” people’s perspectives into account and giving them a greater say in planning and of an evaluation process. The “public” can be average local people, the specific stakeholders of a particular project or activity, NGO’s and even members of government agencies or commercial entities. All stakeholders decide together how to measure results and what actions should follow once this information has been collected and analysed.

There are a number of participatory methods as illustrated in this comparative chart from the Participatory Methods Toolkit.

Participatory Methods Toolkit - A practitioner’s manual
http://archive.unu.edu/hq/library/Collection/PDF_files/CRIS/PMT.pdf