Measuring benefits – the JISC impact calculator

I spoke last week at Cimtech’s conference on using the impact calculator tool.  No point posting the slides, as there were only 3 – I took a risk and did a live demo (luckily it paid off) in place of death by powerpoint.

These are some personal reflections on the impact calculator – the formal project documentation and evaluation will be available from the JISC infoNet website – probably at but I’ll post the correct link when available.   JISC infoNet also plan to publish a study

The calculator has been useful in providing headlines about tangible/monetary benefits.  Whilst it’s applicable to projects of any scale, I think it’s probably most useful for larger projects.  Our pilot limited the applicability of the tool because of the restricted scope of the process change for which the project had a mandate, and the relatively small size of the specific IT system in the wider systems to which it belongs.  It was only feasible to develop and gather data for a limited number of metrics, which showed an overall cost in monetary terms with very little in the way of tangible benefits .  However in the case of this particular change project, senior management had decided that the intangible benefits outweighed the few tangible benefits shown.

I said in the final report

Despite the limited applicability of the impact calculator to the pilot project, the experience suggests the following lessons for information and records managers responsible for implementing and advocating change to processes in their organisations:

  1. deconstructing a process as a whole, whilst potentially more time-consuming and requiring greater effort up-front, may better justify change in the long run;
  2. the impact calculator is useful tool for considering the tangible benefits of a process change where metrics are readily identifiable and can be expressed effectively;
  3. potential detailed metrics should be considered before change implementation, and used as part of the change requirements discussion;
  4. there will be occasions when intangible benefits are accepted as the driver for a process change, although having a measure of the tangible benefits assists transparency and evidence-based decision-making;
  5. the timing and planning of required systems development can have a significant impact on the cost; where there are few tangible benefits or small monetary benefit the intangible benefits should be considered carefully before proceeding.

Finally, as with all projects, this project reinforces the need to plan and monitor the progress of the project itself, and to be realistic about scope and capacity in the context of ongoing service delivery.

For those interested in the detail of what the project was about, the University has a wider, long-term project to implement an electronic document and records management system (EDRMS), Wisdom.   During the academic year 2009-10 the focus for the EDRMS implementation was planned to be on records relating to the quality of modules.  This work has been carried out as planned, with additional focus on records relating to the quality of programmes (courses) as required by senior management.

This additional work on programmes was encompassed by the JISC-funded project which implemented a change to the publication and dissemination of programme specification documents (following the completion of course validation processes).  The intention of this change was to

  • provide “a single point of truth” about the University’s programmes
  • enable validated changes to programme specifications to be disseminated automatically;
  • retain information about programmes consistently across the University.

The project, which ran from January – June 2010 in line with the requirements from JISC infoNet, had four main phases of work outside project initiation and closure:

  1. developing the EDRMS fileplan and populating it with the programme specifications;
  2. developing a web service to publish documents from the EDRMS to the web and automating the publication of the individual specifications;
  3. supporting staff to embed the change;
  4. measuring benefits using the impact calculator tool.

Phases 1, 2 and 4 have been completed successfully and further work will be undertaken on phase 3 (as this is a longer-term package of work).


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s