Wednesday 29 September 2010

What is success?

It's always fun to think seriously about success at the start - much more encouraging than thinking in terms of risks and all the things which could go wrong.

So, what could go right, how would we know it had gone right, and which things going right should we focus on?

We've already blogged about one thing we could measure and which could go right - deposit rates into the IR (or possibly subsequent access rates); this is a very measurable element and in fact measures itself.

User satisfaction is another great thing. If we can create a community of happy researchers and academics who are using Mendeley and our project deposit system without problems, and who feel it benefits them in some way, then that's another good success for DURA. To assess if we manage this, we'll need to do some combination of user testing, interviews and surveys (which will give us specific information about how the researchers using our tool feel about it as well as what the experience of using the tool is like) and potentially measure support requests and usage levels, which give us an indirect measure of how well things are going for users, but which could be affected by other factors too.  We are already thinking hard about user experience, particularly around setting up the deposit system for the first time for a user, which is where we hit the exciting technical challenges around authentication and authorisation. Getting the setup process right will be key, because without that, no one will make it through to the truly simple day to day operation of deposit, where we hope to have "no UI" because it will automagically happen whilst researchers use Mendeley normally.

Another form of success which we'd love to have is to be playing an active role in a thriving community of people, all contributing to awesome scholarly infrastructure around repositories and access and preservation. Our engagement with SWORD2 and the whole JISC DEPO programme is part of that, and so is our connection to the community of Mendeley users at Cambridge and beyond. This is a bit harder to measure...

So we will probably focus on the first two kinds of success - happy users, and checking the deposit rates. The system design, including the user interfaces but also the overall technology architecture decisions we make, will play a major role in making sure our researchers find the system easy to use and useful; these are things we are working on today. Later on, we'll also have to make sure we have seamless deployment plans and good support systems in place, as well as the processes to make sure we know how satisfied our users are.  The deposit rates we'll look at later on, and again the pilot deployment and publicity and so forth will be the big areas which will affect this kind of success.

Having written all that, it's back to the bits of the project which are not all successes - yet. The real challenge in this coming term for me locally is the package of institutional issues around coordinating diverse bits of the university to come together around the project and our forthcoming pilot deployments (of which more soon) - plus our partner companies. These aren't all technical issues - there are policy issues and communication challenges, translating between teams with very different backgrounds and priorities, and of course the inevitably slow progress of other university activities which DURA may depend on later. But that's the fun of a project like DURA, bringing together lots of different things to deliver something new :)

No comments:

Post a Comment