At this point I decided to look in more detail at the user interface and our typical usage pattern around time capturing. A large part of our business is consulting services where we log and bill time to clients. This is therefore something that every employee will use on a daily basis and from past experience we know that this can cause major frustration and disruption to the business if it doesn't work optimally. And of course logging time doesn't exactly add value to our customers so the less time our people spend on this activity the better.
To test this I devised a simple test in three parts as follows:
- I log in, create two projects with three tasks each, log out.
- I log in, capture four time entries with different project/task combinations, log out.
- A colleague logs in on a typical staff account and logs eight time entries on various project/task combinations, then logs out.
This test revealed some interesting points. For instance, our user-friendliness ratings for Zoho wasn't particularly high, but it did very well on timing, suggesting that our subjective rating is definitely subjective.
I also noticed very quickly that the app that is tested first is at a disadvantage as I got to know the data I was capturing better with each repetition. To counter this effect, we did everything twice and did the second run in reverse order (i.e. if Freshbooks was first in the first run, it would be last in the second run) and then used average time readings.
The end result of this process gave us the following scores:
- FreeAgent: 8.69
- Freshbooks (+ Kashflow): 10.16
- Triggerapp (+ Xero): 2.36
- Zoho: 9.70
- Paymo (+ Kashflow): 8.45
- Harvest (+ Xero): 10.46