Lean manufacturing vs Scrum - motion waste (part 4)

This is a continuation of the previous post in this series.

In short, the number one goal in lean manufacturing is eliminating waste. In this series I'm looking at the seven categories of waste identified in lean manufacturing, how these translate to software development, and how scrum can help eliminate that waste. This is the fourth post in the series.

In this post I want to focus on motion. This is one of the seven categories of waste targeted by lean manufacturing.

In manufacturing motion refers to motion of people or machines, but not work product (work product that moves around is transportation). For example, if a person on an assembly line needs to get up and walk 10 steps to get a specific tool every now and then this is motion waste. Motion waste isn't so much about the wasted effort of moving, but more about the wasted time. Because while you're moving to get that tool, you're not adding value to the product. Motion also provides opportunity for work disruption that wouldn't exist if you avoided the motion in the first place.

Similarly, in software development, motion waste is when people are moving around between places where they might add value rather than actually adding value. This could be travelling between your office and a client site, walking from a meeting room to your desk, walking to a colleague's office to ask a question, etc. I also believe virtual motion is potentially a big waste that is often neglected. If you are busy working on an IDE on your computer and you need to remote desktop to a separate server from time to time, this can also be considered motion as the action of switching between environments doesn't add value, it wastes time, and it provides an opportunity for disruption.

How does this tie back to scrum?

Scrum isn't really very prescriptive about motion, but it does put a lot of emphasis on team work and collaboration, a lack of which is often a leading cause of motion waste. It is advisable as far as possible to put a team in the same room to remove physical barriers to teamwork and collaboration and to also put the team's scrum board in the same room.

Scrum also provides a fixed framework of events (scrum planning, scrum review, scrum retrospective, and daily scrum) that helps to eliminate excessive meetings, another major cause of motion. The scrum master's role also becomes very important in eliminating waste. The scrum master is responsible for a lot of things that can directly affect, and therefore minimize, motion waste. The scrum master ensures that scrum events take place, the location and timing of which can have a great impact on motion waste. The scrum master also has responsibility for removing impediments and physical or virtual impediments in the tools and workspace can often necessitate motion.

In summary, scrum can help eliminate wastes that can be classified as motion because:

1. Scrum events provide a fixed structure that helps eliminate unnecessary meetings that often cause motion.
2. Teamwork emphasis, which should in practice be extended to putting team members, as far as possible, in the same physical space.
3. Emphasis on identifying and removing impediments that often cause motion waste.

Lean manufacturing vs Scrum - inventory waste (part 3)

This is a continuation of the previous post in this series.

In short, the number one goal in lean manufacturing is eliminating waste. In this series I'm looking at the seven categories of waste identified in lean manufacturing, how these translate to software development, and how scrum can help eliminate that waste. This is the third post in the series.

In this post I want to focus on inventory. This is one of the seven categories of waste targeted by lean manufacturing.

It is somewhat counterintuitive, even in manufacturing and especially for those with financial backgrounds, to see inventory as anything but an asset. Yet, inventory can be your greatest enemy. I'm not going to go into the manufacturing theory in detail here, but in summary inventory is a major waste. It basically means that you've tied up resources (including cash), into something that is not being used. Aside from tying up resources, there are also additional costs in handling of inventory from its storage and transportation to simple administrative costs. But perhaps the greatest problem with inventory is that it tends to hide other forms of waste. Inventory creates an illusion that things are well when they're not.

In software development any partially completed work is inventory and should be minimized. Partially completed tasks are mostly problematic because they hide the truth. There are usually complex integration or quality issues hidden in those partially completed tasks that you will never know about until you finally get around to trying to complete them.

To explain this I'll revert back to a manufacturing analogy. Imagine you produce custom furniture and have just won an order for a large batch of chairs. To minimize cutover times on machine setups you produce a large batch of legs, then a large batch of backrests, then a large batch of seats, then you assemble. Come assembly time, you realize that many of the legs are faulty and cannot be fitted onto the seats without some rework. Now you have more machine setup time, more work on the legs, and finally you can start assembly again. Now look at all the waste. Aside from the cost incurred because of effort, storage space, etc. and the waste associated with the rework, the inventory was hiding a quality issue. If you had made four legs, one backrest, one seat, assembled and repeated, you would have discovered the quality issue the moment it started happening, but instead you continued to build up inventory that you thought would be an asset, but quickly turned into a liability.

This is a very simplistic example of something that is usually much more nuanced and complex in manufacturing and even more so in software development, but it highlights how inventory can hide other problems.

Back to the world of software. To make matters worse software projects have traditionally made extensive use of progress reports. Progress reports are great perpetuators of the lies of inventory. People want to look good in progress reports so they overestimate progress. They also tend to vastly underestimate the amount of work remaining on any given task. And progress reports tend to accumulate a false narrative up to a climax where the whole project crashes down. At the start of a project there will be zero tasks with any progress, when you get to the 80% completed stage, this tends to be represented not by 80% of tasks completed, but by a small number of tasks completed and a large number of tasks almost completed (typically with progress percentages in the 75%-95% range). If this sounds familiar, you're not alone.

Consider the problem with the chairs again. If you were asked to estimate progress after the complete batch of legs had been done, you might have broken the work up into 4 tasks (legs, seats, backrests, assembly) and you probably would have said that manufacturing of the components is more effort so will carry more weight. Thus, by the time the legs were done the overall completion estimate might well have been about 30% (100% legs, 0% seats, 0% backrests, 0% assembly). At the start of assembly you would be forgiven for thinking that you were 90% done. But because of the quality issue hiding in inventory, you would have been completely wrong and 50% would probably have been a much more accurate number.

Unfortunately, these kinds of errors only cut one way. There are always hidden quality issues or complexities that you didn't realize would exist, but there are never golden eggs that make your life magically easier. Legs that work so much better than expected that you only need to fit two per chair or legs that assemble themselves simply don't exist. Legs that break or don't fit do exist.

How does this tie back to scrum?

Scrum puts a lot of emphasis on task completion. You must finish a task before starting the next one. A feature must be completed before moving on to the next one and the goal for a sprint is to complete features, not to have them at some point of perceived progress. This is important because it eliminates all the risk associated with work in progress.

In scrum there are no progress estimates and the only measurement of progress is when a feature is completed. Look back to the chair example: If progress was measured by completed number of fully functional chairs (i.e. the only thing of real value to an end user), you would have been sitting at 0% progress when you first started assembly. This sounds harsh, but it would have been a good warning sign of the trouble that lay ahead. If the batch was for a 100 chairs, you would have been at 1% after completing the first fully functional chair and by the time you got to 80%, the remaining 20% would most likely have been a walk in the park.

The definition of done becomes very important here (and there are lots of online resources that delve much deeper into this topic) as it directly affects how we perceive progress. A software feature isn't done when it just needs to be tested, or when it just needs to be integrated with something else, or when it just needs to be deployed. It's done only when there are no justs left. Exactly what the team's exact definition of done is depends on the context of their work, but it is important the definition of done is agreed upfront and is transparent. This definition of done is very central to scrum and an important part of managing software development without getting nasty surprises.

Scrum also puts a lot of focus on delivering shippable product at the end of each sprint. Artefacts like requirements specifications or design documents have no real value to the customer, so delivery is focused on working software. Before you have working software delivered and being used by real-world end users, it's not delivering real value.

To summarize, I think scrum is great at eliminating inventory waste and does so in many ways. Particularly:

1. Emphasis on task completion and how a task needs to get done before starting the next.
2. There is no reward or recognition for partially completed tasks.
3. There are no progress estimates that give a false sense of security.
4. Delivery of shippable product in each sprint is emphasized.
5. Working software is emphasized over work artefacts that don't have inherent value.

Scrum isn't prescriptive about everything that could impact inventory waste though, and it is important that the team also keeps an eye on this. I think what is especially important here is the definition of done. Scrum puts a lot of emphasis on this, but ultimately the team has to decide what their definition of done is and if this definition allows any scope for partial work to be considered done, the end result is inventory waste.

Lean manufacturing vs Scrum - transport waste (part 2)

This is a continuation of the previous post in this series.

In short, the number one goal in lean manufacturing is eliminating waste. In this series I'm looking at the seven categories of waste identified in lean manufacturing, how these translate to software development, and how scrum can help eliminate that waste, or not. This is the second post in the series.

In this post I want to focus on transport waste, one of the seven categories of waste targeted by lean manufacturing.

Transport doesn't sound like something that's really relevant in software development, but there are some typical wastes that happen in software development projects that are analogous for various reasons.

Transport waste in software development doesn't happen when physical material is transported between locations, but rather when focus is shifted and tasks are transported. I like to split this into two categories:

• Task switching
• Task reassignment

Task switching is similar to transport waste in manufacturing because you're really moving your thought process around when you switch tasks. This doesn't require physical movement, but the mere act of moving between different streams of thought wastes a lot of time.

Software development can be very complex and, especially when done well, requires deep levels of concentration. To exit that state of focused concentration and switch to something else, which in turn most likely requires a similar level of deep concentration, is not trivial. If you don't take the time to focus properly you're likely to make lots of errors and what you produce is likely to cause more problems than it fixes, in other words it will create more waste.

Modern culture has created the false impression that humans can "multi-task", but this is a myth. There are lots of studies in psychology on this topic and it falls somewhat beyond the scope of this post, but the findings show that people do not multi-task, they just task-switch. I'm not going to go into more detail here, but this is nicely summarized in this blog post by Aza Raskin.

The bottom line is, switching between tasks costs time. The time spent switching between tasks will make everything go slower and is waste.

Another form of waste in software development is task reassignment. This is similar to transport waste in that you're essentially transporting work from one person to another, either internally in the team or to new team members that are joining the team. It's almost intuitive (especially for most project managers) when a project is in trouble to throw extra resources at it. But this is fundamentally flawed. Software development complexities mean that ramp-up time for new people joining a project will be big and often prevent the existing team members from effectively continuing with their work. Adding and removing people from a project, or moving tasks between people on the team, creates a special type of waste because of handovers. The goal with these changes is usually to make things go faster, but the result is usually that things actually go slower, at least in the short term.

How does scrum address this?

Let's start with task switching. There is a lot of emphasis in scrum on getting things done and especially on not starting anything if you still have partially completed tasks. Implemented properly by a team, this eliminates task switching waste very effectively. It's therefore important to make sure team members are doing one thing at a time. Daily scrums are especially useful for this purpose to keep this focus.

Ideally, the team should also focus on completing one thing at a time, i.e. nobody should start working on feature two if there are tasks waiting on feature one. Two partially completed features at the end of a sprint will do no good. This may seem counterintuitive, but when a team gets it right it does wonders to bring focus and increase flow.

On to task reassignment. Teams change over time, especially if you're busy with long term projects, and in most cases it would be naive to assume that a team can remain completely static for the duration of a project. Also, if something isn't working on a team you have to try and fix it. In extreme cases that may involve changes to the team structure. But this should be a last resort and should come from within. Scrum puts a lot of emphasis on team autonomy and generally a change shouldn't be made to a team unless the team demands it, however scrum doesn't really dictate when our how this should happen, so the onus is on the team to handle this responsibly.

The iterative nature of scrum does provide an advantage though, in that when task reassignments happen the impact is smaller. Since the project team will always be working on a relatively small subset of overall requirements, the potential damage in task reassignment is somewhat limited. To be clear, it still causes a lot of waste, but the potential size of that waste is much bigger if you're using a waterfall-type approach.

In summary, scrum directly addresses transport waste by encouraging task completion before moving on.

More indirectly and incidental scrum also reduces the potential risk in task reassignment due to its iterative nature.

Where scrum doesn't have a direct impact and the team needs to make sure they apply good common sense and engineering practices is in watching for individual task switching at a micro level (e.g. individual team members should apply discipline and not be distracted too often by things like e-mail) and by making sure task reassignments are only done when absolutely necessary.

Lean manufacturing vs Scrum - intro (part 1)

This is the first post in a blog series in which I intend to explore the relationship between lean manufacturing and software development, and how scrum can, or can not, help eliminate the seven wastes of lean.

We use scrum for software development at Xpedia. If you don't know what that is, check our methodology write-up on the subject or if your attention span is shorter try our slide deck.

Before starting, it is important to note that scrum has a strong aspect of continuous improvement, especially through retrospectives at the end of each sprint. While the retrospectives are an integral part of scrum, the improvement could be totally unrelated to scrum and more often have to do with the team's environment and specific challenges. These continuous improvements should therefore help eliminate waste of all flavours, so in an indirect way one could argue that scrum addresses all categories of waste. Similarly the important role of a scrum master can be argued to have a role to play in eliminating any sort of waste, but again this is rather indirect and probably depends heavily on how good the scrum master is.

For purposes of this series though, I'm going to ignore indirect benefit though these mechanisms and rather focus on the direct waste-elimination benefits of core scrum components as I see it.

You may also be familiar with the happily acronymed lean software development (LSD). There is a relation with scrum, but it's not quite the same thing. The terminology for LSD (yes, I have to call it that now), originated (according to Wikipedia at least) with a 2003 book, Lean Software Development: An Agile Toolkit, by Mary and Tom Poppendieck, who specifically took the principles of lean manufacturing and applied these to software development to come up with LSD. However, scrum predates this by several years and by 2003 was already fairly well established. Of course, the Poppendiecks knew scrum well when they wrote their book and it contains lots of nods to scrum, but it seems their background in various manufacturing environments led them to more directly apply the principles of lean manufacturing to software development in general and not just specifically through scrum. I'm going to probably lean rather heavily on the Poppendiecks' work throughout this series and will definitely also make use of various other sources, but will rather list these in the final post to make sure I include them all. I do recommend this book though, definitely a worthwhile investment of your time if this stuff interests you.

Before LSD, the overlap between scrum and lean manufacturing principles seems to be a bit more coincidental. Various authors did pick up on this, but to my knowledge the Poppendieck's seem to have been the first ones to actually take lean manufacturing principles and apply them directly to software development.

Let's get to the seven wastes now.

Lean manufacturing focuses primarily on eliminating waste and specifically identifies the following 7 types of waste:

• Transport
• Inventory
• Motion
• Waiting
• Overproduction
• Over processing
• Defects

Others have attempted to add to this list over time, but mostly lean manufacturing focuses on these seven. Proponents of scrum may think this was stolen from scrum, but I assure you lean manufacturing is somewhat older than that! 

In the following posts I'm going to explore the extent to which there is an analogous waste in software development  and how this is minimised by following a good scrum process and also in what areas you'll need to think for yourself and cannot rely on scrum alone.

I'm back!

I've been lazy with this blog for too long. Going to start posting like crazy soon. Watch this space!

The great cloud accounting software search of 2014 (part 6 - the final)

This is a continuation of my search from the previous posts (part 1, part 2, part 3, part 3b, part 4, part 5).

After much effort, I finally arrived at a stand-off between Zoho books or Xero in combination with Freshbooks.

To get to a final decision, I decided to make a list of our various accounting requirements, weight each of these on a scale of 1 to 10 depending on how important the requirement is, then score each of the options on each requirement, again on a scale of 1 to 10.

The requirements I rated against included a long list that I'm not going to bore you with, but will provide a short summary of significant differences. The areas where there were significant differences are listed below.

Areas where Zoho had the upperhand (note that in some of these Xero/Freshbooks also has a solution, but for some reason or other I just found Zoho's better or easier to use):

• Visibility over which projects have unbilled time
• Google apps integrated login
• Consulting rates definitions/setup

Areas where Xero/Freshbooks had the upperhand (again note that in some of these Zoho also has a solution, but for some reason or other I just found Xero/Freshbooks's better or easier to use):

• Export features to Excel/Google docs
• Sending customer statements in bulk
• Automatic exchange rate
• Fixed asset register
• Automated depreciation

Ultimately the combination of Xero and Freshbooks advantages outweighed Zoho and the final score on my whole list of requirements was 1005 for Xero/Freshbooks and 966 for Zoho. It was therefore a pretty close contest, but the bottom line was that Xero in combination with Freshbooks provides a slightly better fit for our requirements than Zoho.

To make the decision a little more difficult Zoho will cost us $24/month whereas Xero/Freshbooks will cost about $240 per month initially, but will grow by $10 per month with every additional employee we add on Freshbooks. I eventually decided to stick with my decision on Xero/Freshbooks though, since I did decide from the outset that as long as I stay within budget, I would prefer a better solution over a cheaper one.

Thus our journey with Xero and Freshbooks begins. 

The great cloud accounting software search of 2014 (part 5)

This is a continuation of my search from the previous posts (part 1, part 2, part 3, part 3b, part 4).

Last night I conducted a standard test of a few typical accounting functions that I would use to further eliminate candidates from my list of Zoho, Xero + Harvest, and Kashflow + Freshbooks.

In the process I realised that there is an additional integration option of Xero + Freshbooks. I had not considered this earlier because for time tracking options with Xero, I had consulted the list of Xero partners in this section on the Xero website where for some reason Freshbooks is not listed. When I looked at the integration settings in Xero there was a section for Freshbooks though, so I decided to add this as an additional option and this brought the list back up to 4.

The standard test took much the same format as my previous test, although I put much less emphasis on time, more emphasis on accuracy, and the tests themselves were invoicing and accounting functions rather than time-logging. The results came in as follows:

• Kashflow + Freshbooks: 22.5
• Zoho: 52
• Xero + Harvest: 50
• Xero + Freshbooks: 53

Kashflow-Freshbooks integration gave me an issue whereby a USD invoice was incorrectly integrated to Kashflow as a ZAR invoice. It should be noted that Kashflow's Freshbooks integration is still in beta. This issue severely affected the score for this option as can be seen above. This is something that we will use regularly. As long as there are other options available that do this properly, the Kashflow-Freshbooks option is therefore disqualified. I have logged the issue with Kashflow so they're aware, but for now I have to move on. If they resolve it shortly or come back with a satisfactory answer (perhaps I was being a doofus) I might reconsider.

I've also decided to disqualify the Xero-Harvest option based on a combination of the above and price. Harvest will work out considerably more expensive (±$200 per month) than Freshbooks (±$60 per month) for our organisation and from the testing I've done so for Freshbooks is a marginally better fit for our requirements anyway. It therefore doesn't make sense to consider the Harvest option any further. Again, nothing bad in Harvest, it just seems in our case Freshbooks will be a slightly better fit.

And then there were two:

• Zoho
• Xero-Freshbooks

To be continued...

The great cloud accounting software search of 2014 (part 4)

This is a continuation of my search from the previous posts (part 1, part 2, part 3, part 3b).

At this point I decided to look in more detail at the user interface and our typical usage pattern around time capturing. A large part of our business is consulting services where we log and bill time to clients. This is therefore something that every employee will use on a daily basis and from past experience we know that this can cause major frustration and disruption to the business if it doesn't work optimally. And of course logging time doesn't exactly add value to our customers so the less time our people spend on this activity the better.

To test this I devised a simple test in three parts as follows:

1. I log in, create two projects with three tasks each, log out.
2. I log in, capture four time entries with different project/task combinations, log out.
3. A colleague logs in on a typical staff account and logs eight time entries on various project/task combinations, then logs out.

For each part of the test, the tester (either me or the colleague) rates the user friendliness subjectively on a scale of 1 to 10 and for each part of the test we also time how long it takes.

We then come up with a combined score for each app that combines our subjective user friendliness ratings with an overall time spent rating. The time spent rating combines the time entries with weighting factors to estimate the total person hours that our organisation might spend on time capturing in a typical month. The user friendliness scores are added together (and in the case of time capturing averaged) and multiplied by a weighting factor representing the number of users who would be exposed to this functionality. We then take the combined user friendliness rating and divide it by the estimated man hours per month spent on capturing to get to an overall score and pick the top 3 from this.

This test revealed some interesting points. For instance, our user-friendliness ratings for Zoho wasn't particularly high, but it did very well on timing, suggesting that our subjective rating is definitely subjective.

I also noticed very quickly that the app that is tested first is at a disadvantage as I got to know the data I was capturing better with each repetition. To counter this effect, we did everything twice and did the second run in reverse order (i.e. if Freshbooks was first in the first run, it would be last in the second run) and then used average time readings.

The end result of this process gave us the following scores:

• FreeAgent: 8.69
• Freshbooks (+ Kashflow): 10.16
• Triggerapp (+ Xero): 2.36
• Zoho: 9.70
• Paymo (+ Kashflow): 8.45
• Harvest (+ Xero): 10.46

You will notice that Harvest scored strongest. Their branding is also orange, so my initial reaction was that I must have been biased. However, my user-friendliness rating for them on the first part was joint-lowest and for the second part was joint in a 4-way tie for second (Freshbooks being the only one that scored higher and Triggerapp being the only one that scored lower), so I don't really think that I favoured them unfairly in any of my subjective ratings.

I should mention that Triggerapp scored particularly low because it seems to be designed on the premise that you will always (or at least the majority of the time) use the timer to start/end time recordings. In our case I don't foresee our users actually doing this. This had a significant impact on our ratings and times as it meant we had to start timer, stop timer, edit time to capture what would otherwise simply have been creating an entry. I'm not sure if we just missed something obvious, but neither of us could find another way of doing this. The app itself actually looked really nice and had some funky ideas, so if the timer start/stop mechanism is your cup of tea it's well worth a look.

To conclude, the top 3 that move on to the final round of my evaluation process are therefore Xero + Harvest, Kashflow + Freshbooks, and Zoho. 

In the next round of evaluation, I'm going to generate invoices from time captured (this will also test the Freshbooks-Kashflow and Harvest-Xero integration) and do some accounting entries that I know we require and that would stretch something that isn't proper accounting software.

The great cloud accounting software search of 2014 (update to part 3)

In the previous post I mentioned 6 options that would work for us, but afterwards realized that Xero + Timely wasn't going to work for our requirements. Timely seems much more geared towards rostering / scheduling, which really isn't our requirement. BTW, Timely looks really neat if this is your requirement, just not what we're looking for at the moment.

[Edit on this post] At first I wasn't sure how this slipped through in the previous round, but later realised that I switched comments meant for Timely and Harvest, two of the Xero time/project management options. So this meant that the Xero + Timely option was replaced with Xero + Harvest.

The great cloud accounting software search of 2014 (part 3)

This is a continuation from this previous post.

In my search for new accounting software, I've not narrowed down the options to four that I feel are viable alternatives, namely FreeAgent, Kashflow, Xero, and Zoho.

However, I need a complete solution that also incorporates time tracking. Xero and Kashflow do not offer this functionality except through add-ons. In both cases numerous add-on options are available, so before the next round of evaluation, I decided to check out the various add-ons and narrow these down to a few usable options so that I can ultimately compare apples with apples.

My next step was therefore to evaluate the various time tracking options available for Xero and Kashflow. In the project / time management categories Xero has 28 add-ons listed on their website and Kashflow 6, with only one common entry. 33 options is a bit too much, so I decided to quickly eliminate options based on a few basic criteria:

1. Eliminate anything that seems overly complex.
2. Eliminate anything that is likely going to push the total solution (with Kashflow/Xero) price over $250 per month.
3. Eliminate anything that does not have a free trial or requires me to enter credit card info to qualify for the free trial.
4. Eliminate anything that clearly won't work (e.g. if it's an Outlook add-on rather than a cloud-based time-tracking app, etc.)

This resulted in quite a quick elimination process with the majority of options falling out on price and eventually left me with 4 options, 2 with Kashflow (Freshbooks and Paymo), and 2 with Xero (TriggerApp and Timely).

Thus, we're now left with 6 options in total:

• FreeAgent
• Kashflow + Freshbooks
• Kashflow + Paymo
• Xero + TriggerApp
• Xero + Timely
• Zoho

Time capturing efficiency is quite an important part of our requirement, so from here I decided to compile a standard time capture test that I will complete with each time capture option and to then proceed with only the best three. I'll cover this in my next post.

The great cloud accounting software search of 2014 (part 2)

This post is a continuation of my search (see previous post here)

After a rather exhaustive googling exercise, I arrived at a list of products that I initially looked at. The list got quite long, so I started to eliminate as I went along by performing these basic checks:

• Does it have a free trial that includes all functionality, or at least all advertised functionality that I'm interested in?
• Does it support multiple currencies? Specifically I checked for the following:
• Supports ZAR as base currency
• Can I, in the trial, create an invoice in a different currency (USD), either by specifying the currency on the invoice or by specifying the currency on the customer record.
• Check if some basic accounting features are supported, especially functionality for deferred revenue and work in progress, even if only through manual journals.
• Price: I quickly realised that I would quite easily be able to find a good enough solution for less than $250 (USD) per month, so anything that would cost more than this based on our expected usage was eliminated.

The majority of the options that were eliminated were eliminated due to multi-currency shortcomings. This doesn't necessarily mean that the software doesn't have any multi-currency functionality, it just means that I could not get it to do what I want it to do within a reasonable time (about an hour) of trying in the trial version. Thus, in some cases elimination on grounds of lacking multi-currency functionality was rather a reflection of usability or trial edition limitations. Regardless, for my purposes, this is sufficient reason to not waste any further time evaluating such options.

Using this procedure I was able to eliminate the majority of options. I have to mention that some of these looked like really great software, but for some or other reasons simply didn't fit our requirements. This does not mean that it's bad software, it might be a much better fit for someone else. The full list in the order in which I evaluated them: Freshbooks, Xero (still in the running), Clearbooks, Kashoo, Kashflow (still in the running), Acumatica, Sage One, Intacct, Zuora, FreeAgent (still in the running), Quickbooks online, Netsuite, Outright/Godaddy bookkeeping, Big Red Cloud, Bean Cruncher, Zoho books (still in the running), Saasu, SMEasy, Bluubin, PingAccounts/CapitalOne.

Thus, I'm left with four options that I feel could do the job:

• Xero, www.xero.com
• Kashflow, www.kashflow.com
• FreeAgent, www.freeagent.com
• Zoho books, www.zoho.com

In both Kashflow and Xero's case some kind of time tracking add-on may be necessary too, but both of those seem to have a generous number of add-ons available. Although this adds extra cost it still looks like most combinations will be significantly cheaper than some of the more expensive options I evaluated and hopefully still well within my $250/month limit.

I'm now moving on to a next round where I'll compare features and usability against our requirements in a bit more detail and hopefully arrive at a final answer shortly. Although the available options may be quite varied in pricing I'm not too worried about price at this stage. As long as the price remains within certain limits (like $250/month), finding the best fit is a much higher priority than saving a few bucks.

The great cloud accounting software search of 2014 (part 1)

Exciting times as we're moving into a new phase of Xpedia's history and I'll hopefully be posting more often in the near future. As a starting point I'm going to document my search for new accounting software here.

I'm looking for a better accounting solution for Xpedia's applications division. Going forward this will be a separate legal entity that I will lead. Our current software is Acumatica, which is a very feature rich ERP suite. We host it ourselves, but it is completely browser-based so I've become spoilt with having something that's always available from anywhere. The only problem with Acumatica is that it is complete overkill for our requirements. This means the processes are too cumbersome as we don't have dedicated staff who can specialize in a single discipline/module like debtors clerks etc. When you have the staff to use Acumatica properly I think it would be a fantastic product to use, but we're simply not there yet. It also means the software is a little pricey. The Xpedia Applications business has a headcount of 20 and annual turnover in the R5 000 000 to R10 000 000 range (that's roughly $500 000 to $1 000 000 in USD), so a full-blown ERP is a little bit out of our league.

So for part 1 I'm going to list requirements and hopefully get some additional ideas on this and from part 2 onwards I'll expand on our search, the products we've looked at and the process to evaluate and arrive (hopefully very shortly) at a conclusion.

Here's a list of features we need:

• Cloud-based, hosted by provider
• Easy to use. Everyone always says that, I know... But it must have a free trial so I can check it out first hand and see if I find it easy to use.
• Must support South African Rand (ZAR) as base currency for our business.
• Must support multiple currencies (i.e. I want to be able to send a client in Mozambique or DRC a USD invoice instead of a ZAR invoice).
• Must support recurring invoice items (we have lots of clients with monthly retainers or recurring software license fees)
• Must support recurring cost items (e.g. recurring software license fee costs, our own retainers to suppliers etc.)
• Must support fixed asset depreciation, revenue deferral, work in progress, even if only through manual journals.

Nice-to-haves:

• Banking integration (we will use an First National Bank of South Africa account)
• Time tracking
• Support ticketing
• Project management
• Departmental transactions and reporting (not that we really have departments, just want to keep track of profitability of different products and services separately and traditionally something like a departmental breakdown appears to have been the easiest way to achieve this)

varchar vs nvarchar (and text, ntext, char, nchar)

Everyone who's ever created a table in SQL has probably at some point wanted to add a field to store text values and wondered about the difference between all the various options. These are numerous. The most commonly used are varchar, nvarchar, char, nchar, text, and ntext. You could also store text in binary, nbinary, or image columns. Actually you could store text in almost any data type if you're willing to put yourself through the hassle of very complicated conversions.  Let's stick with the original six for now: varchar, nvarchar, char, nchar, text, ntext.

The n prefix denotes unicode (more about this later), so ignoring that, we're dealing with three data types here: varchar, char, and text. Varchar is by far the most commonly used and the rest of this article will focus on this type, but in short these three can be differentiated as follows:

• char: Fixed length string. In other words, char(4) will always have four characters. Thus if you want to store "A" it will become "   A" (A padded with 3 spaces). This is not used very often, but can be useful in certain scenarios. 
• text: This stores the data in a blob (i.e. not in the row data of the SQL table). This is useful when you expect to store huge text values as there's no limit on the size of the data. However, this type is being deprecated and is being replaced (sort of) by varchar(max).
• varchar: Up to 8000 characters this stores text in the row data (similar to char) but not always of fixed length. In other words if you create a varchar(4) column and store a value "A" in it, this will use just one byte to store the A. AA will use two bytes, AAA will use 3 and so on. This implies that the length of various rows in the table will vary from one row to the next. varchar always uses an additional 2 bytes for storage, essentially for meta data so that SQL server knows how much data to expect in each specific instance. That means if you know you will always need exactly 4 characters, char is much more efficient in terms of storage requirements and processing overhead, since it doesn't need the additional 2 bytes of meta data nor any processing power to interpret that meta data. Having said that, in the majority of modern applications, this type of gain from using char instead of varchar is trivial. In the olden days when memory, processor cycles, and storage space were more limited, this was a serious consideration though. In previous versions of SQL Server, varchar could only go up to 8000 characters, but as of SQL 2005 varchar(max) will allow you to store any amount of data. This works like varchar described above up to the 8000 character limit and once data size goes over this it switches over to a blob to store the data and behaves more like the text data type (presumably this is also why the text data type is no longer necessary and being deprecated).

Now, on to varchar vs nvarchar. These two types are essentially the same, except that varchar is non-Unicode and nvarchar is Unicode. What does this mean?

Unicode is a standard for encoding text that was specifically designed to allow for storage of a larger number of different characters and character sets to cater for the requirements of different languages. The wikipedia article for Unicode describes this in more detail.

In SQL Server, this means nvarchar supports multiple languages. If your data is only going to be in English, this is not necessary. If there's any chance that your data will contain other language data, especially languages using non-English character sets, like Arabic, Russian, etc., etc. then you will need to use nvarchar. The cost of using nvarchar is that it almost doubles the amount of storage used to store the data as it uses two bytes for each character of data, where varchar uses only one byte per character. Both have the same overhead of 2 characters, so effectively varchar will use 3 bytes to store "A" and nvarchar will use 4 bytes for the same thing. To store "QWERTY", six characters of data, varchar will use 8 bytes (2+6) and nvarchar will use 14 bytes (2+6*2). 

My recommendation: If you're absolutely 100% sure that you will not need non-English data, and you will not need it in the foreseeable future either, then use varchar. In all other cases, use nvarchar.

Software I can recommend

This list isn't meant to be exhaustive, but I wanted to highlight a few things I use regularly that I can highly recommend. I also use a lot of other stuff (like MS Office) that is really good, but most people already know about those, so these are the lesser known ones that I think is worth a try:

• Pixlr: Great for quickly editing a picture or logo. Works like a charm, completely web-based, no install necessary. And it's free.
• VIPRE: Vipre is my anti-virus of choice. Over the years I've tried various products including TrendMicro, AVG, Norton, and McAfee, but VIPRE stands out. I haven't had any issues with viruses since I started using it about 2 years ago. It's really lightweight and resource-friendly compared to some of the others, and I've been very impressed with their responsive support, especially on chat and e-mail.
• Xara: Not one of the mainstream design software names, but very good and competitively priced. I haven't used it very extensively, but from what I've seen it does the job and comes in much cheaper than it's main competitors. I've also received good support from them on something that they really didn't need to help with.
• Google apps: Perhaps a bit too mainstream for this list, but it's so good that I really have to mention it. Xpedia started using Google apps a few years ago and I cannot recommend it highly enough. I use forms, spreadsheets, and sites regularly and I simply don't think there is an easier/cheaper way of doing what you can do with these tools. The mail is fantastic too. A lot of our users still use Outlook, but I don't recommend this because the Gmail web interface is just so good that it isn't necessary to have a mail client anymore. While Word/Excel are more feature rich than the comparative tools in Google apps, the ease of collaboration and sharing in Google apps counters this nicely. I use a combination of both, so it depends on what you need to achieve, but I think a lot of organizations can get great value from Google apps. The aforementioned tools are just a fraction of what you get when you sign up for Google apps though, so well worth checking out. And hey, if you're small enough, it's free!
• Spybot S&D: Spybot is a great tool for keeping your PC free of malware and spyware. I've been using it for years and it really is a great project. Easy to use and works well with most AV products.
• Mozy: I've been using Mozy's backup service for years and the backup part has always impressed me. It's easy to use and set up and doesn't interfere with your daily work (i.e. not a resource hog). Then I had a crash recently and had to "test" restoring. This could not have been easier. What a pleasure. All my backups, 100% up to date, available easily and immediately.
• Red-Gate: More specifically Red-Gate's SQL tools. The ones I use often are Compare and Data Compare, but every one of their tools that I've tried has been simply fantastic. Easy to use software that just works. For people in the MS SQL world this is probably also considered mainstream, but in case you haven't tried it, do yourself a favour.
• WinRar. Yes, I'm one of those people who actually bought a compression agent! WinRar is really good though. Supports a huge list of compression file formats, works quickly and well.

The cloud: Hype or real?

The ever-increasing hype about the cloud is getting to me. My personal experience is that a lot of people in our industry love throwing around the buzzword, but simply have no idea what it means. To dispel some  myths, I recently (ok it was months ago, but I'm only getting around to blogging about it now...), did some research and gave an internal presentation at one of our company functions.

I've made the slideshow accompanying my presentation available for download here. Some of the slides are very specific to our business and these were designed to accompany my talk so it isn't intended as a comprehensive narrative in itself, but hopefully something that can be of use to readers anyway.

The highlights:

1. The term "cloud" actually comes from network diagrams that used clouds to represent portions of the network that did not need to be known for purposes of the diagram. In this context "cloud" means something like "Unknown magic happens here".
2. In most modern contexts "the cloud" typically means the Internet or some subset thereof.
3. A lot of people seem to refer to cloud when they actually mean cloud computing. This can be defined as data, services, or applications hosted on anonymous Internet-based infrastructure. Peder Ulander of Citrix gives a more comprehensive and pretty good description in this video: http://www.youtube.com/watch?v=C4krCK1PZHY
4. Cloud computing can be categorized by service type as Infrastructure as a Service (IaaS), Platform as a Service (PaaS), or Software as a Service (SaaS). 
5. While PaaS (e.g. Azure, Amazon Web Services, etc.) is fairly new, IaaS and SaaS are as old as the Internet itself. IaaS is really nothing but hosting services and web hosting companies have been doing this for years. It has changed and evolved over time though and there is a lot of new stuff out there in this space, but it's still the same concept. Earliest SaaS examples are probably web mail services like hotmail. This field is growing very quickly with ever more advanced apps. Good examples of business applications in this area are Salesforce.com, Netsuite, and even our very own NuQ HR/Payroll system (see www.nuq.co.za).
6. Cloud computing has advantages and disadvantages. 
1. For me, the biggest advantage is not having to worry about infrastructure. Your application can scale with your business without you even having to think about it, much less buy dedicated hardware and software to support it. This is based purely on my personal experience and probably more relevant to SaaS than IaaS or PaaS.
2. The biggest disadvantage (again only in my personal experience) is that support is often impersonal. Services are designed to scale well and that means personal service is usually not a priority. When everything works well this isn't an issue, but when you do have a problem that really affects your business this can be extremely frustrating.

In conclusion, the cloud is just hype, but it's also real. 

It's hype, because it's existed for far longer than the cliched terminology that the buzzword brigade so love to use today. There's nothing new about the concept and it's simply an ongoing evolution of services available on the Internet. It's also very real, because this evolution has reached a point where the services on offer are becoming rapidly more useful in our day-to-day business and personal lives.

In my experience when you meet someone who goes on about the wonders of the cloud, this is a good indication that they've heard the hype, but don't understand what it is. In most cases you can replace "cloud" with "Internet" in their every sentence.

TRANSACT-SQL: JOIN...ON condition or WHERE condition - which is better?

A colleague suggested I write about this topic as she often wonders about it, but doesn't really understand the implications and has never had a chance to test it.

Basically, the question is whether or not it's better to add a condition in a query in the WHERE clause at the end of the query or to make it part of an ON clause in the JOIN when writing queries in Transact-SQL.

The very short answer is: It depends.

The short answer is:

• INNER JOIN: It really doesn't matter and is a case of personal preference more than anything else.
• OUTER JOIN (LEFT, RIGHT, or FULL): There's a logical difference that in some cases will cause result sets to differ, so it depends on what you're trying to achieve with your query.
• CROSS JOIN: Has no ON clause and therefore isn't relevant for this topic.

Now comes the long answer.

Let's look at INNER JOIN first.

By means of example, say you have two tables called Manufacturer and Product respectively. You want to run a query that returns a list of all products with the manufacturer's name next to it. This is fairly easy to achieve like so:
SELECT Product.Code, Product.Name, Manufacturer.Name FROM Product INNER JOIN Manufacturer ON Product.ManufacturerID = Manufacturer.ID

Now, suppose that you want to add a condition to this. For instance, you only want a list of products for Manufacturers from a specific country, which you specify in a string literal. This could be done in one of two ways.
Option 1, adding a WHERE clause:
SELECT Product.Code, Product.Name, Manufacturer.Name FROM Product INNER JOIN Manufacturer ON Product.ManufacturerID = Manufacturer.ID WHERE Manufacturer.Country = 'Country1'

or Option 2, adding a condition to the JOIN clause:

SELECT Product.Code, Product.Name, Manufacturer.Name FROM Product INNER JOIN Manufacturer ON Product.ManufacturerID = Manufacturer.ID AND Manufacturer.Country = 'Country1'

The difference is subtle and the result set will be exactly the same. So which is better?

The answer is neither, or both. At least in terms of performance there is no difference between these queries. If you look at the query execution plan for these two queries, you will find that they are exactly the same. Basically SQL server will try to optimize the query regardless of where you put the condition.

To prove the point, I created some random data in the Product and Manufacturer tables, using 500 manufacturers, 150 countries and 1000000 products and then ran the query for one country using both methods. I did this twenty times for each query and the results were very closely matched. This wasn't a very well controlled test environment (I'm simply running it on my laptop with lots of other background services busy all the time), so saw a fair bit of variance and initially it looked like the option 2 might be faster, but over time the two averages converged. Most significantly the fastest times for each option were identical down to the last millisecond (i.e. on the odd occasion that nothing else was consuming processing power on my laptop, they were really identical).

Personally, my preference is generally to put these kinds of conditions in the ON clause of the join. This makes more logical sense to me as it feels like I'm limiting the result set earlier in the logical process and therefore I find this easier to read, understand and maintain. However, in practice it actually makes no difference, so it really is personal preference. There might be a good argument for putting all your conditions in one place (the WHERE clause) so you can easily see all the limitations that the code is putting on the result set even when dealing with multiple joins.

Now lets look at OUTER JOINS. I'm not going to go into all three types of outer join as for purposes of this article they will all behave similarly, so let's stick to the one that is most frequently used, LEFT JOIN.

With a left join, the two options are logically quite different. Looking at the above two queries, a left join will deliver vastly different results in the two respective options.

Option 1 will produce the exact same result set as the inner join does. This is partly because the Product table is first in the query and the foreign key constraint on Product.ManufacturerID and the unique key on Manufacturer.ID ensures that there will be no NULL countries nor multiple rows for a single product (i.e. every Product row will successfully be matched to a unique row in Manufacturer) and partly because the WHERE clause narrows the selection down to a specific country only.

However, option 2 will produce a completely different result. In this instance, the result set will be a complete list of products. Rows where the country condition is met will appear in the result set with a manufacturer name, but rows that do not meet this condition will still appear in the list, only with a NULL value instead of an actual manufacturer name.

In conclusion, for LEFT JOIN (and the same goes for RIGHT and FULL OUTER JOIN), the decision rests on what you're trying to achieve. Whether you want rows not meeting the condition to appear in the result set with NULL values for the secondary table or whether you prefer if these values are not there.

Note: Testing for this article was done on SQL Server 2008 R2 developer edition.

SSRS no longer using IIS

For those who haven't discovered this yet: Since SQL Server 2008, Microsoft decided that the reporting services web sites should no longer run in IIS.

This may seem a strange decision, but there is some logic to it. For one thing, you don't need to install IIS in order to run reporting services. I suppose it also prevents guys like me from fiddling with IIS settings that could break the report server, but now I've lost an aspect of control, which is rather frightening. And I don't think I'm a control freak.

It is what it is though and I cannot change that. So after a bit of research, I uncovered that this is done by SSRS using http.sys directly. Which begs the obvious question of whether this will cause a port conflict (if one assumes most sites and report server installs will both use the default port 80).

But alas it seems not. This technet article explains how http.sys handles these apparent port conflicts. It seems the only time there might be an issue is if you choose to install another virtual directory using the same name as the report server. I cannot think of any scenario where you would have a good and unavoidable reason to want to do this, so that should not be much of a problem.

Also see this post by Andrew Fryer highlighting some of the benefits of doing it this way. Ultimately it makes good sense and I don't imagine MS would change course any time soon.

Editorial note: This article was written some time ago for a blog that I never really got into very much, so just transferring it here to get all my content in one place. I know it's old news!