Blog Post
The Sheaf Digital Test - 12 Steps to Better Bespoke Software
This post was inspired by Joel Splosky’s The Joel Test: 12 Steps to Better Code. That post is now more than 20 years old, but it is still very relevant and has plenty of good, clear advice for its software developer audience. This one is slightly different because my intended audience is customers - people with little or no software development expertise who want to commission bespoke software. It’ll take you less than 5 minutes to read and you never know, it could save you a lot of time, hassle and money.
1. Have you described what you want as simple user requirements?
You need to work out exactly what you want and write it down as a set of user requirements. Don’t write long, complicated paragraphs because people read them. Instead, write short statements that describe simple user scenarios. For example “A member of the sales admin team should be able to sort/filter a list of new customer enquiries.”
Put the statements into tables, group them by business function or user role or something else that makes sense to you. Keep the format as simple as possible.
2. Is there an off-the-shelf alternative to bespoke software that will cost less?
Double check this. There are lots of affordable cloud based software solutions across all sorts of industries and in all sorts of niches. Software as a Service (SaaS) has become the standard way to buy business software. Do you really need to build bespoke software?
3. Do you understand the difference between T&M and Fixed Price?
I’m guessing that you’re going to pay a software development company to build your bespoke system. Be clear about the difference between time & materials and fixed-price before you start.
Remember - time & materials means you pay for each day of work they do. You pay extra if the project overruns - that risk sits with you. On a fixed price project, the risk sits with the software developer. They commit to delivering a system that meets your requirements.
Software development companies will naturally and reasonably build in some contingency when quoting for a fixed price project. They’ll also try to charge you extra during the development when they think the requirements have changed from the original brief.
4. Have you got a project plan that includes regular work in progress reviews?
Be clear with your supplier about how you want the project to run. Schedule in a meeting with your supplier every three or four weeks and ask to see the software as it is being developed. Say at the start that you want that kind of flexible, agile approach, even on a fixed-price project. There should be no surprises when the software is finished.
5. Does your preferred software developer do daily builds?
There’s a similar step in Joel Spolsky’s post, but his step is written for a software developer audience. From your point of view - the customer - daily builds are good because it means your software developer can update and rollout new releases quickly. Bugs are fixed faster and new software releases cost less. Daily builds also take a bit more effort to set up - it’s a sign that you’ve got a well organised, professional software development company.
6. Do you know how your software developer does testing?
It goes without saying that your software needs to be tested before you start using it. Find out how your software development company does testing. What is the split between coding days and testing days? Do they have a good issue tracking system for managing bug fixing and can they show you the stats - how many bugs are found and fixed. You want to understand the quality assurance processes they will wrap around your software as it is developed.
7. Do you know who long it will take you to do acceptance testing?
You need to do customer acceptance testing after your supplier has finished their development and testing. You need to do that and be happy with the results before you sign off the system and pay your final invoice. Make sure you allow enough time - a week is possible, two weeks is more likely. You’ll need a team of people to help. Acceptance testing is a good opportunity to get your colleagues familiar with the system.
8. Who pays when bugs in your bespoke software need fixing?
This is an interesting question because it sounds reasonable to say that the software supplier should always pay for bugs to be fixed. On the other hand, you’ve commissioned someone to develop something that’s bespoke and part of the job of developing the software is finding and fixing bugs. That’s an activity that someone has to pay for.
The other thing to remember is that it’s impossible to find every bug before the software is released. So there has to be an expectation that bugs will be found (probably by your users) and then fixed after the software is released.
The answer to the question is that ultimately you should pay. You should set aside some budget for post release bug fixing - that’s what a support contract is for.
9. How quickly will bugs in your bespoke software get fixed?
You need a support contract with your software supplier. As part of that you also need a service level agreement that defines the different priority levels (with resolution times) you will use to categorise bugs. Critical bugs - things that have a big negative impact on your business - should be fixed and released to live within 24 hours.
10. Do you understand how your data is backed up by your software supplier/hosting provider?
How often is your data backed up and what happens if things go wrong? Do you know how long it will take to restore the system from a database backup? In theory you’ll not lose data if your software/hosting provider does real time data replication - data that changes is copied to a separate database, ideally at another location.
From time to time you should ask your software/hosting provider to test a full restore of data. How long does it take?
11. Who owns your bespoke software?
By default, unless you’ve got a contract that says otherwise, your software developer owns your bespoke software. Switching software suppliers part way through a development or taking the development in-house may not be as easy as you think.
12. Do you understand a bit about the technology being used to develop your software?
Most people know if their house is made from wood or bricks and they’ve kind of got an idea of the advantages/disadvantages of each. You should ask your software developer to explain why their favoured technology stack is the right toolset for your project. Are there any risks? For example, will it be easy to find another supplier with the same skill set if you decide to part company with your original supplier at some point?