Business Analysis Notes

Techniques BABOK Guide v3, 10.49

Vendor Assessment

5 min read · Updated

In short Vendor assessment checks whether a supplier of a solution, or of part of one, can keep its commitments and contribute to the solution’s success over time. Combined with a scored comparison of products against your requirements, it gives you a defensible choice and a list of risks to cover in the contract.

Based on: IIBA, BABOK® Guide v3, technique 10.49 “Vendor Assessment”; Watermark Learning, BABOK v3 Study Tables (2016); K. Wiegers, J. Beatty, Software Requirements, 3rd ed., ch. 22.

On this page

What it is#

In the BABOK® Guide, vendor assessment evaluates third-party suppliers on their ability to meet their obligations and to support the solution’s success. Typical questions: is the vendor financially stable, can it keep enough staff, does it comply with the relevant standards, and will it provide qualified people to support the solution?

The BABOK Guide groups the assessment into five elements:

  • Knowledge and expertise: does the vendor know your domain and the technology, and can it transfer that knowledge to your people?
  • Licensing and pricing models: how costs grow with users, sites, or transactions, and what is included.
  • Vendor market position: leader, niche player, or newcomer; how many customers like you it serves.
  • Terms and conditions: service levels, support, data ownership and export, liability, and exit terms.
  • Vendor experience, reputation, and stability: track record, references, and financial health.

The vendor and the product are separate questions. A good product from a fragile vendor is a risk; a strong vendor whose product does not fit is a waste. For packaged (COTS) solutions, Wiegers and Beatty (ch. 22) give a repeatable method for the product side:

  1. Rate the importance of each of your requirements, from 1 to 10.
  2. Rate how well each candidate meets each requirement: 1 for fully, 0.5 for partly, 0 for not at all. Base the rating on documentation, answers to your request for proposal, and hands-on checks. Hands-on checks are essential for high-priority requirements, because written answers may not match reality.
  3. Total the weighted scores for each candidate.
  4. Separately assess cost, vendor reputation and support, external interfaces, and fit with technical constraints. Assess cost deliberately after the functional comparison, not mixed into it.

They add three practices: test each candidate against your most important use cases, including error conditions; have at least one person take part in every evaluation, so scores are interpreted the same way; and when there are many candidates, evaluate in stages, ending with a trial on real work rather than a demo.

When to use it#

  • Buying a packaged product or a hosted service, outsourcing development or operations, or choosing a supplier that becomes part of the solution.
  • In the BABOK Guide it supports Analyze Current State (6.1) and Define Future State (6.2), for current vendors and the potential value of vendor options; Define Change Strategy (6.4); Define Design Options (7.5), where a third-party solution and its vendor are assessed together; and Measure Solution Performance (8.1).

It lowers the risk of choosing the wrong vendor and supports a better long-term relationship. Its limits: assessing several vendors takes time, it cannot remove the risk that a vendor fails later, and assessments can be biased. Counter the bias by fixing criteria and weights before you meet vendors and by recording the evidence behind every score.

Scale the effort to the stakes. Wiegers and Beatty point out that the effort should match the cost and risk of the purchase, not the number of candidates; a cheap, easily replaced tool needs only a light check.

How to do it#

  1. Define the need at the level of use cases or user stories, with priorities. Skip detailed screen designs; the vendor has already designed those.
  2. Set product and vendor criteria and their weights before contacting vendors.
  3. Make a long list, then a shortlist using a few knock-out criteria.
  4. Request information or proposals, and ask for demonstrations of your own scenarios, not the standard demo.
  5. Score product fit and vendor criteria, recording the evidence for each score.
  6. Check references and financial health, and talk to the vendor’s technical staff, not only to sales.
  7. Assess total cost of ownership for the expected contract period, after the fit comparison.
  8. Trial the finalists with real data and real users.
  9. Recommend, listing the risks and the contract terms that mitigate them: service levels, data export, exit support.
  10. Reassess after go-live as part of measuring solution performance.

Example#

The Workshop, a fictional chain of bike-repair workshops, compares three fictional hosted booking services. Weights and scores are illustrative.

Requirement Importance Vendor A Vendor B Vendor C
Online booking per workshop 10 1 1 1
Status messages by text and email 9 1 0.5 1
Parts orders sent to the supplier 7 0 1 0.5
Owner’s report across all workshops 6 1 1 0.5
Weighted total 25 27.5 25.5

Vendor B fits best on paper, but it is a young company and charges extra for data export. Vendor A has a long track record and includes export, but lacks parts ordering. The analyst recommends a one-month trial of A and B in one workshop, and puts data export and exit assistance on the list of contract terms for whichever vendor wins.

Common mistakes#

  • Choosing by demo → ask for scripted demos of your scenarios and trial with real data.
  • Comparing on price first → compare fit first, then cost.
  • Setting criteria after meeting vendors → fix criteria and weights in advance.
  • Talking only to sales → involve the vendor’s technical specialists early, as Wiegers and Beatty advise.
  • Ignoring the exit → check data export, transition help, and termination terms.
  • Too many candidates or criteria → shortlist using the business objectives as a filter.
  • Leaving users out → involve them in the selection to reduce later rejection.

Template#

Template
Product fit
| Requirement / use case | Importance (1-10) | Vendor A (0 / 0.5 / 1) | Vendor B | Evidence |
|---|---|---|---|---|
| Weighted total | | | | |

Vendor profile
| Criterion | Vendor A | Vendor B | Evidence / source |
|---|---|---|---|
| Domain and technical expertise | | | |
| Licensing and pricing model | | | |
| Market position | | | |
| Terms: service levels, support, data export, exit | | | |
| Experience, references, financial stability | | | |
| Compliance with required standards | | | |
| Staffing and support capacity | | | |

Total cost of ownership (<n> years, <currency>): A <amount> | B <amount>
Key risks and contract mitigations: <list>
Recommendation: <vendor>, because <reasons>

Sources and further reading#

  • IIBA. A Guide to the Business Analysis Body of Knowledge (BABOK® Guide), version 3. International Institute of Business Analysis, 2015, technique 10.49 “Vendor Assessment”.
  • Watermark Learning. BABOK v3 Study Tables. 2016. (study aid for the BABOK Guide), technique 10.49.
  • K. Wiegers, J. Beatty. Software Requirements, 3rd ed. Microsoft Press, 2013, ch. 22 (evaluating and selecting packaged solutions).