A method of scoring options or solutions against a prioritize requirements list to determine which option best fits the selection criteria.
Note: This is the first in a two part series of articles describing the Weighted Scoring Model, a.k.a., Weighted Scoring Method - a technique under the category of Multi-Criteria Decision Analysis. The second article of the series describes some of the shortcomings or "holes" of this method.
The Weighted Scoring Method
Weighted Scoring is a technique for putting a semblance of objectivity into a subjective process. Using a consistent list of criteria, weighted according to the importance or priority of the criteria to the organization, a comparison of similar “solutions” can be completed. If numerical values are assigned to the criteria priorities and the ability of the product to meet a specific criterion, a “weighted” value can be derived. By summing the weighted values, the product most closely meeting the criteria can be determined.
Ok, that sounds a bit confusing, so let’s make it simpler.
When choosing between Product A, B or C, which product most closely matches your needs? For most people and organizations, they simply guess. “This one just seems to be the best.” “This is the number one product out there, so it must be good.” “But my brother-in-law sells this one and he tells me it is really great.” There is no objectivity or way to tell what is fact and which is fiction.
This Weighted Scoring Method can be use when selecting projects or anything where we must compare one item to another.
For example, when purchasing a new car, how do you pick the one you want? You might make a list of items the car must definitely have to be considered. Then you write down additional options you’d like to have. And you leave a few spaces to note features one car has the others don’t.
After trips to the various dealers, you tally up the list of matches and buy the one which meets the list the best. While you might not be this formal, you do it mentally. You are simply weighting some features and functions of the car of higher importance than others and if a car does not meet one of those important criteria, it is thrown out of the running.
When selecting technology for your organization or projects in your portfolio, you probably have a lot more money and impacts on the business involved. A wrong decision can have dire consequences.
Here’s an example which describes how powerful this approach is.
We were called by a potential client who wanted to select some technology for their company. They had formed a taskforce from three different areas in the company. For the past 18 months, each area had different ideas as to the right technology to select. None were willing to compromise. They had actually become hostile towards one another.
Our job was to get this taskforce to consensus on a solution within two days. If we were successful, we got paid. If we failed, there would be no check in the mail.
Using a weighted scoring model and our prepared requirements list, we had the client review and weight the requirements’ priorities. Using the products selected by the three factions, we compared the functionality of each to the requirements and gave them a score. We multiplied the priority with the score to compute the weighted value, summed the weighted values and determined which product best fit their organization.
After careful analysis of the results, each member of the taskforce agreed we had our selection and to move forward with the implementation. Total time to consensus: 1.5 days. We collected our check.
The Weighted Scoring Method is best done in a spreadsheet where the requirements can be listed, a priority entered, and the products to be compared recorded.
Here’s a screen shot of a typical spreadsheet format:
|Simplified Weighted Scoring Template|
Column A: Requirement number uniquely identifies each requirement. Numbering can be strictly numeric or alpha-numeric to identify various identification indicating requirement’s business unit, initiative classification, budget account, etc.
Column B: Requirement description. The requirement description should be short, but descriptive enough so others understand the requirement. This may be tied to more complete descriptions in other documents via the requirement number.
Column C: Priority value assigned by the team for the requirement. Requirements should match key stakeholders’ expectations and requests.
Priority values are the backbone of this method. They should align with the company's objectives and goals, be set and agreed to by the team of key stakeholders, and can be either numeric (cardinal scale) or descriptive (ordinal scale). Let's describe both formats for clarity.
Numeric or Cardinal Priority Scale
Priority values can be numeric. Values that work best are 0, 1, 3, and 5.
- 0 means the requirement or criterion does not apply to this particular study or project. Philosophically, why would the list contain a requirement of 0? Once built, the template used for product, project or technology selection can be re-used for many purposes. The standard set of criteria may contain some values that are not applicable for this particular application. If the requirement list is standardized, then removing the requirement rather than rating it a zero may generate some questions later about it being missing. The 0 documents it was purposely removed from the rating.
- 1 means the requirement is of low importance at this time.
- 3 means the requirement must be met.
- 5 means the requirement is essential.
Descriptive or Ordinal Priority Scale
Over the years, we have grown fond of a descriptive scale. With the numeric scale, we had to remind the team repeatedly 5 was the highest value while 1 was the lowest. Using a descriptive scale, we eliminate the confusion. Since we use Microsoft Excel to implement the rating system, we can easily translate to a numerical value so the scoring can be easily calculated. Here is the scale we like to use with their corresponding numerical value:
- Not Applicable (0): the requirement or criterion does not apply to this study.
- Nice to Have (1): the requirement is lower in need and is nice to have. If the solution or option being score contains it, it is a bonus point.
- Important (3): the requirement is important, the solution or option should contain it. If the option does not, it will impact the score.
- Essential (5): the solution must contain this requirement. If the solution or option doesn't at least meet the requirement, the lack must be highlighted, discussed and resolved for it to be considered viable.
The scoring values are No Support (0), Supports Criterion (2), Meets Criterion (4), and Exceeds Criterion (6).
- No Support (0) means the item doesn't include the requirement at all.
- Supports Criterion (2) means the item may meet some of the requirement, but not all of it.
- Meets Criterion (4) means it meets the requirement.
- Exceeds Criterion (6) means the item exceeds the requirement.
Column E: Weighted Score simply multiplies the priority with the score and the results become the weighted score.
Conduct the scoring for each requirement as a team and let the spreadsheet calculate the weighted score. When all requirements have been considered for the candidates, simply add the weighted scores for each item compared. Again, the spreadsheet can do this easily. The product or project with the highest score most closely matches the prioritize requirements.
Caution: After picking the top three scorers, a review of the scores against the requirements is necessary. The top scorer may have some low or zero scores against high priority items. Just because the winner scored zero against a high priority requirement doesn't knock it out of consideration necessarily. Knowing it doesn't meet or exceed a requirement is crucial for future planning. Since its deficiency is known, work-arounds and other approaches can be taken or planned proactively rather than retroactively when discovered later.
See our article, Plugging the Holes in the Weighted Scoring Model, to learn about the deficiencies of the Weighted Scoring Model and how to overcome them.
The Weighted Scoring Method is a powerful but flexible method of comparing similar items against a standard, prioritized list of requirements or criteria. We've used this method in less formal ways when buying personal items without even recognizing it.
It provides a level of objectivity in matters where subjectivity can have major negative consequences. It can be used for technology, project and product selection, risk response analysis and solution design. The method described here has been proven in real-world scenario and is structured for efficient use of time for those involved.
In the end, the choice is yours. The weighted scoring model is simply a tool, a technique to help guide your decision making.
Plug the holes in this model for better decision making.
PMBOK® Guide is a registered trademark of the Project Management Institute. All materials are copyright © American Eagle Group. All rights reserved worldwide. Linking to posts is permitted. Copying posts is not.