A method of scoring options against a prioritize requirements list to determine which option best fits the selection criteria.
Note: This is the first in a two part series of articles describing the Weighted Scoring Model, a.k.a., Weighted Scoring Method - a technique under the category of Multi-Criteria Decision Analysis. The second article of the series describes some of the shortcomings or "holes" of this method.
The Weighted Scoring Method
Weighted Scoring is a technique for putting a semblance of objectivity into a subjective process. Using a consistent list of criteria, weighted according to the importance or priority of the criteria to the organization, a comparison of similar “products” can be completed. If numerical values are assigned to the criteria priorities and the ability of the product to meet a specific criterion, a “weighted” value can be derived. By summing the weighted values, the product most closely meeting the criteria can be determined.
Ok, that sounds a bit confusing, so let’s make it simpler.
When choosing between Product A, B or C, which product most closely matches your needs? For most people and organizations, they simply guess. “This one just seems to be the best.” “This is the number one product out there, so it must be good.” “But my brother-in-law sells this one and he tells me it is really great.” There is no objectivity or way to tell what is fact and which is fiction.
This Weighted Scoring Method can be use when selecting projects or anything where we must compare one item to another.
For example, when purchasing a new car, how do you pick the one you want? You might make a list of items the car must definitely have to be considered. Then you write down additional options you’d like to have. And you leave a few spaces to note features one car has the others don’t.
After trips to the various dealers, you tally up the list of matches and buy the one which meets the list the best. While you might not be this formal, you do it mentally. You are simply weighting some features and functions of the car of higher importance than others and if a car does not meet one of those important criteria, it is thrown out of the running.
When selecting technology for your organization, you probably have a lot more money and impacts on the business involved. A wrong decision can have dire consequences.
Here’s an example which describes how powerful this approach is.
We were called by a potential client who wanted to select some technology for their company. They had formed a taskforce from three different areas in the company. For the past 18 months, each area had different ideas as to the right technology to select. None were willing to compromise. They had actually become hostile towards one another.
Our job was to get this taskforce to consensus on a solution within two days. If we were successful, we got paid. If we failed, there would be no check in the mail.
Using a weighted scoring model and our prepared requirements list, we had the client review and weight the requirements’ priorities. Using the products selected by the three factions, we compared the functionality of each to the requirements and gave them a score. We multiplied the priority with the score to compute the weighted value, summed the weighted values and determined which product best fit their organization.
After careful analysis of the results, each member of the taskforce agreed we had our selection and to move forward with the implementation. Total time to consensus: 1.5 days. We collected our check.
The Weighted Scoring Method is best done in a spreadsheet where the requirements can be listed, a numerical priority entered, and the products to be compared recorded.
Here’s a screen shot of a typical spreadsheet format:
Column A: Requirement number uniquely identifies each requirement. Numbering can be strictly numeric or alpha-numeric to identify various identification indicating requirement’s business unit, initiative classification, budget account, etc.
Column B: Requirement description. The requirement description should be short, but descriptive enough so others understand the requirement. This may be tied to more complete descriptions in other documents via the requirement number.
Column C: Priority value assigned by the team for the requirement. Requirements should match key stakeholders’ expectations and requests. Priority values should be numeric. Values that work best are 0, 1, 3, and 5.
- 0 means requirement does not apply to this particular study or project. Philosophically, why would the list contain a requirement of 0? If the requirement list is standardized for a particular type of project such as project selection or even technology selection, then removing the requirement rather than rating it a zero may generate some questions later about it being missing.
- 1 means the requirement is of low importance at this time
- 3 means the requirement must be met
- 5 means the requirement is of high importance
Column D: Score First column of Project A or Product A or whatever is being compared. The product or project is scored against the requirement independently of the priority. Does the item meet or exceed the requirement? Does the requirement even show up on the radar for this item or is it not even an honorable mention? The scoring values are 0, 2, 4, and 6.
- 0 means the item doesn’t include the requirement at all
- 2 means the item may meet some of the requirement, but not all of it
- 4 means it meets the requirement
- 6 means the item exceeds the requirement
Column E: Weighted Score simply multiplies the priority with the score and the results become the weighted score.
Conduct the scoring for each requirement and let the spreadsheet calculate the weighted score. When all requirements have been considered for the candidates, simply add the weighted scores for each item compared. Again, the spreadsheet can do this easily. The product or project with the highest score most closely matches the prioritize requirements.
Caution: After picking the top three scorers, a review of the scores against the requirements is necessary. The top scorer may have some low or zero scores against high priority items. Just because the winner scored zero against a high priority requirement doesn’t knock it out of consideration necessarily. Knowing it doesn’t meet or exceed a requirement is crucial for future planning. Since its deficiency is known, work-arounds and other approaches can be taken or planned proactively rather than reactively when discovered later.
The Weighted Scoring Method is a powerful but flexible method of comparing similar items against a standard, prioritized list of requirements or criteria. We’ve used this method in less formal ways when buying personal items without even recognizing it.
It provides a level of objectivity in matters where subjectivity can have major negative consequences. It can be used for project and product selection, risk response analysis and solution design. The method described here has been proven in real-world scenario and is structured for efficient use of time for those involved.
In the end, the choice is yours. The weighted scoring model is simply a tool, a technique to help guide your decision making.
Plug the holes in this model for better decision making.
PMBOK® Guide is a registered trademark of the Project Management Institute. All materials are copyright © American Eagle Group. All rights reserved worldwide. Linking to posts is permitted. Copying posts is not.