Published criteria do two jobs. They tell applicants what to write, and they tell assessors what to reward. When a round goes wrong, it is usually because the criteria did the first job and not the second: applicants wrote to them faithfully, and the panel then scored on something else.
Too many criteria is the default failure
Nine criteria at eleven per cent each is not a framework, it is a way of avoiding a decision about what matters. Assessors cannot hold nine dimensions apart, so they form an overall impression and distribute scores to match it. The rubric then documents a judgement it did not produce.
Four to six criteria is the range where the weightings actually change outcomes. If one of them is worth less than ten per cent, it is not a criterion. It is either an eligibility requirement or a note.
If a criterion is worth less than ten per cent, it is not a criterion.
Separate eligibility from merit
A surprising amount of assessment effort is spent scoring things that are actually pass or fail. Whether the applicant is incorporated, whether the project is inside the funding area, whether the budget is within the cap: none of these are merit questions, and scoring them wastes panel time and distorts the totals.
Move them to eligibility, screen them before an applicant starts writing, and the panel reads a smaller pile of applications that all deserve reading.
Write the anchors, not just the labels
A criterion called “Impact” scored out of ten means ten different things to ten assessors. Anchors fix that: say what a 2 looks like, what a 5 looks like and what a 9 looks like, in the language of the applications you actually receive.
- Anchors reduce the spread between assessors more than any amount of briefing does.
- They make moderation a conversation about evidence rather than about generosity.
- They give declined applicants feedback that is specific enough to act on next round.
Check the spread, not just the total
Two assessors scoring 8 and 2 average to the same 5 as two scoring 5 and 5, and the two situations could not be more different. A convenor who can see the variance can send the first pair back to talk and leave the second pair alone. A convenor who sees only the average cannot.