Detailed Assessor for an ARC application?

The assessment of ARC applications was discussed at the recent Australian Education Research Leaders’ Summit. Informal advice from current and previous ARC College of Experts indicates that Education could do better in the way we provide assessments. For example, many Australian Education researchers provide formative feedback on ARC assessment reports, confusing applicants, and not delivering the clarity required.

The ARC requires assessors to provide well-targeted summative feedback with respect to the assessment criteria. For example, assessors are required to provide ‘Detailed comments on the merits or otherwise of the application with respect to the selection criteria’.

There are a number of resources available to guide assessors here. Of particular relevance are the guidelines for writing a quality peer review and examples available here. 

Ensure comments align closely with scores that we give against the criteria. For example, if a grant application is outstanding, explain clearly why it is outstanding – this helps the College of Experts to make positive decisions. The Assessment criteria and Scoring Matrix for Discovery Projects below, gives an indication of the language to use for each score.

Assessment criterion	(A)OutstandingOf the highest quality and at the forefront of research in the field. Approximately 10% of Applications should receive scores in this band.	(B)ExcellentOf high quality and strongly competitive. Approximately 15% of Applications should receive scores in this band.	(C)Very Good Interesting, sound and compelling. Approximately 20% of Applications should receive scores in this band.	(D)GoodSound, but lacks a compelling element. Approximately 35% of Applications are likely to fall into this band.	(E)UncompetitiveHas significant weaknesses. Approximately 20% of Applications are likely to fall into this band.

The Summit also emphasised the need for assessors to take care when evaluating methodologies with which they are unfamiliar, to ensure applications are not unfairly disadvantaged.