MERLIN PROJECT 3.1 PROFESSIONAL

If you are interested in more details regarding the quality of the ratings and the difficulty of the single rating criteria, please consult the technical report. If you compile your own corpus based on CEFR levels, these are also based on the fair average ratings ( Define a subcorpus » Overall CEFR rating). In MERLIN, the fair average is calculated based on a holistic scale (see 1.2 rating instruments). Therefore, the whole re-rating process was repeated for Italian resulting in a satisfying rating quality. The intra-rater and inter-rater reliability was generally very high in MERLIN, with some exceptions for Italian. The latter is a probabilistic statistical procedure often used in language testing which allows for a correction of rating tendencies (e.g., leniency/harshness) and makes it possible to arrive at a fair average rating for each text. The reliability of the re-ratings was examined with the help of Classical Test Theory and a Multi-Facet Rasch analysis.

MERLIN PROJECT 3.1 PROFESSIONAL

However, to have explicit and direct information about the CEFR profiles of the written productions themselves (and not only of the tests as a whole), for MERLIN all texts were re-rated independently by two professional raters per language. The tasks were in use until 2013 and are now freely available on the platform. These institutions are ALTE-audited ( ALTE-homepage). The MERLIN texts are the writings sections of CEFR-related, standardized high-quality tests from telc (Frankfurt/Main, Italian and German tests, homepage) and ÚJOP (Prague, Czech tests, homepage).

Merlin project 3.1