A year back, I mind hirpling aroon the leid subs on Reddit an cam across somethin caad the CEFR. This is:-
The Common European Framework of Reference for Languages (CEFR) is an international standard for describing language ability.
This wis the first time I hid heard o sic a thin.
The CEFR organises language proficiency in six levels, A1 to C2, which can be regrouped into three broad levels: Basic User, Independent User and Proficient User, and that can be further subdivided according to the needs of the local context. The levels are defined through 'can-do' descriptors.
This isnae the same thin as the staunardisation o Scots or how it is taught in schuils, this is purely fae measurin.
There isnae a Scots language CEFR test, an there’s naebody wirkin on one. Ah drapped aff an email tae a couple of English GCSE exam boards an asked whit sort of steps wid be required tae put thegither a Scots CEFR test, and nane hid a clue. Cambridge Assessment International Education responded -
we do not offer the Scots Language Proficiency Test … recommend you contact the organization or entity that offers this test
Aye, Ah ken, thats ma point, there isnae a test.
Education Scotland administers yon Scottish exams an they dinnae hiv a CEFR test, there’s the SQA Scots Language Award, bit it isnae aboot proficiency. We micht feel that haein a proficiency test is jist normal couthie “language infrastructure”, the sort of trappings o normal European leids, French, Norwegian, Faroese.
Suppose the Scottish Census 2022, reported that 46% of the population unnerstuid Scots, and 22% cuid read and scrieve Scots, and this compelt newspapers and broadcasters tae recruit Scots leid reporters or Scots leid correspondents, fowk wha are able tae articulate different registers an styles o Scots screivins fae thins lik sports commentary or news reportin, or informational science explainin.
How wid a non-Scots spikkin organisation (like the BBC, STV, The Herald and Holyrood) distinguish atween twa candidates fae sic a role?
Lik, gin the BBC wantit a French correspondent or Japanese correspondent wha wir fluent in thae languages, the CEFR wid provide a framework.
Furthermore, ain o the objections tae the Scottish Census Scots leid question wis that naebody kent whither ane person answerin they could unnerstaun, spik, read and scrieve in Scots hud the same proficiency as someane wha cuidnae. Wis it eneuch tae jist ken a few wirds?
Is it impossible
A wee keek on the Facebook Scots Leid steid suggestit that it widnae be possible tae test, wioot staunardisation -
“which Scots would the questions be in, Doric (costal, city, rural)? Shetland? Orkney? Fife? Dundee? East coast? West coast? North? South?”
I dinnae believe it fae a moment, this is a wee difficulty that cin be easy owrecome.
There maun be some wey tae distinguish atween someone wha is well-versed in all byleids, someone wha is verse-welled in ainly their ain local byleid, an thae monoglot English-spikkers wha cin muddle throu, the sort ae chap wha chuffs on the Twitter that they cin unnerstaun it fine wioot studyin it at schuil.
At this point we maun consider that there are already online Scots leid tests. I tried yin an it wir keech, wi spellins that arnae fund in the corpus ae actual contemporary scrievers - “tuithpaste”, “thenkfu”, “doacter”, “sweetcoarn”. Thae are jist wirds wi some made-up orthography uised by nane.
Whit wid it luik lik
I would probably figure out how to render it in PHP, or using Google forms, but conceptually, we might imagine a deck of question cards, about a thousand of them, maybe more. Each card has a question with multiple choice answers, taking various forms - "select the missing word", some would be "select the most appropriate response", a picture round maybe, and some would be "which answer is wrong", or “which answer is written in Orcadian”, “which answer would be most widely understood”.
The cards are written in a wide variety of regional varieties. Some of them are easy, an English monoglot could figure out the answer, some might be easy for Doric speakers, but hard for Shetlanders. Some might depend on knowledge of grammar and some might depend on vocabulary alone.
We would shuffle the deck of cards, and then test them with several sample groups:-
English monoglots
L2 English speakers (might have learnt English overseas)
People from each Scottish region
People who reported they could speak Scots but not write it
People who we believe are C2 omniscient Scots speakers, professors etc,
Then after sufficient cards have been tested, we can analyse the results. Perhaps some cards can just go straight in the bin. And the surviving cards can have a weighted scoring system applied.
The test is merely to distinguish people into people who don’t understand Scots, and then six levels of proficiency.
A first attempt
Ah’ve hid a go on Google Forms noo wi aicht questions, feel free tae hiv a gae.
The text o each question is taen fae some owreput Scots literature, foond in ma Corpus of 21st Century Scots Texts -
The chiel wantit tae gie his **** a wee bosie tae mak her feel better. (1 point)
(a) hoose
(b) forby
(c) quine
(d) ken'I jist dinna ****,' she said. 'It micht be onybody.' (1 point)
(a) ken
(b) know
(c) lunch
(d) bosieYe can look doun at aa **** boys near the baur. (1 point)
(a) thee
(b) thoa
(c) thae
(d) those
The first tranche of folk haein a go o the test aw did gey well, some initially claimin tae hiv the best C2 grade proficiency, an getting almaist aw o the questions correct. We micht expect an L1 native Scots spikker and reader tae be C1 or C2.
The answers that people got wrong were mostly with homophones or where non-standard spellings were perceived as “wrang”.
Correcting wrongs
So this has identified an issue. Ideally, someone with C2 grade skills would just handle a non-standard spelling without difficulty, maybe accepting that it indicates a specific regional style or the writer’s fondness for archaic spellings, but otherwise the meaning of the text shines through.
But suppose two answer options are given one that “standardises” the text and one that corrects a factual phrase, which option should our C2 candidate chose? Should the candidate be penalised for choosing between two equally perceived “wrongs”.
The easy response is to just eliminate any ambiguous answer options. But does this make the test two easy?
We want people to get answers wrong in order to establish the grading.
Who’s authority
At this point I wonder who would be in charge of the CEFR test? Is it up to the Scottish government? Or some examination board? Perhaps a Scottish university’s language department? Perhaps some private company or individual knocking up a website.
At the moment no one is working on this. No one cares, there’s no money in it, there’s no funding for it.