The good, the bad and the ugly policies
Things have been moving fast since I started this blog. Six months ago, there was almost no AI policy other than a few statements that warned students that using ChatGPT might be plagiarism.
I have recently completed work as part of the “AI: Policy and Procedures” and “AI: Basic Education” working groups at my institution, and I hope the outcomes will be shared with the public in due course. Of course, other institutions have put similar initiatives in place.
Therefore, as of today there have been several policies, guidelines, principles and statements put out by universities and colleges across the world. Let’s look at some of them, trying to capture the different approaches that have been implemented to face the monumental challenge of dealing with generative AI.
Russell Group
On the 4th of July, Russell Group universities published a short statement outlining five principles on the use of AI in higher education:
Universities will support students and staff to become AI-literate.
Staff should be equipped to support students to use generative AI tools effectively and appropriately in their learning experience.
Universities will adapt teaching and assessment to incorporate the ethical use of generative AI and support equal access.
Universities will ensure academic rigour and integrity is upheld.
Universities will work collaboratively to share best practice as the technology and its application in education evolves.
In the full release, each principle is then articulated in several paragraphs that look at various aspects, but I want to highlight two aspects.
1) AI is inevitable: staff must do more than just mitigate its impact or safeguard certain aspects of the degree. Universities are urged to adapt what they do to incorporate AI and support access. In other words, students must learn to use AI critically and ethically - this is an intended learning objective now. And I cannot agree more: AI enhances productivity and performance, and student grades are scaled in the UK: as I mentioned previously, with typical Moloch dynamic students that reject AI would see their grades drop because everyone else is doing better.
2) How must academic rigour and integrity be upheld? This is by far the shortest section.
Only two things are made explicit: the first is that policies should be clear and transparent. In other words, students must know exactly what is inappropriate, what constitutes poor practice, and what constitutes malpractice. It is the job of course leaders to tell each student what they can and cannot do using AI, which is itself a huge task. The perception of AI among academics varies wildly, and there is no consensus, especially on malpractice: these principles recognise that, and answer by allowing different approaches, as long as they are clear, transparent, and established by AI-literate staff.
The second is that this must be a scenario where students should be free to ask questions and receive answers without fear. Again, this embraces the uncertainty of this transition phase - policies should be as clear and transparent as possible, but when questions arise, it is good to deal with them with the same clarity and transparency.
The rest is demanded to policies that, as far as I can tell, have not been extensively reviewed yet as each university, faculty, school, department figures out their approach.
Stanford
This policy was published back in February and was, as far as I am aware, one of the first attempts at a generative AI policy. This is a policy of uncertainty and freedom, whose main aim is to safeguard some structure in what was (and still is) a chaotic situation.
It starts by saying that “AI shall be treated analogously to assistance from another person”, but only “absent a clear statement from a course instructor” and, moreover, even then students should not “avoid” but “default to disclosing such assistance when in doubt”. It then follows up by saying that “Individual course instructors are free to set their own policies regulating the use of generative AI tools in their courses”.
There is only one real limit imposed in the policy: using generative AI tools to substantially complete an assignment or exam (e.g. by entering exam or assignment questions) is not permitted. The word “substantially” is doing the heavy lifting here so, essentially, everything is again left to the judgement and initiative of each instructor.
The main flaw in such a policy is that it can only work with a robust program of staff training. AI is new: there can’t be any reasonable expectation for academic staff to already be AI-literate. Generative AI is a “hot topic”, meaning that the internet is full of noise and even clear misconceptions can make it to the front page of a respectable newspaper. So, with such a policy different academics might take opposite views and approaches, and students are left to deal with a puzzle of vague policies, not knowing what is right.
Boston
I have seen many policies with many issues, but there is one that stood out. I am including it here as the archetype of what I’d recommend avoiding (and I am really sorry for the students and staff working there). The silver lining in someone actually using such a policy is that I expect we will soon have explicit case studies of this backfiring. I include Wayback Machine links instead of direct ones because I do not expect this policy to last long in its current form.
This is a policy on using AI in coursework, which I imagine was largely intended to safeguard the “asynchronous unsupervised essay” format of assessment. The policy asks that:
[students] add an appendix showing (a) the entire exchange
We have already mentioned that this is a narrow view of generative AI. As soon as the technology becomes more integrated, it is likely that there will be no “exchange” to show. If someone asked you to include evidence of your usage of autocorrect in text messages, what would you submit? Further, any good usage of AI to co-create content will be made of hundreds of messages. Students might wish for some of those to remain private.
[students] employ AI detection tools and originality checks prior to submission, ensuring that their submitted work is not mistakenly flagged.
This policy asks students to venture out and find AI detection tools, submit their original work, and ensure that it is not mistakenly flagged. I am not sure I need to make explicit why this is a terrible policy, but just in case:
Detectors do not work well and they’ll become worse as AI improves. Even if you disagree, you can probably meet me in the middle if I state that “there is no proof that AI detection is reliable”.
There is no list of detectors in the policy! Some detectors are behind a paywall, and staff could decide to use some new or exotic detector. A student can never be sure their work won’t be flagged.
A student that has written original work without even thinking about AI could now be forced to do some editing because some coin-flip detector on the internet flags it. There is evidence that this would disproportionally affect non-native English speakers.
Most AI detectors on the internet have no privacy policy, or have one in which you basically forfeit any copyright and void any confidentiality on any content you submit.
[instructors shall] employ AI detection tools to evaluate the degree to which AI tools have likely been employed.
Dually, staff cannot even disengage from the detection approach. They are actually required to use the tools and act accordingly.
I’ve genuinely only seen this kind of policy in novels by Kafka, perhaps in movies about witch trials.
Some instructors may prefer stronger restrictions on the use of AI tools and they are free to impose them so long as care is taken to maintain transparency and fairness in grading.
Again, all this does is harm honest/compliant students via Moloch. Once it is accepted that AI usage cannot be detected, any blanket ban is equivalent to giving an advantage to dishonest students that will ignore it and obtain better grades.
More policies
The Russell Group principles paint a picture where general common principles are agreed upon, good practice is shared, and then each course leader is free to develop their own approaches. I believe that this represents a good balance between Stanford (fully decentralised) and Boston (fully centralised), and ultimately the correct approach, if there is one.
Of course, the three namesakes of this post do not represent all that can be done. Making extensive use of this excellent resource by Lance Eaton, let’s look at a few different ideas and approaches found in other policies:
Percentages
Any student work submitted using AI tools should clearly indicate what work is the student’s work and what part is generated by the AI. In such cases, no more than 25% of the student work should be generated by AI. (Colorado University)
This has the same issue as “share your prompt” policies. Proper usage of AI, as a “copilot”, makes it impossibile to determine how much of the work was done by the human or the AI. A typical example would be asking the AI to write a paragraph for an essay, articulating what the student is telling it verbally. How do percentages work here? Policies never specify, just mandate.
If we assume, charitably, that co-created content is excluded… then I ask: is it really okay for 25% of an essay to be fully written by AI?
Full ban
Students are prohibited from using any generative AI tools such as ChatGPT, Bing AI, or Bard when completing course assignments. Use of these tools, or other similar generative AI tools, will not be tolerated and will be considered plagiarism and could result in the student failing the course. (Oklahoma State University)
We have argued against this repeatedly - in the absence of reliable AI detection, all this does is punish students who comply. Quite often this kind of policy states that AI is plagiarism - I’d argue it is instead some kind of weak contract cheating, since content written by AI is new.
At your own risk
Within this class, you are welcome to use foundation models (ChatGPT, GPT, DALL-E, Stable Diffusion, Midjourney, GitHub Copilot, and anything after) in a totally unrestricted fashion, for any purpose, at no penalty. However, you should note that all large language models still have a tendency to make up incorrect facts and fake citations, code generation models have a tendency to produce inaccurate outputs, and image generation models can occasionally come up with highly offensive products. You will be responsible for any inaccurate, biased, offensive, or otherwise unethical content you submit regardless of whether it originally comes from you or a foundation model. […] the use of foundation models is encouraged, as it may make it possible for you to submit assignments with higher quality, in less time. (University of Pennsylvania)
This is the kind of policy that I would have loved to have if I were still a student. However, it comes with risks: how can the lecturer be sure that assessments remain authentic, and that students indeed learn what they are supposed to learn? AI evolves quickly and prompt engineering can enhance the abilities of existing models - it is hard to say with certainty that AI cannot complete an assessment without any meaningful input from the human using it.
Looking ahead
The use of generative AI tools is not permitted in this course for the following activities: Impersonating you in classroom contexts, such as by using the tool to compose discussion board prompts assigned to you or content that you put into a Zoom chat. (Temple University CAT)
I am including this one because of this sentence. Student impersonation by AI will surely be a problem, but I am sceptical it is one right now. Better safe than unable to sanction…
Copilot, not pilot
AI as Support, Not Replacement: AI tools should augment the learning process, not replace original thinking. While these tools can support idea generation, fact-checking, or language revision, they must not substitute the individual's critical thinking, problem-solving skills, and thought process. Students should consider AI a tool for enhancement and refinement, but the essence of the work must come from their intellectual effort. (Warner Pacific University)
In my opinion, this is mostly the correct way of thinking about good usage of AI, and it is great to see it included in policies, because it enables students to use AI for all permitted usage without major worries about bad consequences. Of course, it must be done in a context that minimises the misdetection of good AI usage as bad AI usage.
Banned unless it’s not
If your module leader has given you instructions on the use of AI for a particular assessment, you should follow these. In the absence of any such specific instructions, the key point is that when you hand in work for an assessment, you are stating that it is your own original work. (City University)
This is a good interim policy which, essentially, gives some ground to tackle the obvious cases of academic malpractice while encouraging staff to adapt their courses and give guidance to students. However, this kind of policy will soon be obsolete: as soon as generative AI is embedded in tools like word processors, or accessibility ones, that kind of usage cannot be part of a blanket ban. It would be like asking me to take an exam without my glasses (I have bad vision).
Today is only yesterday’s tomorrow
As of now, and for the foreseeable future, submission of content that is partly AI generated is not permitted for any of your assessments, unless you have been explicitly instructed to do so in the assessment brief. (Saint George's University of London)
I highlight this one because it clarifies in its wording that it is an interim policy. Why might that be important? Well, because degrees are short. A student starting next year might finish their degree and enter a job market where proficiency in the usage of generative AI is expected. This is a framework where we would like the student to use AI, but we need time to figure out how to make that happen without undermining academic integrity and the authenticity of assessment. An interim ban with exceptions is a sensible policy, but it is crucial that this is made explicit.
What about this post?
I use AI in most things I do1, with the “copilot” approach. This means that AI is used to brainstorm, refine writing, spellcheck, reformulate, articulate oral speech into writing, generate pictures, translate, suggest improvements, and probably more things. It is not used to generate ideas (I know what I want to say when I start) or find source material (only because it’s bad at it).
I think that this is good usage, in the sense that I’d be fine if my students used AI in this way. How would the various policies apply?
Russell: A clear yes.
Stanford: I’d say yes. The use of AI was not “substantial” in any reasonable meaning of the word.
Boston: No, it’s weeping and gnashing of teeth for me.
Colorado: Unclear, I have no way to quantify the AI’s contribution to each sentence.
Oklahoma: Obviously not.
Pennsylvania: Obviously yes.
Warner Pacific: Yes.
City University: No.
Saint George: No.
Except of course for those things where AI usage would be illegal, immoral, or against a policy or guidance. For those things, I do not use AI.