LLMs and students’ writing: Thinking it through

I’m just sharing my experiences and thoughts because I think it’s an important consideration for using large language models (LLMs), e.g. Google’s Gemini, OpenAI’s ChatGPT in language learning & teaching.
It’s interesting to see people’s attitudes to AI generated content. I experimented with using an LLM to generate “thoughtful” responses to the content (ideas & opinions) of my students’ writing. The idea was to cultivate more interest in making meaning and using language as a tool for mediating relationships (between writer and reader) so as to put the emphasis on more conscious and purposeful language use, i.e. The writer thinks in terms of, “I want to have this effect on my readers.”
However, me being me, I had to do this ethically and I was clear about what I was doing and why with my students. They initially loved the idea of getting responses to their writing but I could very visibly see their disappointment when it sunk in that I hadn’t actually written the responses. They weren’t at all interested in what a machine had to say about their thoughts and ideas.
Putting it into words like this, it becomes overtly clear that there’s a strong contradiction between intent (meaningful purposeful language use) and action (using a machine to respond to meaningful, purposeful writing). In my haste to find interesting new uses for AI and LLMs, I hadn’t thought through what it was that I was actually doing.
I think this is a big issue with AI generated content, not just for education: I reckon what we’re really interested in, at a fundamental level, is sharing our thoughts, ideas, preferences, and interests with each other, and even with people we’ll never meet or may be long dead. Language is the medium and the tool we use to do that and the resulting texts are merely byproducts of that social engagement, i.e. it’s more about the process than the product.
I think keeping the language use processes, interactions, etc., as “human” and as socially meaningful as possible is a highly desirable and motivating part of learning a foreign language, and that it’s easy to lose sight of that in our deliberations and efforts to help our students develop the language and skills they need to do that.