Categories
H5P Learning Moodle Multimedia

Online ELT & CALL activities in Moodle

H5P

Great news! For those of us who use Moodle to create online English language learning/teaching (ELL/ELT) activities & courses, things just got better. Previously, I’ve held off from writing about H5P because, although it is very useful & has great potential, I found the Moodle integration problematic & difficult to manage, but with the latest Moodle upgrades I think that we can take online ELL/ELT to a whole new level.

H5P Integration

Moodle 3.9 now has much tighter integration with H5P, a framework for creating & deploying 43 types of multimedia learning resources & activities, many of which are ideal for computer assisted language learning (CALL), e.g. dictations, learners can record themselves speaking, various matching & memory games, flashcards, & various cloze & gap-fill activity types, & an experimental feature that allows a speech analysis service to grade learners’ spoken responses to prompts & questions.

H5P resources & learning activities can be embedded anywhere in Moodle, e.g. as the initial prompt or activity in a forum discussion thread, in Moodle quizzes & tests, & in lessons & presentations.

Institutionally supported free & open source

Like Moodle, H5P is free & open source software (FOSS) & supported by several universities & non-profit organisations. This means that continuing development & adoption are assured & the community of developers & users is set to continue to grow substantially. They’re also building a free & open H5P learning resources repository/library so that course creators can publish & import ready-made learning resources & activities into their Moodle courses & edit & adapt them to their specific situation, context, & learners’ needs. This kind of collectivising learning resources certainly helps to lighten the typically substantial load of materials writing & development for individual educational organisations, thereby bringing costs & development times down.

Check out some examples

You can see & try out examples of the activity types, not specifically designed for ELT but they give you a good idea of what they look like & how they work. See the H5P resource & activity types summary page here: https://h5p.org/content-types-and-applications

Especially useful for CALL

Of the 43 H5P resource & activity types currently available, the following are of notable interest for CALL:

  • Interactive video: Useful for socially situated (contextualised) dialogues, & watching recorded language presentations with interactive prompts, pauses, & questions, as well as many other applications. You can really get creative with this one.
  • Presentations: Collate & sequence multiple H5P resources & learning activities into complete coherent mini-lessons. H5P items can also be sequenced & mixed in with Moodle quiz items.
  • Audio recorder: Ss can record themselves & submit it to their teacher for review, feedback, &/or grading – Also note that Moodle itself has audio & video recording capabilities so that you can, for example, create voice &/or video discussion forums (similar to VoiceThread). Learners just click, talk to their computer/phone, & submit.
  • Multimedia flash cards: Flash cards with text, images, & audio. Great for reviewing language in terms of meaning, form, & pronunciation (MFP). Flash cards also strengthen memory of basic language elements, which results in faster recall & more fluent, complex, & accurate spontaneous speech & writing (similar to Quizlet but with better multimedia support).
  • Dictation: Traditional dictation activity, except that learners control the audio playback (as many times as they need/like) & there’s option to provide additional versions of each section of speech, e.g. reduced tempo (time-stretched audio) or clearer, more emphatic annunciation. Learners’ responses are graded as % of correct words – each correct word receives a grade rather than the typical whole answer being graded in a binary (exactly) right or wrong fashion.
  • Drag & drop: Multimedia multiple matching activity – Learners can match text, images, & audio. It has a wide range of applications from vocabulary to dialogue sequences & TPR (be aware that drag & drop has issues with accessibility & section 508 compliance).
  • Drag & drop cloze (gap-fill): Learners have list of language items, e.g. words, to insert into correct positions in text. Similar to multiple choice cloze but quicker & easier to create & easier for learners to complete. Especially suitable for lower-level learners, i.e. CEFR A0 – A2.
  • Cloze tests (gap-fill): Learners type in the blanked words into a text. Although Moodle’s Quiz activity also does cloze deletion tests well, the text formatting & input in this H5P activity type is easier to read & therefore exerts lower extraneous (i.e. bad) cognitive load. An advanced cloze type allows for multiword blanks & gives percentage scores for the number of correct words within a blank, which is useful for practising Cambridge keyword sentence transformations. (BTW, cloze deletion texts are among the most valid & reliable forms of reading comprehension test known).
  • Hotspots: Learners identify items in an image by clicking on them. Can be used to teach & test vocabulary & phrases in contextually relevant images/scenes.
  • Memory game/match the pairs of cards: The classic memory games that learners seem to love & help to strengthen retrieval, fluency, & accuracy in spontaneous language production.
  • Image sequencing: Drag & drop the images into the correct order. Good for TPR style activities, e.g. to check initial/gist understanding of narratives, stories, news items.
  • Mark the words: Indicate words in a text, e.g. nouns, pronouns, objects, subjects, noun verb phrases, adjacency pairs, mistakes, etc.. Good for getting learners to do analyses of texts &/or error analysis/proof reading practice.
  • Personality quiz: A good ice-breaker/getting to know you activity for fun & cultivating social presence.

What other Moodle activities do better than H5P

H5P also has multiple choice & multiple matching question types however it’s better to do MCQs with Moodle’s Quiz module. The Quiz module has more options & it includes automated item analyses to help you to improve the quality of MCQ items at a granular level, e.g.

  • Facility index: How easy the learners find the item),
  • Discrimination index: How well the item distinguishes between low & high knowledge learners, i.e. the learners’ level of understanding),
  • Distractor efficiency: How feasible/convincing the wrong answers (distractors) are & how consistently proficient learners get an item right & low-knowledge learners get the item wrong – Contributes to the discrimination index.

Also see Moodle’s Feedback activity for opinion polls & different kinds of feedback, which allows teachers to share the aggregated results with learners, e.g. Opinion polls can be used to spark forum discussions as they show learners where consensus’ & differences on a specific topic are across the whole group.

Over to you

As you can see, there’s a long list of options to explore that can strongly enhance the instructional quality of online learning interactions & instructional sequences, as well as making them more varied & enjoyable for learners. What online ELT ideas do you think you could implement with these tools?

Categories
H5P Learning Moodle Multimedia

Monitoring & supporting students during online synchronous lessons

GDocs Multipage

Introduction

In this article, I have a suggestion for ways to improve the quality of English language teaching (ELT) via webinar apps, e.g. BigBlueButton, Blackboard Collaborate, Adobe Connect, Cisco WebEx, or Zoom, while making minimal changes to learning & teaching as they are currently being practised as emergency remote teaching due to the COVID-19 pandemic & subsequent shelter-in-place measures.

During this time, an extraordinarily high number of teachers & students have been continuing their face-to-face classes online, possibly the highest number in the history of distance education. Now that so many more people have become familiar with at least some aspects of learning & teaching online, I think it’s likely that its flexibility & convenience can be better understood &, with better planning & preparation, will have an important role to play in enhancing ELT in the foreseeable future.

By now, many teachers will have been using webinar apps for several weeks to host remote online lessons for their students. They’ve more than likely discovered that they’re not entirely appropriate for delivering face-to-face classroom style lessons & that they’ve had to adapt their teaching styles, lesson planning, & re-design their learning activities, staging, etc..

A particular issue with webinar platforms is that they’re primarily designed to be a one-to-many presentation tool. Although teachers can elicit responses from students, which corresponds to whole-class concept checking questions, eliciting examples, etc., in the classroom, it is difficult for teachers to monitor how well individual students may be doing during a learning task, i.e. the online remote equivalent of walking around the face-to-face classroom & looking at students’ work & giving them guidance/feedback while they attempt learning tasks. It’s an essential part of effective teaching whereby the teacher sees what & how each student is doing, corrects misunderstandings, clarifies instructions, fills in gaps in students’ knowledge, & gives personalised guidance & feedback. The following is a way to use existing free web tools/services to facilitate & augment this kind of monitoring.

Useful tools for monitoring students

Synchronised, online document editing platforms, e.g. Collabora & Google Docs (GDocs), enable teachers & students to simultaneously view & edit office documents in real-time, e.g. word processor, spreadsheet, & slide show presentation documents. They include collaboration & feedback tools such as text chat windows, highlighting, & commenting, all of which make them suitable for some types of learning tasks in remote online learning & teaching on webinars. However, monitoring groups of students at once usually involves switching between students’ online documents, which is less than ideal. I’ve written a GDocs Multipage web app, which embeds up to 8 live GDocs into one web page so that the teacher has an overview & therefore can monitor them more easily.

N.B. Test that this works without crashing your computer – Having a webinar app & several GDocs open in your web browser simultaneously uses a lot of memory & processing power! This probably won’t work for teachers on netbooks, Chromebooks, tablet PCs, & other low-power computers. Also, the bigger your screen, the easier it is to see what students are doing.

Conceptual overview

The following is an outline of a strategy to use a combination of 3 web-based tools during an online lesson:

  • A web conferencing service
  • Google Docs (GDocs)
  • My GDocs Multipage web app ‘hack’

In this scenario, we’ll assume that a teacher is teaching 8 students:

  • During the lesson, the teacher will give the 8 students tasks to complete in pairs, i.e. 4 groups of 2 students, in their respective GDocs.
  • While the students are collaborating on the tasks, the teacher will be able to see all 4 GDocs on one page, being edited in real-time.
  • The teacher will be able to drop in on any of the students’ GDocs, communicate with the students with the GDocs text chat, highlight text & other elements, write comments, & make edits & suggestions.
  • In the whole-class follow up, the teacher will have the option to show & discuss examples from the students’ GDocs with the screen sharing function in the webinar app, taking & uploading screen shots, &/or copying & pasting text.

The lesson

In this example, we’ll assume that students have already done learning activities in previous lessons &/or asynchronous, self-study activities, in which they’ll have watched & listened to the lecture, read the transcripts & checked their listening & reading comprehension, covering lexical & grammatical items as necessary.

Topic: Listening skills & the academic lecture genre

Objectives: Learning to follow & understand lectures more easily by:

  • Listening for ‘topic & comment’ utterances, i.e. introducing the beginning of a new topic or sub-topic
  • Listening for linking adjuncts & other expressions that introduce examples

Materials:

  • Video of an academic lecture
  • The lecture transcript

Tasks: Highlight & comment on the phrases or sentences (markers) that the lecturer uses to signal…

  1. …new parts of the lecture.
  2. …supporting examples.

Step by step instructions

We’ll assume that the students don’t have GDocs accounts. If all the students have accounts, then these steps should be modified to make the GDocs more private & secure.

N.B. Please consider the ethical implications of requiring students to have Google accounts & wherever possible, consider more open, privacy oriented, ethical, & GDPR compliant options such as Collabora: https://www.collaboraoffice.com/collabora-online/

  1. Create a template doc which contains all the instructions, steps, texts, etc. for the students to work on.
  2. GDocs template
  3. N.B. When you duplicate GDocs, it doesn’t preserve comments so it isn’t convenient to use these for task examples, instructions, etc..
  4. Duplicate the template, one for each pair of students, i.e. 8 students in 4 pairs = 4 documents, so that you have something like this:
  5. GDocs duplicates
  6. Set the document properties of each copy to: share with a link, anonymous users can edit.
  7. Copy the 4 shareable GDoc links to a document that you can use to copy & paste from during the webinar. You can also use this document to keep notes on how the students’ task performances go & reflect on them later, e.g.
    1. Doc1: https://docs.google.com/open?id=[document code]
    2. Doc2: https://docs.google.com/open?id=[document code]
    3. Doc3: https://docs.google.com/open?id=[document code]
    4. Doc4: https://docs.google.com/open?id=[document code]
  8. Shortly before the lesson, copy the 4 shareable links into this web form https://matbury.com/html/gdocs-multipage/index.php, which should look like this:
  9. GDocs Multipage
  10. N.B. If you close the GDocs multipage that you’ve set up, you’ll more than likely lose it & will have to re-enter the GDocs links to set the page up again, so keep those links handy just in case!
  11. You can adjust the width & height so that they fit more conveniently into your web browser window. You can also scale web pages (i.e. zoom in & out) with Ctrl+ mouse scroll wheel. I recommend experimenting to find which combination works best for you.
  12. At the appropriate stage of the webinar lesson, use the screen sharing function to show the document to the students & talk them through the tasks so that they have a clear understanding of what they have to do.
  13. In the document where you’ve copied the 4 GDocs links, write the names of the pairs of students next to each link, e.g.
    1. Joan & David: https://docs.google.com/open?id=[document code]
    2. Marta & Mireia: https://docs.google.com/open?id=[document code]
    3. Xavier & Carles: https://docs.google.com/open?id=[document code]
    4. Magali & Montse: https://docs.google.com/open?id=[document code]
  14. On the webinar platform, put the students into pairs & send them to breakout rooms so that each pair can maintain audio communication.
  15. Copy & paste the corresponding links to the students’ breakout rooms so that they can access their copies of the GDoc. This ensures that the students go to the copies of the document that you intend them to & so you know who is collaborating on which document.
  16. Go to the GDocs Multipage that you created just before the lesson & you will see the students arrive on the docs (their avatars show up on the top right of each GDoc window) & their cursors as they type, highlight, etc..

Additional suggestions

To hear a pair of students speaking & to speak with them, you can go into their corresponding breakout room in the webinar app.

It’s a good idea to have a reflective activity set up on the webinar main shared whiteboard, e.g. reflective questions, so that as students finish & return to the webinar, they have something to discuss & consolidate their learning while they are waiting for others to finish & come back.

I recommend practising this procedure until you are comfortable with it before trying it out in a live lesson, where there’ll be distractions, issues, etc., to deal with: Just like classroom teaching, digital teaching is a complex set of skills that take time & practice to coordinate & master.

Once you’ve got the hang of this, you can try it with different types of docs, e.g. spreadsheets & slide shows, & different types of tasks, e.g. collaborative story telling, peer-review, inductive focus on form, extensive think-pair-share tasks, & to introduce & start longer-term projects.

Conclusion

This article has outlined a practical suggestion for mitigating one of the limitations of webinar apps & thereby improving the quality online distance English language teaching. It facilitates an essential component of classroom teaching practice; monitoring students while they perform learning tasks so that in the moment, personalised guidance & feedback can be given. In this case, we can see how pedagogical principles, rather than EdTech novelties, have driven the modifications to the design of the teacher’s view of the learning activity & that having a clear & purposeful concept of what we want to achieve in teaching at a distance enables us to see & adapt to the shortfalls of available digital tools.

I hope you find it useful.

P.S. You can find the GPL3 (free and open source) licensed source code for the GDocs web app ‘hack’ on my GitHub.com account: https://github.com/matbury/gdocs-multipage

Categories
Assessment

Placement Tests: Communicative vs. Grammatical Competence

The blog post marks the end of a long hiatus in my blog writing. I’m starting back with a post about language assessment in general & placement tests in particular, & I’ll compare & contrast two English as a Foreign or Second Language (EFL/ESL) placement test assessment methods.

Context

I recently had the opportunity to compare & contrast two EFL/ESL assessment methods: semi-structured interviews & discreet-point multiple choice question (MCQ) tests. Both tests were administered to 475 international students who attended an EFL/ESL summer school in Canada. A sample size of n=475 gives a reasonably strong statistical power & this comparison took place in a typical summer school context.

From many years of experience of English language assessment in general & placement testing in particular, I believed that the semi-structured interview would be the more appropriate score to use to assign students to cohorts in similar levels of English language proficiency at the summer school. Semi-structured interviews place the main emphasis on communicative competencies, which are the ultimate goal of language learning. In other words, they’re a more direct form of assessment than typical structurally focused tests. Consequently, students were placed in their classes based entirely on their performance in the semi-structured interviews & only minor adjustments for incorrect student placement had to be made, mostly due to inter-rater reliability errors & that some students had under-performed due to jet-lag &/or test anxiety.

It is preferable to assess communicative competence at international summer schools because they bring together young students from around the world into mixed first language & nationality classes, making English the lingua franca that everyone has in common. In other words, English is the only language in which they can all understand each other & make themselves understood. Summer schools are an ideal opportunity to put into practice the English language that students have been studying year-round & further develop their communication skills. To accommodate this opportunity, English language summer schools often promote learning & teaching methods, such as communicative language teaching (CLT), task-based language learning (TBLL), content & language integrated learning (CLIL), & project-based learning (PBL), which place a strong emphasis on student-to-student cooperation & communicative competence.

Comparison of The Assessment Methods

Firstly, the semi-structured interviews & the discreet-point MCQ test assess different aspects of linguistic knowledge & ability (Table 1, below).

  • Semi-structured interviews assess students’ overall production of coordinated, integrated language skills & communicative competence, i.e. their ability to understand & make themselves understood in English & their ability to maintain a conversation through taking turns & giving appropriate responses.
  • Discreet-point MCQ tests primarily assess students’ ability to recognise correctly & incorrectly formed language structures, appropriate selection of vocabulary, & sometimes appropriate responses to given cues, prompts, &/or questions.

Table 1: A comparison of aspects of language assessed by discreet-point MCQs & semi-structured interviews

Discreet-point MCQs vs. Semi-structured Interviews
Test language recognition Test language production
Closed, tightly controlled Open-ended, spontaneous
Pre-defined Adaptive
Isolated knowledge & skills Coordinated, integrated knowledge & skills
Often focus on structural aspects (Usage) Focus on communicative competence (Use)

Details of the Two Tests

Discreet-Point MCQ test

The grammar test was a typical structural, norm-referenced, progressive discreet-point MCQ test, consisting of 50 items, of the type & quality frequently used in summer schools & language academies.

It’s worth noting that the design & development of effective MCQ language tests is challenging & requires a high degree of expertise. They require highly-skilled, careful, well-informed item writing, frequent statistical analyses on students’ responses to test items, e.g. facility index, discrimination index, & distractor efficiency. In this sense, effective MCQ tests are developed through an iterative process rather than designed.

In contrast, administering MCQ tests is relatively straightforward & easy to grade, e.g. untrained staff members can grade the tests with an answer key or even computer administered & graded tests can be implemented.

So MCQ tests require high expertise, time, & effort to develop but low expertise to administer & grade.

The Semi-Structured Interview

The speaking test was a semi-structured interview with communicative competencies & language functions as outlined in the Common European Framework of Reference for Languages (CEFR). The range of the speaking test was subdivided into 12 levels of language functions & communicative competencies, which is more fine-grained than typical placement tests, aligned with Trinity College London’s Graded Examinations in Spoken English (GESE), as indicated in Table 2 (below).

Table 2: Correspondence between CEFR & speaking test levels.

CEFR Level
A1.1 1
A1.2 2
A2.1 3
A2.2 4
B1.1 5
B1.2 6
B2.1 7
B2.2 8
B2.3 9
C1.1 10
C1.2 11
C2 12

In contrast to the discreet-point MCQ test, the semi-structured interviews required qualified, experienced teachers to administer each interview, for an average of 5 minutes per interview. Testers were provided with a criterion-referenced rubric & some additional visual materials to act as stimuli in order to elicit language, particularly from lower-proficiency students.

The exact duration of each interview mostly depends on the students’ language proficiency. The lower the proficiency of the student, the less time the interview would take & conversely, the higher the level, the longer the interview. For example, a cohort of 150 students would take 10 teachers about 60 minutes to interview (150 * 5 / 10 = 75) but if the levels are skewed towards the lower end, which is often the case on EFL/ESL summer schools, they would take substantially less.

Administering the speaking test is therefore more time-consuming & requires a larger number of more highly-qualified & experienced testers. However, the design is relatively quick as it relies to a high degree on the testers’ expertise in order to judge students’ language proficiency.

Comparison of the Test Results

As mentioned earlier, the semi-structured interviews were a valid & reliable predictor of students’ ability to participate in classes.

A frequency analysis (Figure 1, below) revealed that the distribution of grades was indeed skewed towards to lower end of the English proficiency scale, centred at around level 4 (CEFR A2.2).

Figure 1: Frequency distribution of the students’ scores (n=475) from the semi-structured interviews.

Frequency distribution of students' semi-structured interview scores

A scatter-plot of the test results for each student is a quick & easy to interpret way to compare each student’s score with their speaking test score. Each dot represents one or more students’ scores with their speaking test score on the x-axis (horizontal) & their grammar test score on the y-axis (vertical). If there were a strong correlation between the speaking & grammar tests, we would expect to see a relatively close grouping along a diagonal line from bottom left to top right (Figure 2, below).

Figure 2: Hypothetical representation of a strong correlation between the discreet-point MCQ test & the semi-structured interviews.

But what we see from the analysis of the actual test results a wide variation between the semi-structured interviews & the discreet-point MCQ test (Figure 3, below).

Figure 3: Representation of the actual correlation between the discreet-point MCQ test & the semi-structured interviews.

For example, the students who scored 48-50 in the discreet-point MCQ test, scored across a range of levels 4-11 (CEFR A2.2-C1.2) in the semi-structured interviews. This shows error margins of up to 9 levels when compared to the semi-structured interviews. Using the discreet-point MCQ test results to place students in classes would result in students with level 4 & 11 being placed in the same class. In other words, the MCQ test was a poor predictor of students’ communicative ability or their ability to participate in the appropriate level of class at the summer school.

This analysis of the test results clearly illustrates how pronounced the difference typically is between assessing grammatical competence & assessing communicative competence.

Where Does This Leave Discreet-Point MCQ Tests?

This is not entirely a critique of the discreet-point MCQ test format per se. These tests can be designed better & to assess a wider range of linguistic knowledge skills. The problem in this case, which is not atypical either, is that the items within the test focused almost entirely on structural aspects of language, i.e. grammatical competence, with too little attention given to how language should be used from a functional perspective, i.e. communicative competence.
As mentioned above, effective MCQ tests require highly-skilled & experienced item writers & are resource intensive & time-consuming to develop. Additionally, it’s even more challenging to develop MCQ items which assess many of the constituent knowledge & skills that make up communicative competencies. The question remains as to whether developing a discreet-point MCQ test that’s appropriate for EFL/ESL summer schools is a feasible goal. The first step is in understanding where the challenges lie & developing item-writing strategies the meet them.