AI Supported IELTS Preparation With Human Adherence Support Under Equal Planned Study Time

Full abstract

Abstract

Background: AI-supported language learning is increasingly offered alongside established forms of IELTS preparation, but randomized comparisons of complete preparation services remain limited.

Objective: To compare AI-supported digital preparation with non-instructional human adherence support (AI plus support) against self-study, large-group classes, and one-to-one tutoring under equal planned study time.

Methods: In this four-arm, individually randomized, stratified trial, 960 adults with estimated entry IELTS Academic bands of 4.5–6.0 were allocated equally to four preparation formats for 40 planned hours over 8 weeks. In the AI arm, instruction, practice, feedback, and question answering were delivered through the Yingwu Zhixue (Parrot Learning; 鹦鹉智学) platform, comprising a mobile app and a companion web interface; human staff provided study supervision only. The primary outcome was a four-skill mock-test composite averaged across two post-test forms and adjusted for the average of two pre-test forms. Writing and speaking were independently scored by masked raters. The intention-to-treat analysis used multiple imputation and accounted for clustering within classes, tutors, and adherence-support staff caseloads. Three prespecified comparisons used Bonferroni-adjusted 98.33% confidence intervals, providing at least 95% simultaneous coverage. The primary criterion required all three lower bounds to exceed zero.

Results: Post-test completion was 82.4% (791/960). Adjusted differences favored AI plus support over self-study (0.54 composite-score units, 98.33% CI 0.33–0.76), large-group classes (0.45, 0.20–0.71), and one-to-one tutoring (0.45, 0.21–0.69); all Bonferroni-adjusted p values were < .001. Complete-case estimates were similar. Satisfaction estimates favored AI plus support over self-study and large-group classes; the comparison with one-to-one tutoring did not establish superiority or equivalence.

Conclusions: Under equal planned study time, assignment to AI plus support improved mock-test performance relative to the three comparison services as implemented in this trial. The findings concern the combined service and do not establish AI-specific effects, per-hour efficiency, or gains in official IELTS scores.

Study design

What was compared

The trial randomized 960 adults in a 1:1:1:1 ratio, with 240 participants per arm. Eligibility was based on study-estimated IELTS Academic entry bands of 4.5–6.0. Every arm received the same planned study budget: 40 hours across 8 weeks. Planned time was not the same as actual time studied.

Primary outcome

Main adjusted results

The primary outcome was a study-defined four-skill mock-test composite, averaged across two post-test forms and adjusted for the average of two pre-test forms. It is not an official IELTS band score and is not a standardized effect size.

• 960randomized participants, 240 per study arm
• 791completed post-testing: 82.4% of randomized participants
• 40 hplanned study time per arm across 8 weeks

AI plus support compared with each service

The three confidence intervals were Bonferroni-adjusted and provide at least 95% simultaneous coverage. All three lower bounds were above zero, meeting the prespecified primary criterion.

Interpretation

What the results do and do not show

Within this trial, assignment to the complete AI-plus-support service produced higher adjusted mock-test composite performance than assignment to each comparison service under the same planned time budget. The estimate applies to the implemented package, population, assessments, and eight-week period.

• The design does not isolate the effect of AI from non-instructional human adherence support or from other differences between the implemented services.
• The outcome is a study-defined mock-test composite. It cannot be translated directly into official IELTS band gains.
• Equal planned time does not establish greater efficiency per hour because actual engagement differed across arms.
• The results do not establish superiority over every self-study resource, class, tutor, or future implementation.
• Post-test outcomes were missing for 169 participants. The primary analysis used multiple imputation, but other missingness mechanisms could change estimates.
• Satisfaction was measured among participants who knew their allocation. The comparison with one-to-one tutoring established neither superiority nor equivalence.

Transparency

Funding, product role, and public materials

Zhiying Technology (Hangzhou) Co., Ltd. funded the study and provided the commercial product evaluated in the AI arm. The manuscript states that an external research team controlled the protocol, recruitment integrity, randomization, assessment, analysis, and publication. The sponsor could review factual accuracy and confidential information but could not suppress findings or decide whether results were published.

Version 1.0 was released on Zenodo on 17 October 2026 under CC BY 4.0. The complete methods, results, and discussion are available through the DOI and full PDF.

Reuse and citation

Recommended citation

Chen, L., Okafor, A., & Weber, J. (2026). AI Supported IELTS Preparation With Human Adherence Support Under Equal Planned Study Time: A Four Arm Stratified Randomized Trial (Version 1.0) [Preprint]. Zenodo. https://doi.org/10.5281/zenodo.23434361