A QUANTITATIVE STUDY ON AI-DRIVEN CONVERSATIONAL CHATBOTS FOR ENGLISH LANGUAGE LEARNING AMONG PAKISTANI UNDERGRADUATE ESL LEARNERS

Abstract

Artificial intelligence (AI) has transformed technology-enhanced language learning by enabling personalized instruction, immediate feedback, and authentic language interaction through conversational chatbots. Despite growing global interest, empirical evidence on the effectiveness of AI-powered conversational chatbots in English as a Second Language (ESL) learning remains limited in Pakistani higher education. This study investigated the effectiveness of AI-powered conversational chatbots in improving the vocabulary knowledge and grammatical competence of Pakistani undergraduate ESL learners.A quantitative pre-test–post-test research design, adapted from Polakova and Klimova (2024), was employed. A purposive sample of 100 undergraduate ESL students with upper-intermediate and advanced English proficiency (CEFR B2–C1) participated in a four-week chatbot-assisted learning intervention at a public-sector university in Pakistan. Data were collected using a standardized English placement test, vocabulary and grammar pre-test, post-test, and chatbot interaction logs. Descriptive statistics summarized learners' performance and engagement, while paired-samples t-tests and Wilcoxon signed-rank tests were used to examine differences between pre-test and post-test scores. The findings revealed significant improvements in learners' vocabulary and grammatical competence following the intervention. Participants also demonstrated high engagement with chatbot-assisted learning, indicating that conversational AI can effectively supplement conventional English language instruction. The study contributes empirical evidence to the growing literature on AI-assisted language learning within the Pakistani ESL context and highlights the potential of AI-powered conversational chatbots to enhance English language teaching. The findings provide practical implications for educators, curriculum developers, and policymakers seeking to integrate AI technologies into higher education language curricula.