Task: Academic reading 📚

Start by working through the following scientific text as you normally would. As you read, the two different people in the chat at the bottom right are available to answer any questions you may have about the text or to help you read scientific texts. You can chat or talk to them.

The aim is for you to be able to answer basic questions about the article.

Designing Pedagogical Conversational Agents in Virtual Worlds

Authors: Bijan Khosrawi-Rad, Heidi Rinn, Dominik Augenstein, Daniel Markgraf, Susanne Robra-Bissantz

Abstract

Pedagogical conversational agents (PCAs) are intelligent dialog systems that can support students as chatbots or voice assistants. However, many users find interactions with PCAs less engaging. One solution to increase learners’ engagement is to embed the PCA in a virtual world, e.g., as a humanoid avatar that facilitates collaborative learning. Such a learning setting could be beneficial because virtual worlds positively affect fun and immersion. In this paper, we derive prescriptive design knowledge for PCAs in virtual worlds based on the results of nine expert interviews synthesized with findings from the literature. This design knowledge aims to enable the meaningful design of PCAs in virtual worlds. We contribute to research and practice by demonstrating how PCAs in virtual worlds can be designed to increase students' motivation to learn.

Introduction and Motivation

Pedagogical conversational agents (PCAs) are intelligent dialog systems that support learners by processing natural language [HM19]. They can teach content, motivate learners, or moderate collaborative learning [Kh22a]. With AI advances, PCAs are getting better at helping learners with individual concerns, as the example of ChatGPT shows. However, users often perceive interactions with PCAs as not motivating, leading to the rejection of PCAs [Be22]. Combining PCAs with game approaches is one way to counteract this issue. This can be done by integrating PCAs as human-like avatars into virtual worlds (VWs) [GG16]. They can, for instance, guide learners through the VW while presenting them with lively challenges [KGR23]. Such VWs have been showing positive effects for years, such as immersion and collaborative learning [DMJ12]. Recent examples such as Roblox or Minecraft show that VWs can be fun for users. They can also be used for serious purposes such as education [HC22]. Hence, by combining both trends, PCAs and VWs, educators could benefit from the positive effects of both [KGR23]. Current literature reviews show that embedding PCAs in VWs is novel and that there does not yet exist prescriptive design knowledge on PCAs in VWs [KGR23, Kh22a]. However, this design knowledge helps developers and educators implement PCAs in VWs. We address this research gap using the design science research methodology to derive design knowledge for PCAs. Hence, we address the following research question (RQ): How to design PCAs in VWs to foster students' motivation to learn? We follow the procedure of [He07] to answer our RQ. We derive design knowledge based on nine expert interviews. We aim to ensure scientific rigor and practical relevance by incorporating this body of expertise. We formulate the design knowledge as design principles (DPs), i.e., abstract requirements for PCAs in VWs [GCS20]. These are complemented by meta-requirements (MRs), design features (DFs), and overarching design guidelines (DGs). In this short paper, we report the results of our first design cycle as a tentative conceptual design.

Research Background

PCAs interact with their users either via text (as chatbots) or speech (as virtual assistants) [HM19]. They can take on different roles, which lead to different functionalities [Kh22a]: organizer, tutor, mentor, motivator, and moderator. PCAs, as organizers, support learners in navigating the learning environment. The tutor role focuses on imparting learning content. The mentor role goes beyond merely imparting learning content, so learners are accompanied in the long-term, e.g., through feedback and study tips. The PCA as a motivator serves to promote learners’ engagement, e.g., through gamification. The moderator mediates collaborative learning or brainstorming. PCAs can be incorporated into VWs to guide learners in these human-like roles [GG16]. VWs are immersive 3D environments where avatars communicate with each other [DMJ12]. VWs promote immersion and contribute to a social presence experience, and learners thus enjoy learning [ibid.]. In classic game environments, virtual agents were usually implemented by non player characters (NPCs). While these were not AI-based, PCAs, as an extension of NPCs, enable individualized communication [KGR23]. Moreover, AI allows learners to receive social and empathic support if the PCA acts as a friendly virtual companion [St22]. Some authors already propose design recommendations for PCAs, e.g., for argumentative writing support [WSL20]. However, there is yet no design knowledge for PCAs in VWs.

Study Design and Results

We conducted semi-structured interviews with nine experts from different research streams (psychology, computer science, information systems) and industry practitioners. All experts had prior experience with designing PCAs or VWs. The interview guide consisted of questions about design recommendations and desired features for PCAs in VWs. The interviews lasted between 42 to 64 minutes. We transcribed and analyzed all interviews based on a pre-defined coding guide in two coding cycles. The coding guide followed our interview guide and included categories on the benefits of PCAs in VWs and learners’ challenges to be solved, the design of PCAs and VWs, the design of the PCA roles in the VW, and mentioned kernel theories. Following [Mö20], we synthesized the results with supporting literature and formulated MRs for PCAs in VWs. We then combined MRs that relate to each other into DPs and illustrated these DPs with example DFs [Mö20]. In formulating the DPs, we followed a unified scheme according to [GCS20]. We structured the DPs according to the PCA roles (see Chapter 2) so that each DP corresponds to a particular PCA role. We choose this approach because, according to social agency theory, different roles of human-like agents lead to varying expectations of users and functions to realize [Be22, MSM03]. Since the experts also mentioned aspects that apply to all roles, we furthermore formulated role-independent design guidelines (DGs). Figure 1 outlines the DGs and DPs.

Introduction Image

Figure 1: An example illustrating the importance of PCAs in virtual worlds.

Conclusion and Outlook

PCAs can be integrated into VWs to make learning immersive and motivating. We have derived design knowledge to assist researchers and practitioners in creating such learning scenarios. We are implementing these abstract conceptual considerations into a virtual design thinking training in the VW Unity that includes two PCAs to support learners, one using the chatbot service Rasa and one interfacing with ChatGPT. The Rasa-based PCA is to take over simple tasks like welcoming the learners to the VW and introducing them to the design thinking method. The ChatGPT-based PCA will take on more complex tasks, such as moderating the training, and generating creative ideas. We plan to evaluate our approach’s effectiveness using focus group interviews with our target group.