Journal of New Approaches in Educational Research

Articulo main

José-María Romero-Rodríguez María-Soledad Ramírez-Montoya Mariana Buenestado-Fernández Fernando Lara-Lara

Abstract

Artificial intelligence (AI) and AI-based chatbots, such as ChatGPT, are transforming the approach to education. In particular, ChatGPT's potential to process large amounts of data and learn from user interactions makes it a beneficial resource for students, albeit with some reluctance from some teachers. This study aimed to explore the acceptance of ChatGPT by university students. The researchers administered an online survey to 400 Spanish university students aged 18-64 (M = 21.80; SD = 6.40). The results of the methodological approach based on the UTAUT2 model for technology adoption showed that: 1) gender was not a determining variable in any construct while the experience of use was a factor conditioning a higher score on all constructs; 2) experience, performance expectancy, hedonic motivation, price value, and habit were influential in behavioral intention to use ChatGPT; 3) facilitating conditions, habit, and behavioral intention were conditioning factors in user behavior. Finally, this report discusses the findings and practical implications of the work and recommends some good uses for ChatGPT.


La inteligencia artificial (IA) y los chatbots basados en IA, como ChatGPT, están transformando el enfoque de la educación. En particular, el potencial de ChatGPT para procesar grandes cantidades de datos y aprender de las interacciones de los usuarios lo convierte en un recurso beneficioso para los estudiantes, aunque con ciertas reticencias por parte de algunos profesores. El objetivo de este estudio era explorar la aceptación de ChatGPT por parte de los estudiantes universitarios. Los investigadores administraron una encuesta en línea a 400 estudiantes universitarios españoles de entre 18 y 64 años (M = 21,80; DE = 6,40). Los resultados del enfoque metodológico basado en el modelo UTAUT2 para la adopción de tecnología mostraron que: 1) el género no fue una variable determinante en ningún constructo mientras que la experiencia de uso fue un factor condicionante de una mayor puntuación en todos los constructos; 2) la experiencia, la expectativa de rendimiento, la motivación hedónica, el valor del precio y el hábito fueron influyentes en la intención conductual de uso de ChatGPT; 3) las condiciones facilitadoras, el hábito y la intención conductual fueron factores condicionantes en el comportamiento del usuario. Por último, se discuten los resultados y las implicaciones prácticas del trabajo y se recomiendan algunos buenos usos de ChatGPT.

Keywords
ARTIFICIAL INTELLIGENCE; CHATBOT; COMPLEX THINKING; HIGHER EDUCATION; UTAUT2; EDUCATIONAL INNOVATION
INTELIGENCIA ARTIFICIAL; CHATBOT; PENSAMIENTO COMPLEJO; EDUCACIÓN SUPERIOR; UTAUT2; INNOVACIÓN EDUCATIVA
Section
Articles

INTRODUCTION

The reality of education today includes addressing the digital transformation affecting universities and other educational institutions (Anderson et al., 2023). The development of Artificial Intelligence (AI) has led to digital disruption in the education system because of the rapid advances it necessitates for education (García-Peñalvo, 2023). Along these lines,UNESCO (2019) differentiates three dimensions of linking AI and education: (i) learning to use AI tools in the classroom; (ii) learning to know AI and its technical possibilities; (iii) raising public awareness of the impact of AI on people's lives.

Among the tools offered by AI, ChatGPT has become very popular by the first quarter of 2023. This AI launched for free to the public on 22 November 2022. In the first five days, it had one million users; in the following two months, it attained 100 million (Tong & Zhang, 2023). ChatGPT allows the creation of texts, codes, stories, poems, etc., of considerable quality (Rozencwajg & Kantor, 2023). It is a highly complex language model that brings together more than 175 billion parameters to generate coherent responses in the context of the conversation with this AI (Kung et al., 2023). It also facilitates dialogue and interaction between the user and the AI. The response is generated quickly, structured, well-written, and based on various sources of information (Carrasco et al., 2023).

Since the publication of this tool, its impact in the university context has been almost immediate. While there is little empirical peer-reviewed research on its impact on universities (Crawford, Cowling, & Allen, 2023), there are known implications for teaching, learning, academic research, epistemology, the digital transformation of educational institutions, and even ethics (García-Martínez, Fernández-Batanero, Fernández-Cerero, & León, 2023; García-Peñalvo, 2023; Stokel-Walker, 2022; Stokel-Walker & Van Noorden, 2023; Tamboleo-García, 2023).

In particular, the University of Tasmania has published a statement on the uses of AI for teachers and students, highlighting its potential for learning and the need for care in its use to respect academic integrity. The Royal Melbourne Institute of Technology University has organized teaching courses for ChatGPT. The University of Granada is leading training courses for its use in the university (Torres-Salinas & Arroyo-Machado, 2023). However, some universities, such as those in Hong Kong and some in France, have banned its use or have established severe sanctions in some instances. Other universities are updating their academic integrity policies and adapting exams to prevent misuse of ChatGPT by students (Tlili et al., 2023).

In the field of research, some published studies have recognized the authorship of ChatGPT (Stokel-Walker & Van Noorden, 2023). This tool saves time by making certain processes more flexible and focusing more deeply on the experimental design of the research. However, concerning the authorship of scientific articles, it would not currently qualify unless the ICMJE/COPE guidelines are modified (Sallam, 2023). This technology cannot yet create new ideas, show spontaneity or manifest critical thinking (García-Martínez et al., 2023).

The use of ChatGPT by students and teachers does not seem widespread currently. However, there is a range of questions regarding its use. It offers various advantages for training, research, and management. Concerning the main issues, the following stand out:

  • The answers it provides contain errors. The probability of a more or less correct answer may vary depending on the context of the question, and the responses must be checked for accuracy before being considered valid. Interestingly, the study by Carrasco et al. (2023) shows that the ChatGPT could answer 51.4% of the questions in the Medical Intern Resident (MIR) exam correctly.

  • Replacing human writing with artificial writing would affect the acquisition of key competencies (O'Connor, 2022). Educational tasks should be redesigned to favor more complex university assessments (Salvagno, Taccone, & Gerli, 2023). Teachers may be overburdened and stressed with new control functions over the results provided by this tool (Lim, Gunasekara, Pallant, Pallant, & Pechenkina, 2023).

  • The reply does not include a reference for its information and lacks acknowledgment of intellectual authorship. It implies the need to reinforce training in academic ethics, good practice, ethics, self-awareness, and the definition of plagiarism in the use of ChatGPT (Graf & Bernardi, 2023).

  • Responses may include existing socio-cultural biases (Curtis, 2023). The information in their responses comes from big data provided primarily by Western cultures. The inclusion of these tools in education policy justifies the need for a process of technological domestication (Engen, 2019).

  • Access to and registration in this tool is currently free of charge. Equity could be affected if the free use of the tool is removed (Kasneci et al., 2023). Thus, universities may need to integrate new technologies to check the misuse of this AI. These technologies would mean increased expenditures to safeguard the quality of teaching. University malpractice can lead to reputational harm and, consequently, to the devaluation of the university degree. Such situations would mainly affect university institutions with fewer resources.

However, most studies so far argue in favor of AI's responsible use (Anderson et al., 2023; Carrasco et al., 2023; Crawford et al., 2023; Dwivedi et al., 2023; Kung et al., 2023; O'Connor, 2022; Sallam, 2023). García-Peñalvo (2023) defends the importance of ChatGPT for improving teaching and learning processes, developing critical thinking in the classroom, training in searching, comparing sources, and preparing targeted questions for this AI. Crawford et al. (2023) warn of the stressful and anxious situations that students suffer to meet the goals established by the universities and reflect on the possibilities for educational innovation that this tool can offer. The latter includes allowing the student to obtain immediate answers to problems, concepts, theories, treatments, and diagnoses (Carrasco et al., 2023). Finally, Anderson et al. (2023) argue for the possibilities that ChatGPT offers to learn a new language, experience personalized learning, correct texts, and be more effective in managing their time.

Based on these considerations, this study aimed to explore the acceptance of ChatGPT by university students.

Theoretical Background and Hypotheses Development

The UTAUT2 model (Unified Theory of Acceptance and Use of Technology 2) is a theoretical framework to understand and explain technology acceptance and use (Venkatesh, Thong, & Xu, 2012). This model is based on its predecessor, the original UTAUT model (Venkatesh et al., 2012), and has been widely used to predict and explain user behavior with technology (Arista & Abbas, 2022; Hassan, Islam, Yusof, Nasir, & Huda, 2023; Uncovska, Freitag, Meister, & Fehring, 2023).

In particular, it considers seven main factors that influence the adoption and use of technology: (i) performance expectancy; (ii) effort expectancy; (iii) social influence; (iv) facilitating conditions; (v) hedonic motivation; (vi) price value; (vii) habit. Furthermore, age, gender, and previous experience can influence the correlations between these factors and technology acceptance and use.

The UTAUT model has been reported in previous research on user acceptance and use of artificial intelligence (Gansser & Reich, 2021; García, Sarmiento, & Antonovica, 2022; Shinners, Aggar, Grace, & Smith, 2019; Sohn & Kwon, 2020). After consideration, we applied this model as the theoretical basis for the present study on using and adopting the ChatGPT artificial intelligence tool.

The proposed research model included the UTAUT2 constructs of performance expectancy (PE), effort expectancy (EE), social influence (SI), facilitating conditions (FC), hedonic motivation (HM), price value (PV), habit (H), behavioral intention (BI), and user behavior (UB). In the hypothetical model, BI is affected by PE, EE, SI, FC, HM, PV, and H. In turn, UB is influenced by BI, FC, and H. The socio-demographic variables of gender and age affect FC, HM, PV, and H. Finally, experience influences FC, HM, H, and BI (Figure 1).

Figure 1. Research model

Performance Expectancy (PE)

Performance expectancy is "the degree to which an individual believes that using the system will help him or her to attain gains in job performance" (Venkatesh et al., 2003, p. 447). This construct is considered one of the strongest predictors of behavioral intention (Venkatesh et al., 2012). In the academic context, performance expectancy refers to academic performance (Khechine, Raymond, & Augier, 2020). Therefore, we proposed the following hypothesis: H1. Performance expectancy has a significant effect on the behavioral intention to use ChatGPT.

Effort Expectancy (EE)

Effort expectancy is "the degree of ease associated with using the system" (Venkatesh et al., 2003, p. 450). Similarly, this construct is considered one of the strongest predictors of behavioral intention (Venkatesh et al., 2012). In the academic context, effort expectancy refers to students' ease of use (Khechine et al., 2020). Considering this construct, we put forth the following hypothesis: H2. Effort expectancy has a significant effect on the behavioral intention to use ChatGPT.

Social Influence (SI)

Social influence is the "degree to which an individual perceives that important others believe he or she should use the new system" (Venkatesh et al., 2003, p. 451). This construct is also considered among the strongest predictors of behavioral intention (Venkatesh et al., 2012). In the academic context, social influence refers to the opinion of other students, teachers, friends, and family about using a given technology (Khechine et al., 2020). The hypothesis linked to this construct was H3. Social influence has a significant effect on the behavioral intention to use ChatGPT.

Facilitating Conditions (FC)

Facilitating conditions are defined as the "degree to which an individual believes that an organizational and technical infrastructure exists to support the use of the system" (Venkatesh et al., 2003, p. 450). Like the previous ones, this construct is considered one of the most influential predictors of behavioral intention (Venkatesh et al., 2012). In the academic context, facilitating conditions refer to human, organizational, and technical support for using technology (Khechine et al., 2020). The hypotheses linked to this construct were: H4. Facilitating conditions significantly affect behavioral intention to use ChatGPT; H5. Facilitating conditions have a significant effect on user behavior in ChatGPT; H6. Age is a factor that significantly affects facilitating conditions of ChatGPT; H7. Gender is a factor that significantly affects facilitating conditions of ChatGPT; H8. Experience is a factor that significantly affects the facilitating conditions of ChatGPT.

Hedonic Motivation (HM)

Hedonic motivation is "the fun or pleasure derived from using a technology" (Venkatesh et al., 2012, p. 161). The hypotheses linked to this construct were: H9. Hedonic motivation has a significant effect on the behavioral intention to use ChatGPT; H10. Age is a factor that significantly affects the hedonic motivation of ChatGPT; H11. Gender is a factor that significantly affects the hedonic motivation of ChatGPT; H12. Experience is a factor that has a significant effect on the hedonic motivation to use ChatGPT.

Price Value (PV)

Price value is "the cost and price structure that can have a significant impact on consumers' use of technology" (Venkatesh et al., 2012, p. 161). The hypotheses linked to this construct were: H13. Price value has a significant effect on the behavioral intention to use ChatGPT; H14. Age is a factor that significantly affects the price value of ChatGPT; H15. Gender is a factor that has a significant effect on the price value of ChatGPT.

Habit (H)

Habit is "the extent to which people tend to perform behaviors automatically because of learning" (Venkatesh et al.,2012, p. 161). The hypotheses linked to this construct were: H16. Habit has a significant effect on the behavioral intention to use ChatGPT; H17. Habit has a significant effect on user behavior in ChatGPT; H18. Age is a factor that significantly affects the habit of ChatGPT; H19. Gender is a factor that significantly affects the habit of ChatGPT; H20. Experience is a factor that has a significant effect on the habit of ChatGPT.

Finally, the last two hypotheses considered the behavioral intention construct: H21. Experience is a factor that has a significant effect on the behavioral intention of ChatGPT; H22. Behavioral intention has a significant effect on ChatGPT user behavior.

METHOD

Participants and Procedure

The researchers adopted a cross-sectional design, applying a self-administered survey at a single point in time to undergraduate students at the University of Granada (n = 400) enrolled during the 2022/2023 academic year. Participant data were collected using Google Forms; the participants received the survey via the University of Granada student mailing list. Thus, the sampling was by convenience, as all students enrolled at the University were invited to participate, and the final sample comprised those students who freely chose to participate.

Participants answered questions about their socio-demographic data (age, gender, and experience) and the UTAUT2 model scale adapted to the ChatGPT tool. Before answering, all respondents received information about the study's purpose and the data's anonymous treatment. All participants had to give their informed consent to participate. The data collected were processed following the legislation in Spain (Organic Law 3/2018 of 5 December on Personal Data Protection and Digital Rights Guarantees). The data collection period took place during the first week of March 2023.

Finally, the sample of university students included 110 males and 290 females, ages 18 to 64 (M = 21.80; SD = 6.40), with ChatGPT usage experience ranging from less than 1 month to more than 1 month (Table 1). The age ranges conformed to the groupings established by the World Health Organization (WHO, 2017): ≤20 = teenagers, 21-35 = young adults, and ≥36 = older adults.

Table 1. Socio-demographic data

n

%

Age

≤20

249

62.3

21-35

135

33.8

≥36

16

4

Gender

Male

110

27.5

Female

290

72.5

Experience

<1

333

83.3

>1

67

16.7

Measures

The UTAUT2 model was applied through a self-reported questionnaire. The constructs of performance expectancy (PE), effort expectancy (EE), social influence (SI), facilitating conditions (FC), hedonic motivation (HM), price value (PV), habit (H), behavioral intention (BI), and user behavior (UB), were adapted from Venkatesh et al. (2012) for the ChatGPT tool. The mode of responses to the questionnaire items was a seven-level Likert scale (1 = strongly disagree to 7 = strongly agree).

Previous studies on UTAUT2 showed adequate internal consistency and good psychometric properties in the different constructs (Arista & Abbas, 2022; Gansser & Reich, 2021; García et al., 2022; Hassan et al., 2023; Shinners et al., 2019; Sohn & Kwon, 2020; Uncovska et al., 2023). For this study, adequate reliability values, calculated through Cronbach's alpha coefficient, were obtained for each construct: PE = .929, EE = .958, SI = .945, FC = .910, HM = .959, PV = .967, H = .886, BI = .907, and UB = .924. Overall reliability was also adequate (Cronbach's α = .977).

Data Analysis

Researchers used IBM SPSS and IBM SPSS Amos, version 25 statistical packages. The statistical tests were: T-test to test for significant differences between two populations (gender and experience variables), ANOVA to test for significant differences between more than two populations (age variable), convergent and discriminant validity (using the measurement model, Hair et al., 2006, 2017) and structural equation modeling (from path analysis, Stage et al., 2010).

Before the path analysis, we confirmed the hypotheses of univariate and multivariate normality of the data employing the Kolmogorov-Smirnov (K-S) test with Lilliefors correction, taking as a reference that the values of skewness were less than three and kurtosis less than 10 as criteria of data adequacy (Kline, 2005), and with Mardia's coefficient (Mardia, 1970). Finally, the goodness-of-fit indices of the model reflected its adequacy.

RESULTS

Descriptive statistics showed differences in the means of each construct according to gender (Table 2). Men obtained higher scores in all constructs of the UTAUT2 model. Thus, statistically significant differences existed for: EE (p = .009); FC (p = .001); HM (p = .017); PV (p = .016).

Table 2. Descriptive statistics and significance based on gender

Construct

Men

Women

p

M

SD

M

SD

PE

15.94

7.541

14.70

7.115

.127

EE

17.95

7.658

15.67

7.759

.009

SI

9.83

5.122

9.22

5.208

.297

FC

18.88

6.795

16.04

7.597

.001

HM

13.47

5.734

11.92

5.773

.017

PV

12.64

6.329

11

5.911

.016

H

10.11

5.749

10.08

6.156

.969

BI

9,78

5.187

8.89

5.137

.121

UB

19.02

9.646

16.87

10.190

.057

Descriptive statistics based on age also showed differences in the means (Table 3). The highest means in the UTAUT2 model were attained by the population over 21 years. Significant differences existed in the constructs of EE (p = .000), FC (p = .000), HM (p = .021), PV (p = .021) and UB (p = .033).

Table 3. Descriptive statistics and significance based on age

Construct

≤20

21-35

≥36

p

M

SD

M

SD

M

SD

PE

14.43

7.315

15.99

7.050

16.56

7.164

.091

EE

15.03

7.759

18.49

7.484

17.56

6.811

.000

SI

9.30

5.082

9.61

5.446

8.81

4.736

.770

FC

15.51

7.562

18.96

6.965

19.38

5.909

.000

HM

11.78

5.895

13.10

5.514

14.88

5.524

.021

PV

10.84

6.076

12.63

6.028

11

4.953

.021

H

10.05

6.081

10.28

6.050

9.12

5.536

.758

BI

8.66

5.128

9.93

5.265

9.69

4.012

.064

UB

16.49

10.099

18.84

9.926

21

9.423

.033

Regarding descriptive statistics based on experience, the highest means in the UTAUT2 model were obtained by the population with the most user experience, with significant differences in all constructs (Table 4).

Table 4. Descriptive statistics and significance based on experience

Construct

<1

>1

p

M

SD

M

SD

PE

14.12

7.130

19.61

6.018

.000

EE

14.95

7.604

23

4.596

.000

SI

9.01

5.161

11.28

4.920

.001

FC

15.57

7.408

23.04

3.929

.000

HM

11.49

5.720

16.63

4.030

.000

PV

10.61

5.834

15.63

5.466

.000

H

9.75

6.067

11.78

5.648

.012

BI

8.42

5.058

12.66

4.140

.000

UB

16.55

10.319

21.99

7.306

.000

Regarding the instrument, the convergent validity had adequate values (Table 5). Thus, the composite reliability (CR) values of the constructs were above .8, and the average variance extracted (AVE) values were above .5 (Hair et al., 2017).

Table 5. Convergent validity measures

Construct

Item

Factor Loading

CR

AVE

PE

PE1

.856

.930

.769

PE2

.884

PE3

.913

PE4

.854

EE

EE1

.921

.958

.852

EE2

.889

EE3

.944

EE4

.938

SI

SI1

.923

.945

.851

SI2

.933

SI3

.913

FC

FC1

.893

.910

.718

FC2

.872

FC3

.871

FC4

.747

HM

HM1

.935

.958

.886

HM2

.959

HM3

.930

PV

PV1

.929

.967

.907

PV2

.973

PV3

.955

H

H1

.872

.891

.674

H2

.781

H3

.710

H4

.907

BI

BI1

.879

.918

.790

BI2

.835

BI3

.949

UB

UB1

.827

.924

.670

UB2

.883

UB3

.826

UB4

.726

UB5

.799

UB6

.845

For the discriminant validity analysis, we used the square root of AVE to correlate the latent constructs (Table 6). Thus, each factor represented a different dimension, and the psychometric characteristics of the instrument were acceptable (Ratchford, 1987).

Table 6. Discriminant validity measures

PE

EE

SI

FC

HM

PV

H

BI

UB

PE

.877

EE

.808

.923

SI

.767

.620

.923

FC

.758

.925

.617

.847

HM

.796

.824

.627

.838

.941

PV

.664

.735

.572

.768

.743

.952

H

.625

.504

.697

.454

.528

.531

.821

BI

.759

.658

.713

.635

.704

.664

.899

.888

UB

.681

.583

.644

.592

.652

.579

.734

.817

.819

Note: Diagonals represent the average variance extracted, while the other matrix entries represent the squared correlations

Before the path analysis, the normality values showed a skewness value of -.237, an asymmetrically negative curve, and a kurtosis value of -.868, translating into a platykurtic distribution. However, the skewness and kurtosis values were within appropriate values (< 3 and < 10). In contrast, the K-S test with Lilliefors significance correction established that the data did not follow a normal distribution (K-S = .079; df = 399; p = < .000).

Although the univariate normality hypothesis was not met, multivariate normality was confirmed (Mardia = 42.77), being a value lower than p*(p + 2), where p was the number of observed variables (in total 34 scale items) (Bollen, 1989) . In turn, the goodness-of-fit indices of the PA model were adequate (Byrne, 2013) (Table 7).

Table 7. Goodness of fit measure

Index

Values obtained

Criteria

χ2

43.71

gl

16

χ2/gl

2.73

≤3

GFI

.983

≥.90

RMSEA

.046

<.05

NFI

.988

≥.90

CFI

.992

≥.90

AGFI

.915

≥.90

SRMR

.406

<.08

Note. df = degrees of freedom; GFI = goodness-of-fit index; RMSEA = root mean squared error of approximation; NFI = normalised fit index; CFI = comparative fit index; AGFI = adjusted goodness-of-fit index; SRMR = standarized root mean-square

The results of the hypothesis testing showed that only 11 of the 22 hypothesized relationships were supported (Table 8). Moreover, these relationships were positive between the different variables, so the higher the score, the higher the two variables increased. The unsupported hypotheses were rejected.

Table 8. Hypothesis testing results

Hypothesis

Relationship

Path coefficient

CR

Results

H1

PE → BI

.102

3.40

***

Supported

H2

EE → BI

-.001

-.025

.980

Not supported

H3

SI → BI

.005

.128

.898

Not supported

H4

FC → BI

.019

.571

.568

Not supported

H5

FC → UB

.218

4.051

***

Supported

H6

FC ← Age

1.205

2.579

.010

Supported

H7

FC ← Gender

-.071

-.152

.879

Not supported

H8

FC ← Exp

4.486

6.230

***

Supported

H9

HM → BI

.143

3.857

***

Supported

H10

HM ← Age

.134

.382

.702

Not supported

H11

HM ← Gender

.134

.333

.739

Not supported

H12

HM ← Exp

3.416

5.507

***

Supported

H13

PV → BI

.069

2.302

.021

Supported

H14

PV ← Age

.381

.908

.364

Not supported

H15

PV ← Gender

-.356

-.740

.459

Not supported

H16

H → BI

.509

19.167

***

Supported

H17

H → UB

.355

3.814

***

Supported

H18

H ← Age

-.321

-.791

.429

Not supported

H19

H ← Gender

.495

.975

.330

Not supported

H20

H ← Exp

.410

.656

.512

Not supported

H21

BI ← Exp

1.416

4.320

***

Supported

H22

BI → UB

.968

7.739

***

Supported

Note: Exp = experience; CR = critical radio; ***Significant at p < 0.001

Finally, in the path analysis, the relationships between the constructs were collected, and only the values of supported hypotheses were established (Figure 2). Based on this, the path coefficients supported 11 hypotheses (H1, H5, H6, H8, H9, H12, H13, H16, H17, H21, H22). These data showed that effort expectancy, social influence, and facilitating conditions did not influence the behavioral intention to use ChatGPT. The coefficients of determination (R2) of the model for each endogenous variable were for PE (R2 = .000); EE (R2 = .000); SI (R2 = .000); FC (R2 = .081); HM (R2 = .051); PV (R2 = .003); H (R2 = .003); BI (R2 = .800); UB (R2 = .606).

Figure 2. Structural measurement model

DISCUSSION

This research explored the acceptance of ChatGPT by university students using the UTAUT2 model as a reference. The results showed assent and recognition of the potential of this chatbot in the learning process. This finding aligns with the international strategy established by UNESCO (2019) to promote artificial intelligence in education so that students learn with it and about it to prepare for its impact on different areas of their lives. This study also aligns with the conclusions of previous studies that defend favoring its use, for example, as a first approach to the information content on a subject (Carrasco et al., 2023) , comparing sources (García-Peñalvo, 2023), or experimenting with more personalized learning (Anderson et al., 2023). Acceptance by university students and evidence about its positive impact on the educational process can lead to educational innovations.

As a novel contribution to this field of study, we explored some socio-demographic characteristics affecting the acceptance of ChatGPT by university students. The confirmation of hypotheses H8, H12, and H21 shows that university students' experience of using ChatGPT determines their perception of facilitating conditions, hedonic motivation, and behavioral intention to use ChatGPT. Gender was not a determining variable. Age was only a determinant in perceiving facilitating conditions (H6). This knowledge can lead to adapting practices and experiences according to the needs of university students and to improving the effectiveness of their use in educational settings.

Another important finding is the confirmation of the influence of user experience (H21), hedonic motivation (H9), and habit (H16) on the behavioral intention to use ChatGPT. It is consistent with the magnitude of user adoption in the short time since its inception (Tong & Zhang, 2023). Evidence has also shown how the price value (H13) influences this behavioral intention to use. So far, access and enrollment in this chatbot are free of charge, but if this condition is removed, it may affect the equity of student use (Kasneci et al., 2023). On the other hand, performance expectancy (H1) also determined the behavioral intention to use. This data was linked to the probability of correct answers to official exam questions (Carrasco et al., 2023). This information helps understand university students' perception of ChatGPT and its influence on the intention to use it in education.

Finally, this research confirmed that facilitating conditions (H5), habit (H17), and intention to use this tool (H22) were conditioning factors of user behavior. This result is explained by the complexity of the language model developed (Kung et al., 2023), giving rise to the ease of access and use, the variety of brief and concrete responses, the number of information sources, user confidentiality, and compatibility with other devices, among others (Carrasco et al., 2023; Rozencwajg & Kantor, 2023). Furthermore, this research indicates the need to study in depth the factors that condition the use of AI tools such as ChatGPT for proposals adapted to training demands and positively impact the quality of education.

CONCLUSIONS

This study has reported on the acceptance of ChatGPT by university students. In terms of the socio-demographic variables of university students, the experience of use (user experience) is the fundamental determinant of this acceptance. On the other hand, usage experience, performance expectancy, hedonic motivation, private value, and habits influence students' behavioral intention to use this artificial intelligence chatbot prototype. More specifically, facilitating conditions, habit, and behavioral intention were factors conditioning the user behavior.

These findings have implications for research and educational practice in higher education. On the one hand, they offer an approach to understanding student perception and behavior in the face of this technological advance within the educational sciences. The acceptance of ChatGPT by university students is due to their perception of the potential use of this technology in the learning process. This evidence is interesting in the context of the debate currently taking place within Spanish universities on developing policies and strategies around the functionality and use of ChatGPT. Moreover, this acceptance of ChatGPT by university students is relevant information to the teaching staff for decision-making and rethinking their teaching and training. Regarding the latter, it seems imperative to train students in the ethical and responsible use of ChatGPT, its potential and limitations (considering that it only complements learning), and the ability to formulate clear and specific questions and verify the responses.

The limitations of this research are those of cross-sectional studies; they provide valuable information, but the results need cautious interpretation for several reasons, including the difficulty of inferring causality and the inability to assess changes over time. Despite these limitations, the results of this study provide an essential first approach to understanding and adapting chatbots in the field of education. Given the short time since the creation of ChatGPT, more research is needed to demonstrate its impact on the quality of the educational process. Further studies must determine the functionality and use of ChatGPT by university students in the learning process to develop proposals to improve its effectiveness. Finally, using this artificial intelligence chatbot prototype within the educational process in higher education necessarily entails exploring the perception and training of university faculty.

AUTHOR CONTRIBUTION

Conceptualization: FLL; Data curation: MBF; Formal analysis: JMRR; Funding acquisition: MSRM; Investigation: FLL, MBF, JMRR and MSRM; Methodology: MBF and JMRR; Project administration: JMRR and MSRM; Resources: FLL and MBF; Software: JMRR; Supervision: MSRM; Validation: FLL; Visualization: MBF; Writing – original draft: FLL, MBF, JMRR and MSRM; Writing – review & editing: FLL, MBF, JMRR and MSRM.

ACKNOWLEDGEMENTS

The authors acknowledge the financial support of Tecnologico de Monterrey through the "Challenge-Based Research Funding Program 2022". Project ID # I001 - IFE001 - C1-T1 – E.

The authors acknowledge the technical support of the Writing Lab, Institute for the Future of Education, Tecnologico de Monterrey, Mexico, in the production of this work.

REFERENCES

  1. Anderson, N., Belavy, D. L., Perle, S. M., Hendricks, S., Hespanhol, L., Verhagen, E. & Memon, A. R. (2023). AI did not write this manuscript, or did it? Can we trick the AI text detector into generated texts? The potential future of ChatGPT and AI in Sports & Exercise Medicine manuscript generation. BMJ Open Sport & Exercise Medicine, 9(1). http://dx.doi.org/10.1136/bmjsem-2023-001568
  2. Arista, A. & Abbas, B. S. (2022). Using the UTAUT2 model to explain teacher acceptance of work performance assessment system. International Journal of Evaluation and Research in Education, 11(4), 2200–2208. https://doi.org/10.11591/ijere.v11i4.22561
  3. Bollen, K. A. (1989). Structural equations with latent variables. John Wiley & Sons. http://dx.doi.org/10.1002/9781118619179
  4. Byrne, B. M. (2013). Structural Equation Modeling With AMOS: Basic Concepts, Applications, and Programming, Second Edition Multivariate Applications Series. Taylor & Francis.
  5. Carrasco, J. P., García, E., Sánchez, D. A., Estrella-Porter, P. D., Puente, L. D. L., Navarro, J. & Cerame, A. (2023). Is "ChatGPT" capable of passing the 2022 MIR exam? Implications of artificial intelligence in medical education in Spain. Revista Española de Educación Médica, 4(1), 55–69. https://doi.org/10.6018/edumed.556511
  6. Crawford, J., Cowling, M. & Allen, K. (2023). Leadership is needed for ethical ChatGPT: Character, assessment, and learning using artificial intelligence (AI) Journal of University Teaching & Learning Practice, 20(3). https://doi.org/10.53761/1.20.3.02
  7. Curtis, N. (2023). To ChatGPT or not to ChatGPT? The Impact of Artificial Intelligence on Academic Publishing. Pediatric Infectious Disease Journal, 42(4), 275. https://doi.org/10.1097/INF.0000000000003852
  8. Dwivedi, Y. K., Kshetri, N., Hughes, L., Slade, E. L., Jeyaraj, A., Kar, A. K., ... Wright, R. (2023). So what if ChatGPT wrote it? Multidisciplinary perspectives on opportunities, challenges, and implications of generative conversational AI for research, practice, and policy. International Journal of Information Management, 71, 102642. https://doi.org/10.1016/j.ijinfomgt.2023.102642
  9. Engen, B. K. (2019). Understanding social and cultural aspects of teachers' digital competencies. Comunicar: Revista científica iberoamericana de comunicación y educación, 27(61), 9–19. Retrieved from https://doi.org/10.3916/C61-2019-01https://doi.org/10.3916/C61-2019-01
  10. Gansser, O. A. & Reich, C. S. (2021). A new acceptance model for artificial intelligence with extensions to UTAUT2: An empirical study in three segments of application. Technology in Society, 65, 101535. https://doi.org/10.1016/j.techsoc.2021.101535
  11. García, M., Sarmiento, J. R. & Antonovica, A. (2022). Application and extension of the UTAUT2 model for determining behavioral intention factors in use of the artificial intelligence virtual assistants. Frontiers in Psychology, 13, 993935. https://doi.org/10.3389/fpsyg.2022.993935
  12. García-Martínez, I., Fernández-Batanero, J. M., Fernández-Cerero, J. & León, S. P. (2023). Analysing the Impact of Artificial Intelligence and Computational Sciences on Student Performance: Systematic Review and Meta-analysis. Journal of New Approaches in Educational Research, 12(1), 171–197. https://doi.org/10.7821/naer.2023.1.1240
  13. García-Peñalvo, F. J. (2023). The perception of Artificial Intelligence in educational contexts after the launch of ChatGPT: Disruption or Panic? Education in the Knowledge Society, 24, 1–9. https://doi.org/10.14201/eks.31279
  14. Graf, A. & Bernardi, R. E. (2023). ChatGPT in research: Balancing ethics, transparency, and advancement. Neuroscience, 515, 71–73. https://doi.org/10.1016/j.neuroscience.2023.02.008
  15. Hair, J. F., Black, W. C., Babin, B. J., Anderson, R. E. & Tatham, R. L. (2006). Multivariate Data Analysis. Upper Saddle River.
  16. Hair, J. F., Hult, G. T. M., Ringle, C. & Sarstedt, M. (2017). A primer on partial least squares structural equation modeling (PLS-SEM) (2nd). Sage.
  17. Hassan, M. S., Islam, M. A., Yusof, M. F. B., Nasir, H. & Huda, N. (2023). Investigating the determinants of Islamic mobile FinTech service acceptance: A modified UTAUT2 approach. Risks, 11(2), 40. https://doi.org/10.3390/risks11020040
  18. Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Kutyniok, . . & G. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences, 103. https://doi.org/10.1016/j.lindif.2023.102274
  19. Khechine, H., Raymond, B. & Augier, M. (2020). The adoption of a social learning system: Intrinsic value in the UTAUT model. British Journal of Educational Technology, 51(6), 2306–2325. https://doi.org/10.1111/bjet.12905
  20. Kline, R. B. (2005). Principles and practice of structural equation modeling (2nd). Guilford Press.
  21. Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., Leon, L. D., Elepaño, C., Madriaga, M., Aggabao, R., Díaz-Cándido, G., Maningo, J. & Tseng, V. (2023). Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLOS Digital Health, 2(2). https://doi.org/10.1371/journal.pdig.0000198
  22. Lim, W. M., Gunasekara, A., Pallant, J. L., Pallant, J. I. & Pechenkina, E. (2023). Generative AI and the future of education: Ragnarök or reformation? A paradoxical perspective from management educators. International Journal of Management Education, 21(2), 100790. https://doi.org/10.1016/j.ijme.2023.100790
  23. Mardia, K. V. (1970). Measures of multivariate skewness and kurtosis with applications. Biometrika, 57(3), 519–530. https://doi.org/10.1093/BIOMET/57.3.519
  24. O'Connor, S. (2022). Open artificial intelligence platforms in nursing education: Tools for academic progress or abuse? Nurse Education in Practice, 66, 103537. https://doi.org/10.1016/j.nepr.2022.103537
  25. Ratchford, B. T. (1987). New insights about the FCB grid. Journal of Advertising Research, 27(4), 24–38.
  26. Rozencwajg, S. & Kantor, E. (2023). Elevating scientific writing with ChatGPT: A guide for reviewers. Anaesthesia Critical Care and Pain Medicine, 42(3). https://doi.org/10.1016/j.accpm.2023.101209
  27. Sallam, M. (2023). The Utility of ChatGPT as an Example of Large Language Models in Healthcare Education, Research and Practice: Systematic Review on the Future Perspectives and Potential Limitations. MedRxiv, 2(21), 1–34. https://doi.org/10.1101/2023.02.19.23286155
  28. Salvagno, M., Taccone, F. S. & Gerli, A. G. (2023). Can artificial intelligence help with scientific writing? Critical Care, 27(1), 99. Retrieved from https://doi.org/10.1186/s13054-023-04380-2https://doi.org/10.1186/s13054-023-04380-2
  29. Shinners, L., Aggar, C., Grace, S. & Smith, S. (2019). Exploring healthcare professionals' understanding and experiences of artificial intelligence technology use in the delivery of healthcare: An integrative review. Health Informatics Journal, 1460458219874641. https://doi.org/10.1177/1460458219874641
  30. Sohn, K. & Kwon, O. (2020). Technology acceptance theories and factors influencing artificial Intelligence-based intelligent products. Telematics & Informatics, 47, 101324. https://doi.org/10.1016/j.tele.2019.101324
  31. Stage, F. K., Carter, H. C. & Nora, A. (2010). Path Analysis: An Introduction and Analysis of a Decade of Research. The Journal of Educational Research, 98(1), 5–13. https://doi.org/10.3200/JOER.98.1.5-13
  32. Stokel-Walker, C. (2022). AI bot ChatGPT writes smart essays - should professors worry? Nature . https://doi.org/10.1038/d41586-022-04397-7
  33. Stokel-Walker, C. & Van Noorden, R. (2023). What ChatGPT and generative AI mean for science. Nature, 614, 214–216. https://doi.org/10.1038/d41586-023-00340-6
  34. Tamboleo-García, R. (2023). Teaching innovation faced the challenges of the limitations of the covid-19 pandemic for industrial and laboral sociology. HUMAN REVIEW. International Humanities Review, 18(1), 1–8. https://doi.org/10.37467/revhuman.v18.4862
  35. Tlili, A., Shehata, B., Adarkwah, M. A., Bozkurt, A., Hickey, D. T., Huang, R. & Agyemang, B. (2023). What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education. Smart Learning Environments, 10(1), 15. https://doi.org/10.1186/s40561-023-00237-x
  36. Tong, Y. & Zhang, L. (2023). Discovering the next decade's synthetic biology research trends with. ChatGPT. Synthetic and Systems Biotechnology, 8(2), 220–223. https://doi.org/10.1016/j.synbio.2023.02.004
  37. Torres-Salinas, D. & Arroyo-Machado, W. (2023). ChatGPT en la universidad: usos prácticos en diferentes contextos académicos. Retrieved from https://www.youtube.com/watch?v=oJultNCHuAM&t=22s
  38. Uncovska, M., Freitag, B., Meister, S. & Fehring, L. (2023). Patient acceptance of prescribed and fully reimbursed mHealth apps in Germany: An UTAUT2-based online survey study. Journal of Medical Systems, 47(1), 14. https://doi.org/10.1007/s10916-023-01910-x
  39. UNESCO. Informe de los Objetivos de Desarrollo Sostenible. Retrieved from https://bit.ly/34nbq60
  40. Venkatesh, V., Morris, M. G., Davis, G. B. & Davis, F. D. (2003). User Acceptance of Information Technology: Toward a Unified View. MIS Quarterly, 27(3), 425–478. https://doi.org/10.2307/30036540
  41. Venkatesh, V., Thong, J. Y. & Xu, X. (2012). Consumer acceptance and use of information technology: extending the unified theory of acceptance and use of technology. MIS Quarterly, 36(1), 157–178. https://doi.org/10.2307/41410412