Application of Analytical Hierarchy Process to Identify the Experience of Online Teaching Platform

Online teaching has increased in the past year due to Covid-19 making the demand for supplementary online teaching platforms, such as the use of Microsoft Teams and Zoom. The aim of this article is to use the Technology Acceptance Model (TAM) (Venkatesh et al. 2003) as a framework for understanding how students perceive an online teaching platform in different environments. The TAM theory suggests that the adoption of technology is determined by the perception of the usefulness of the technology, and ease of use. In other words, students are more likely to use an online teaching platform that they perceive to be user friendly and easy to use. The TAM model has become an important framework for describing the use of course management system and evaluating online teaching as an instruction delivery medium.

TEXT | Adebayo Agbejule
Permalink http://urn.fi/URN:NBN:fi-fe2021111956082

This article explores the critical factors of technology adoption which has been identified in literature and uses the analytical hierarchy process (AHP) to evaluate the critical factors and their priorities in selecting an online teaching tool from students’ perspective.  Currently, there are many web-based learning platforms, such as Microsoft Teams and Zoom that have become supplementary channels for communicating, collaborating and sharing information in virtual learning environment. Researchers (e.g. Buabeng-Andoh, 2018; Weerathunga et al. 2021) have studied many variables, such as perceived ease of use, perceived usefulness, gender, time flexibility, learning flexibility and social isolation that can impact the adoption of web-based learning technologies, and learning abilities of students in a virtual environment.

AHP is a method developed by Saaty (1980) for ranking decision alternatives and selecting the optimal one when the decision maker or a group has several alternatives from which to select. The AHP methodology has been applied in many different fields including human resource planning, selection of renewable energy projects, supplier selection, and adoption of information and communication technology. The AHP methodology breaks down a problem into a hierarchy consisting of three levels : goal, criteria and decision alternatives. In AHP, the decision maker/group decides how each alternative “scores” on a criteria by using pairwise comparison. In a pairwise comparison, the decision maker/group compares two alternatives at a time according to one criteria and indicates a preference. Gupta et al. 2017 summarizes the steps of AHP methodology as follows: (i) structure the problem and develop the analytical hierarchy process model, (ii) collect the data from experts/group; (iii) develop pairwise comparison matrix and  compute the normalized priority weights of main criteria and sub-criteria (iv) check the consistency of each comparison matrix to validate the results; (v) aggregate of all priorities  and (vi) determine the global weights and rank decision alternative. In this article, the  goal is to identify the factors influencing the adoption of  a web-based learning platform. The objective is placed on Level 1, Level 2 comprises of the criteria, and the alternatives are on Level 3. The AHP hierarchical model is shown in Figure 1.

Figure 1. Analytical Hierarchy Model for the selection of on line teaching platform

In this article, the criteria for comparing and Microsoft Teams and Zoom are as follows:

Ease of use: perceived ease of use is “the extent to which a person
believes that using a particular system would be free of effort” (Davis, 1989, p. 320). Students improve their online  learning activities when they perceive that the web-based learning tool is easy to use.

Interaction and collaboration: Researchers have shown that students benefit from using web-based learning technologies that are interactive and requires collaboration between students and teachers (e.g. Parikh and Verma, 2002).

Evaluation tools: the use of the web-based learning tool to provide possibilities for evaluations, such as self test  and quiz (McClelland, 2001)

User friendliness: web technologies are relevant to the educational process when students perceive that they are user fiendly (Mioduser et al. 2000)

The data for the study is collected from the 4th year energy engineering students from Vaasa University of Applied Sciences studying contemporary issues in technology and management. The students were also introduced to the concept of AHP in the selection of innovative projects.  12 students in 3 groups were asked to give their opinions on the importance of factors affecting students’ adoption of learning platforms. AHP is usually employed to obtain opinions from people who have an idea and knowledge about the subject under investigation and therefore, does not require a large sample size (see Appendix 1) for the questionnaire used in the study). Students were asked to give preferences among criteria. In other words, students  stated their preferences between each pair of criteria  using the scales developed by Saaty (1980) shown in Table 1. The responses of the student are presented in a decision matrix.

Explanation of Preference LevelNumerical Scale
If option A (criteria 1)  and  Option B (criteria 2) equally preferred1
If option A (criteria) is moderately more important than option B criteria)3
If option A (criteria) is strongly more important than option B (criteria)5
If option A (criteria) is very  strongly more important than option B (criteria)7
If option A (criteria) is extremely more important than option B (criteria)9
Choose even number for intermediate evaluation2,4,6,8
Table 1. Selecting numerical scales for pairwise comparions

The analysis of the data collected was performed using MS Excel. The aggregation of responses is done using geometric means. The comparison matrix is normalized by dividing each entry by the sum of the corresponding column to obtain a priority vector. The priority vector provides information on relative weights among the criteria being compared. The  results of the decision matrix for the second level criteria is  given in Table 2 below. The priority vector indicates that ease of use and user friendly are the most preferred criteria for students when considering web-based learning technology platform. The least preferred criteria is evaluation tool.

 User Friendly (UF)Interaction and collaboration (IC)Evaluation tool (ET)Ease of use (EOU)Priority vector
User Friendly (UF)1.001.812.151.000.31
Interaction and Collaboration (IC)0.551.001.800.580.18
Evaluation tool (ET)0.460.551.000.770.10
Ease of use (EOU)1.001.725.651.000.39
Table 2. Aggregate score for pairwise comparison of criteria.

Note: comparison matrix is in blue, and priority vector in green

The AHP allows for consistency check and if the consistency ratio is less than 0.1 (10%), then the judgements are acceptable. To check the consistency, the principal eigen is first calculated. The process for determining the consistency ratio is summarised as follow: (i) create the matrix Aw by multiplying the comparison matrix A by its priority matrix w; (ii) create the vector lamda  (λ) by dividing the elements in Aw by the corresponding element w;  (iii) calculate λmax by taking the average of the values in λ;  (iv) calculate the consistency index (CI) as follows: (λmax – n)/ (n-1), where n is the dimension of the matrix and (v) calculate the consistency ratio CR = CI/RI. Saaty proposed a random index table that can be used in determining the consistenct ratio (see Table 3). The consistency index is 0.031, and RI is 0.9 since n, the number of items compared  is 4. This results in a consistency ratio of 0.034 meeting the acceptable level (see Taylor 2016, p. 455 mathematical steps in AHP).

n12345678910
RI000.580.91.121.241.321.411.451.49
Table 3. Random consistency index

The two alternatives considered for the study are Microsoft Teams and Zoom, and a decision is made comparing the alternatives for each criteria. Since the matrix is 2 X 2, the value of RI is zero, and thus the consistency cannot be evaluated. The results are presented in the Tables 4-7 below.

User Friendly (UF)ZoomTeamsPrority Vector
Zoom1.000.610.38
Teams1.631.000.62
Table 4. Comparisons among alternatives on the basis of user friendly

Interaction and Collaboration (IC)ZoomTeamsPrority Vector
Zoom1.000.580.37
Teams1.721.000.63
Table 5. Comparisons among alternatives on the basis of interaction and collaboration

Evaluation Tool (ET)ZoomTeamsPrority Vector
Zoom11.440.59
Teams0.6910.41
Table 6. Comparisons among alternatives on the basis of evaluation tool

Ease of use (EOU)ZoomTeamsPrority Vector
Zoom10.670.40
Teams1.4910.60
Table 7. Comparisons among alternatives on the basis of perceived ease of use

The results presented in Figure 2 shows that  based on user friendliness, interaction and collaboration, and perceived ease of use, Microsoft Teams was the most preferred choice for students. Zoom was the most preferred choice when considering  criteria evaluation tool.

Figure 2. Ranking of web based  learning platform compared to the criteria

The final step in the AHP analysis was to calculate the global weight of the alternatives. The results are presented in Figure 3 below. The graph shows that students preferred Microsoft Teams compared to Zoom. The overall score for Microsoft Teams is 0.59 and Zoom is 0.41.

Figure 3. Global prorities of online teaching platforms

The selection of an online teaching platform is a multi-criteria decision process. The objective of this article is introduce students to tAhe concept of AHP in the course of contemporary issues in technology and management, and to demonstrate the use of AHP in the selection process of an online teaching platform. The results of the study show the students’ preference for Microsoft Teams for online teaching in areas of user friendliness, interaction and collaboration, and perceived ease of use  and preference for Zoom as an evaluation tool. The article shows that multi-criteria decision methods, such as AHP can be employed as a decision making tool and providing guidance in making the appropriate decision for selecting an online teaching platform for students. When the web-based learning platform is user friendly and provides interactive collaboration, students are more likely to adopt and use them.  Future research can be conducted to include teachers, students from business and social and health departments, and different stakeholders in the university, such as administrative staff to make it possible to generalize the results.

AHP QUESTIONNAIRE

Please answer all the questions in Level 2 (Comparison among the criteria)

Criteria 1987654321 equal23456789Criteria 2
User Friendly                 Interaction and Collaboration
User Friendly                 Evaluation tool
User Friendly                 Ease of Use
Interation and Collaboration                 Evaluation tool
Interaction and Collaboration                 Ease of use
Evaluation Tool                 Ease of use
User Friendly, Interaction and Collaboration, Evaluation, Ease of Use

Level 3

Criteria 198765432123456789Criteria 2
Zoom                 Teams
Comparisons among alternatives on the basis of user friendly
Criteria 198765432123456789Criteria 2
Zoom                 Teams
Comparisons among alternatives on the basis of interaction and collaboration
Criteria 198765432123456789Criteria 2
Zoom                 Teams
Comparisons among alternatives on the evaluation tool
Criteria 198765432123456789Criteria 2
Zoom                 Teams
Comparisons among alternatives on the basis of ease of use
References
  • Buabeng-Andoh, C. (2018). Predicting students’ intention to adopt mobile learning. Journal of Research in Innovative Teaching & Learning. Vol. 11 No. 2, pp. 178-191.

  • Davis, F.D. (1989), “Perceived usefulness, perceived ease of use, and user acceptance of information technology”, MIS Quarterly, Vol. 13 No. 3, pp. 319-340.

  • Gupta, K.P., Bhaskar, P., & Singh, S. (2017). Prioritization of factors influencing employee adoption of e-government using the analytic hierarchy process. J. Syst. Inf. Technol., 19, 116-137.

  • McClelland, B. (2001) Digital Learning and Teaching: Evaluation of Developments for Students in Higher Education, European Journal of Engineering Education, 26, 2, 107-115.

  • Mioduser, D., Nachmias, R., Lahav, O., Oren, A. (2000). Web-based Learning Environments: Current Pedagogical and Technological State, Journal of Research on Computing in Education, 33, 1, 55-76.

  • Parikh, M. & Verma, S. (2002). Utilizing Internet Technologies to Support Learning: An Empirical Analysis, International Journal of Information Management, 22, 1, 27-46.

  • Saaty, T.L. (1980). The Analytic Hierarchy Process, McGraw-Hill Book Co, New York, NY.

  • Taylor, B.W. (2016). Introduction to Management Science (Global Edition). Pearson, England. UK

  • Venkatesh, V., Morris, M.G., Davis, G.B. and Davis, F.D. (2003). User acceptance of information technology: Toward a unified view. MIS Q. Manag. Inf. Syst. 27, 425–478.

  • Weerathunga, Prageeth R., W. H.M.S. Samarathunga, H. N. Rathnayake, S. B. Agampodi, Mohammad Nurunnabi, and M. M.S.C. Madhunimasha (2021). "The COVID-19 Pandemic and the Acceptance of E-Learning among University Students: The Role of Precipitating Events" Education Sciences 11, no. 8: 436. https://doi.org/10.3390/educsci11080436