Facilitating educational contents of different subjects with context- agnostic educational game: A pilot case study

Article history: Received 27 October 2019 Revised 25 February 2020 Accepted 26 February 2020 Published 4 April 2020 Educational games are increasingly popular and successful in facilitating various subjects and educational topics. Developing the games are often costly as the developers need to develop specific game content to facilitate each different educational content. One solution to the cost problem is the context-agnostic approach, which allows a game to facilitate educational contents of different subjects or educational topics with zero or minimal modifications to its game content. However, researches on the approach are still scarce, including those aimed at examining real-world applications of the approach. This case study was intended to fill the research gap by examining the application of a context-agnostic educational platformer game in a higher education scenario. The game was used to present educational contents of binary number system and ASCII codes, where both are taught in Informatics Engineering Department. The game was contextagnostic because it was able to flexibly assign educational content elements to various objects in its gameplay. Activity Theory-based Model of Serious Games (ATMSG) was used to guide the design of our educational game. The experiment was conducted with the participation of 60 first-year students of Informatics Engineering Department. The results show that the game can facilitate a more fun and engaging class session for the two educational topics without requiring any changes to the game content. The implication of the results and the implementation complexity of the context-agnostic approach are also discussed.


Introduction
Digital games are a special type of software as they are mainly intended to entertain their users [1]. This "core characteristic" of digital games has remained consistent since the software's inception, although digital games have also acquired additional purposes over the years. These purposes include education [2,3], health promotion and rehabilitation [4,5], and even marketing [6,7]. The aforementioned "core characteristic" of digital games allows them to induce a deep emotional state called "Flow" [8], which makes mundane activities less dull [9].
Among those "games with serious purposes," educational games are one of the most popular. The games have seen promising growth as their acceptance by organizations, both educational and noneducational, is rising significantly [10]. One factor that may hinder the growth is the cost of developing an educational game [11], which can be prohibitive to institutions and practitioners with limited budgets [12]. Finding and studying methods and approaches to keep the cost minimal is, therefore, an important research goal. One such approach is termed context-agnostic, which allows an educational game to facilitate different educational contents flexibly. However, the effectiveness of the approach is still not well-understood as it has received little attention from researchers and practitioners.
This paper aims to fill the research gap through a case study of a real-world application of a context-agnostic game. Platformers were chosen as the game's genre and the game was applied in a hig-So far, educational games have been used mainly to accommodate the learning process of various subjects and topics, with skill acquisition and behavioral adjustment become other important yet less pursued goals [11]. Science, Technology, Engineering, and Maths (STEM) are the most popular subjects to be taught via educational games [11]. Not only in academic activities, but educational games have also been used in extracurricular contexts such as learning cultural heritage [13] and post-disaster management [14]. Meanwhile, games for professional skill training are increasingly used in various fields such as business [15,16] and healthcare [17,18].
On the matter of cutting the development cost, two approaches exist in the literature. The first is through building software architectures that support component reuses. It was achieved by Van Der Vegt et al. with RAGE architecture [19], Stavrev et al. with a modular and multi-input architecture [20], and Carvalho et al. with a service-oriented architecture (SoA) framework [21]. The other approach to cost efficiency is through the interplay between two types of contents of educational games: educational content and game content [22]. In the beginning, the game developer prepares the educational content. The experts of the subject or educational topic may need to work alongside the developer during the preparation. Based on the gameplay requirements of the educational content, the developer will then develop suitable game content to facilitate the educational content. More often than not, the whole process requires considerable resources, especially with more complex gameplay requirements.
A significant increase in cost efficiency may be achieved by allowing the same game content, with minor or even no modifications, to facilitate educational contents of various subjects or educational topics. Such an approach was termed "loose coupling" by Ku et al. [23] and "context-agnostic" by Baron et al. [24]. The approach involves a partial, instead of full, connection between game content and educational content, i.e., some parts of the game content are purely for gameplay purposes and do not affect, and be affected by, the educational content. Baron et al. [24] experimented with using a puzzle game to facilitate contents on math and spelling topics; the educational contents were connected to the elements of the game (pieces of the puzzles) but not to the game's core mechanics. Rosyid et al. proposed a development framework that operated on the same principle as the game of Baron et al., i.e., connecting individual game objects to educational content while leaving the core gameplay mechanics unchanged [22].
Here we can identify one research gap related to the context-agnostic approach: there is still no studies on the real-world application of the approach. Although Ku et al. described the concept of "loose coupling" in their work, they did not experiment on facilitating educational contents of different subjects or topics with the same game. On the other hand, Baron et al. conducted the experiment but not in a real-world scenario. Rosyid et al. developed a dungeon crawler game to demonstrate their framework, but they, too, did not use different educational contents in their experiment.
Meanwhile, to ensure that the educational game can achieve its educational goals, the developer needs to map the components of the educational content to those of the game content. The developer may base the mapping process on models such as Learning Mechanics-Game Mechanics (LM-GM) [25]. The model lists a large number of standard learning mechanics and game mechanics from education and game literature, and a game designer can mix and match the mechanics as needed. The model was expanded into Activity Theory-based Model of Serious Games (ATMSG), which also includes instructional components [26]. ATMSG describes an educational game as a sequence of activities, each composed of sequences of actions, tools, and objectives.

Platformer games
Despite being one of the oldest, platformer games are also one of the most popular game genres ever. The genre has also been utilized in serious and educational games. Fanfarelli and Vie developed Medulla [27], a platformer game that teaches brain structure and function to undergraduate psychology students. Fuchslocher et al. developed Balance, a platformer game for teenagers with diabetes mellitus [28]. Hendrix et al. implemented dynamic difficulty adjustment in a serious platformer game [29]. El-Habr et al. developed a serious platformer game for physical health promotion [30]. One of the earliest commercial educational platformer games is Math Rescue, which was released in 1992 and taught basic arithmetic operations to children (https://3drealms.com/catalog/mathrescue_22/).
As a well-researched game genre, the characteristics of platformer games have been dissected rigorously. It has been known that the gameplay of a platformer game consists of several basic object types: the player's character, platforms, obstacles, movement aids, collectible items, and triggers [31]. The object types are also found in other two-dimensional games [32]. For the game application experiment, we picked platformer as the game genre and we narrowed down the learning scope in our game to cover only question-and-answer activities. Our game, therefore, is essentially a game-based tool for teachers to conduct quizzes or tests. The game presented educational contents of two topics that are taught in the department. The objective of the experiment was to find out the benefits of utilizing our game to present the educational contents, and whether the benefits could be achieved under the context-agnostic attribute, i.e., with no modifications to the game's game content.

Method
We utilized pre-test, post-test, and learning questionnaires to collect data and we analyzed the data with result comparisons between groups of experiment participants and correlation analysis. We also observed how the different educational contents affected the play session of each participant.

Design of the platformer game
The game has five object types: the player's character, enemies, collectible items, platforms, and triggers. One trigger appears in each level: a white flag that serves as the level end position. Figure 1 shows the sequence of our game and its ATMSG components, and Figure 2 shows some screenshots of our game.
The game is composed of several levels, each with educational content in the form of a question. The game displays a level introduction screen before each level, which informs the player about the incoming level's educational topic. The player will then play the level and try to answer the presented question correctly. An in-game interface displays the educational topic, the question, and the player's answer so far. The way for the player to "input" their answer is by eliminating enemies and collecting items. Each enemy and item carries an answer element, and the player must gather every element of the correct answer in the correct order. For example, the question "What is 1500 plus 20?" can be answered by collecting the numbers 1, 5, 2, and lastly 0. We can, therefore, categorize our game as a hybrid puzzleplatformer game, with the puzzles in the form of educational quizzes or tests.
The player is free to exit the level anytime by touching the white flag object. A level result is displayed afterward, which shows the player's answer, the correct answer, and whether the player's answer is the correct one. If there are more levels to be played, the game returns to the level introduction screen; if not, the game result screen is displayed instead, which evaluates the player's performance.

Encoding of educational content
In our game system, educational content is encoded into a machine-readable format stored in text files. Figure 3 shows an example of the encoding. The person performing the encoding has to include six types of information in the text files: the educational topic, the question, the number of correct answer elements, the correct answer elements, the number of extra answer elements, and the extra answer elements. The encoder has to split the correct answer to the question into its elements. The elements will also need to be arranged in their correct order, from the first to the last element, in the text file. The extra answer elements are composed of two elements: duplicates of the correct answer elements and wrong answer elements. The duplicates allow the player to find the correct answer elements more easily, whereas the wrong answer elements serve to make the game more challenging. Figure 4 shows the flow of presenting educational content in our game. The actor who starts the flow is an expert of the subject or educational topic, who composes the educational content and performs the encoding of the content as mentioned. The educational content text files are then read by the educational content module, which is a part of the game software. The module assigns each of the correct and the extra answer elements to a specific enemy or collectible item in the level. The game will then present the level containing the answer elements to the player. As shown in Figure 2, the answer element assigned to an object is displayed above the object's sprite in red.

Flow of educational content presentation
Every level in the game uses the same level design. The answer element assignments in the game are done in a context-agnostic manner, which means the levels are not modified to accept the answer elements. In our experiment, there were 30 enemies and items in a level, so the correct and extra answer elements could not surpass that number. If the total number of the elements were less than 30, some of the enemies and items would be "blank," i.e., not hosting any answer elements.

Design of the experiment
The experiment was performed in two class sessions. We took several best practices of educational game assessment to guide the experiment [33]: 1) The experiment should incorporate a pre-test; 2) The pre-test and the post-test should include similar, but not the same, questions; 3) The participants need to be split into an experimental group, who uses the game, and a control group, who does not use the game, especially if the experiment aimed to compare the game to another method of learning; 4) The number of participants should not be less than 20 per group; 5) Excluding the inclusion of the game, the conditions for the experimental and the control group should be identical.

Educational topics for the experiment
The experiment employed Digital System course material taught in 3rd semester in the Informatics Engineering Department of University of Pembangunan Nasional (UPN) "Veteran" Jawa Timur. From the course material, we took Binary Numbers and ASCII Codes as the educational topics in the experiment. The topics are suitable to be learned through a platformer game because they do not require complex calculations or in-depth analysis. For the ASCII Codes topic, participants were required to memorize a number of ASCII codes to help them converting letters and numbers into their ASCII codes. For binary numbers, participants learned the difference between a binary system and a decimal system, how to convert a number in one system to the other, and how to do basic arithmetic operations on binary numbers.

Experiment participants
First-year students of the Informatics Engineering Department of UPN "Veteran" Jawa Timur participated in the experiment. We split the participants into the experimental group, called Group A, and the control group, called Group B. Each group learned the educational topics through a class session as explained in Section 3.7. We were able to gather 26 participants for Group A and 34 participants for Group B. Figure 5 shows the sequence of a class session. It began with the participants filling out the biodata questionnaire, as seen in Table 1, and followed by the pre-test. A learning session began afterward, which consisted of a cycle of a short lecture on one of the educational topic followed by a short handson practice. The cycle went on until the learning session finished and the post-test began. The last part of the class session was filling out a learning questionnaire, which measured each participant's learning experience.

Sequence of classes
The difference between Group A's and Group B's class session was in the hands-on practice. Group A practiced by playing our game and Group B practiced with pen and paper. The pre-test and the post-test each consisted of 10 questions, five for each educational topic. Both tests had different questions but we kept the problem types and difficulty level the same. We based our learning questionnaire on EGameFlow model [34], which has been widely used to measure the quality of educational games. Table 2 shows our questionnaire items and the corresponding EGameFlow contents and factors. The scores would range from 1 to 5 as the scoring was based on Likert Scale.

Learning questionnaire
As the goal of the questionnaire was to compare the effectiveness of Group A's class session to that of Group B, we selected several EGameFlow contents (measurement criteria) which were applicable to both game-supported and traditional class sessions. Every chosen EGameFlow content became the basis of two identical questionnaire items, one for each educational topic, except for that of the "immersion" factor. We felt that every content of the factor was applicable to both class sessions, except for "I feel viscerally involved..." which would not make sense in a traditional class session. We translated the whole "immersion" factor into a question on whether the class session was fun, as immersion has been closely linked with fun factor in digital game literature.

Result and Discussion
During both class sessions, the participants felt that the ASCII Codes topic was harder than the Binary Numbers one. This was due to the relatively short duration of each class session and the ASCII Codes topic requiring the participants to memorize several codes. The difference in difficulty resulted in lower post-test scores of ASCII Codes topic for both groups. Table 3 shows the background information of the participants, as collected through the biodata questionnaire. An "St. Dev." column shows the standard deviations in a group. There were significantly more technical school graduates in Group B than in Group A, which understandably resulted in more concentration of prior knowledge on Digital System and Binary Numbers in Group B. However, Group A surpassed Group B by a slight margin on the matter of study time length. Of Group A, a majority of the participants were game players, although their dedications and proficiencies varied quite a bit. Most of the participants also had prior experiences with platformer and educational games, although only a small number of them showed preferences toward the games.  There are four items where Group A scored higher (in light green), and there are also four items where the opposite happened (in light red). Group B also slightly surpassed Group A in total score. However, the main takeaway here is that Group A scored better than Group B in item no. 7 and 8 (whether or not the learning process was fun). While various factors in the class sessions (the quality of the lectures, minor technical difficulties, etc.) may have influenced the scores for item no. 1 to 6, item no. 7 and 8 were directly influenced by the inclusion of our game. We can, therefore, conclude that our game was successful in facilitating a more fun and engaging learning process. It even seems that our game was able to make a difficult topic (ASCII Codes) less burdensome to learn.  Table 5 shows the results of the pre-test and post-test. It can be seen that Group A scored worse overall than Group B and only acquired better scores in two, out of nine, items (in light green). What caused the worse post-test results of Group A? Were the results correlated with the participants' educational backgrounds (as reflected by biodata questionnaire item no. 1 to 5)? We investigated the matter by performing a Spearman correlation analysis between post-test results and the questionnaire items. We turned each question of prior Digital System, Binary Number, and ASCII Codes knowledge into a threescale scoring where 0 meant "does not have any prior knowledge," 1 meant "has basic prior knowledge," and 2 meant "has more advanced prior knowledge". The results in Table 6 shows that there were no strong correlations between being graduated from a technical school, having prior knowledge on the learned topics, and study time length and acquiring a good post-test score. Except for that of study time length, it is curious that the correlations were significantly stronger, although still not sufficiently strong, in Group A. In other words, it seems that Group A participants may have had to rely more on their prior knowledges and school experiences, whereas those of Group B could just follow along in their class session to get good results. If it is true, then what may be causing the difference? Was Group A's class session of lower quality than that of Group B? Or were learning capabilities of Group A participants to blame? We investigated this further by performing another Spearman correlation analysis, this time between post-test and learning questionnaire results. Table 7 shows the analysis results.

Pre-test and post-test results
Based on the analysis, it does not seem that a participant's learning experience in the class session, reflected by their learning questionnaire result, is a good indicator of their post-test score. It can, therefore, be said that neither our game nor the topic lectures should be suspected of causing the worse post-test results. This leaves the learning capability of each participant as the probable cause, a factor that is outside the scope of our experiment.

Effects of game proficiency and preference
We analyzed if there was any correlation between a participant's game proficiency and preferences, reflected by their biodata questionnaire, and their learning experience, reflected by their learning questionnaire scores. Table 8 shows our analysis results. Of the biodata questionnaire, the relevant items were item no. 6 (game proficiency), 7 (preference for platformer games), and 8 (preference for educational games).
As biodata questionnaire item no. 7 and 8 were on three-point scales (has never played, has played, and likes playing or frequently plays), we turned item no. 6 into a three-point one too. A "no" answer for the item was equal to one (the lowest value). A "yes" answer was equal to a value between one and three depending on the participant's playtime per week, and the longest playtime was equal to three. The next and last step was calculating the Spearman correlation between every pair of biodata questionnaire and learning questionnaire items. It can be seen in Table 8 that the correlations are consistently weak across all 24 item pairs. It shows that a participant's game proficiency and preferences are poor indicators of their learning experience in a class session supported by our game. The implications are twofold: 1) That even non-game players can enjoy learning through our game; 2) That our game is not yet very successful, as a game software, in entertaining game players.

Discussions
As evidenced by Section 4.2's results, the class session supported by our game was more fun and engaging than the fully traditional one. As the player's enjoyment is the "core characteristic" of games, we can conclude that our game is capable enough as a game-based learning tool. However, the results in Section 4.4 show that the fun of the game is still far from ideal. In the future, we may need to significantly improve the gameplay design of our game to make it more fun to dedicated game players.
Outside of the player's enjoyment, we also need to improve our game's capability in increasing a student's comprehension. As it is now, any advantages our game has over traditional, non-game-based ways of learning may be marginal at best. We believe that our game can support learning better by incorporating more components from the ATMSG models.
The context-agnostic aspect of the game also worked well as no changes to the game content were needed to make both Binary Numbers and ASCII Codes topics fun to learn. However, the random assignments of educational content's answer elements to game objects caused the time taken by a participant in a level to vary noticeably. Sometimes the participant found the correct answer elements quickly, as the elements were assigned to objects that were close to each other; at other times, the opposite happened because the elements were spread away from each other.
The difference in the number of answer elements between the educational contents also contributed to the time variance. Due to the natures of binary and decimal number systems, the questions with binary numbers as the answers (decimal numbers conversions to binary numbers, and arithmetic operations on binary numbers) had significantly more answer elements than those with decimal numbers as the answers (binary numbers conversions to decimal numbers, and characters conversions to ASCII codes in decimal forms). The effect of the difference was that the participants would sometimes find the correct decimal numbers clumped together in a small space, whereas the correct binary numbers, due to being much more numerous, would always be more widely spread in the level. Therefore, the time variance was more drastic in levels with decimal numbers as the answer elements.
Possibly due to the hands-on practice sessions being non-competitive, the participants of our experiment did not find the time variance to disturb their enjoyment. Nonetheless, it may need to be anticipated in the future as it makes the game more luck-based and less reliably challenging, and challenge is an essential element of the player's enjoyment [8]. A powerful way to ensure the correct spatial distributions of answer elements is by utilizing procedural content generation (PCG). The game's levels can be generated procedurally with the intended positionings of the answer elements as one of the generation procedure's fitness criteria [35].

Limitations
The limitations of our experiment are threefold. First, as reflected by our game's ATMSG components, the game only utilizes components for testing the player's comprehension. Therefore, the application of the context-agnostic approach for building comprehension is outside of the scope of our experiment. Second, our game can only facilitate educational content with linear arrangements of their answer elements. Therefore, our results may not apply to the application of the context-agnostic approach to facilitate content with less linear answers, e.g., on molecular structures of chemical compounds (which involve branchings). Third, the context-agnostic attribute of our game does not cover user interfaces (UIs). There was no problem in displaying the answer elements of both binary number system and ASCII codes topics, which was due to each of the decimal and binary numbers requiring only a tiny space on the screen. It is likely that answer elements with much longer names or symbols (e.g., the words that make up a sentence in the English grammar topic) will cause UI problems in our game, as their names or symbols may overlap with each other.

Implications for research and practice
The results of this case study imply that the context-agnostic approach is very much feasible and should be applied whenever possible. With the current trend in the educational games field favoring using and developing specific games for specific educational objectives [11], and with development cost being an obstacle to many practitioners, the application of the context-agnostic approach becomes very logical. Practitioners worldwide need to start considering the possibility of applying the approach whenever they conduct the design and development processes of educational games.

Conclusions
We have conducted a case study on the application of a context-agnostic educational game in a higher education scenario. We have developed a context-agnostic platformer game to facilitate quizzes or tests in a class session, and we have allowed the game to accept the educational contents of two different topics in a context-agnostic manner, i.e., without any modifications to the game's game content. We have assessed our game by applying it in a class session alongside a traditional, non-game-based class session. The results of the assessment show that our game is capable of increasing the fun factor of its class session while under the context-agnostic attribute. Our case study is the first to show the feasibility of the context-agnostic approach in a real-world scenario and that the approach is a solution to the development cost problem of educational games.
However, this case study also uncovers the complexity of facilitating various educational contents. We believe the complexity is worth to be addressed in future researches. One possible direction of the researches is toward understanding the structure and components of a context-agnostic educational game. The structure and components can also be designed and analyzed formally, e.g., under the Mechanics Dynamics Aesthetics (MDA) framework [36].

Acknowledgment
This research was supported by a RISDA (Riset Dasar) grant from University of Pembangunan Nasional "Veteran" Jawa Timur.