Adapting the Survey of Technological Pedagogical STEM Knowledge to the Turkish Language and Determining the Knowledge of Pre-service and In-service Teachers

The purpose of this study was to adapt a survey developed by Chai, Jong et al. (2019) on technological pedagogical STEM (TP-STEM) knowledge into Turkish and to determine the knowledge of pre-service and in-service teachers about TP-STEM. The original survey consisted of four factors and a total of 17 items. These factors included Technological Pedagogical Knowledge in Science (TPSK), Technological Pedagogical Knowledge in Mathematics (TPMK), Technological Pedagogical Knowledge in Engineering (TPEK), and Integrative STEM (iSTEM) knowledge. The participants of the study were 520 pre-service and in-service teachers. The analysis showed that the model fit indices for the validity of the factor structures were acceptable with a value of RMSEA=0.0621 and showed excellent agreement with the values SRMR=0.0346, CFI =0.961, TLI=0.953. The Cronbach's alpha values for the factors ranged from 0.80 to 0.84 (α -value >.70). These results mean that the survey adapted to Turkish language was reliable and valid for further research. The results showed that pre-service and in-service mathematics teachers had lower self-efficacy on the subfactors (TPSK, TPMK, TPEK, and iSTEM) than pre-service and in-service science and computer teachers.


INTRODUCTION
The only constant in life is change (Heraclitus).Time brings change, and a change in one area can bring about other changes.The need for individuals to acquire new skills over time has led to exploring educational approaches and models that are thought to develop those skills more effectively.In recent years, educational models that integrate multiple disciplines have attracted the interest of scholars (Aranda et al., 2020;Bybee, 2013;English, 2015).The approach STEM, which includes science, mathematics, engineering, and technology, is one of the unique approaches today.STEM Education is mainly expressed as integrated STEM education, which links all disciplines together (Blackley & Howell, 2015).
STEM is the integration of knowledge built between the fields of science, mathematics, engineering, and technology (Chai, Jong et al., 2020;Wang & Fan, 2018).Integrated STEM education aims to improve the quality of education of individuals who use science and technology to achieve specific goals and have problem-solving skills in daily life (Bybee, 2013).It is mainly concerned with learners' explorations and problem-solving in everyday life (Bagiati & Evangelou, 2015), encourage their creativity, and enables them to apply the learned information more in other fields (Tseng et al., 2013).It encourages learners to express their thoughts and learn cooperatively with other class students to solve problems requiring knowledge in science, mathematics, engineering, and technology (Kuo et al., 2019).It also aims to provide students with holistic development (Breiner et al., 2012;Smith & Karr-Kidwell, 2000;Tsupros et al., 2009), increase their curiosity and interest in all areas, and promote the production of innovative technologies (Thomas, 2014).
Researchers have pointed out that pre-service and in-service teachers should have specific STEM teaching competencies to conduct effective instruction.Many researchers have pointed out that teaching STEM can pose pedagogical challenges to many teachers, especially if they do not have adequate knowledge in engineering, technology, and design thinking (Chai, Jong et al., 2019;Faikhamta et al., 2020).For example, Akgündüz et al. (2015) found that teachers with inadequate knowledge, experience, and skills related to STEM education negatively affected students' performance in STEM education.Research studies have shown that teachers feel inadequate concerning STEM (Karademir-Coşkun et al., 2020;Köse & Ataş, 2020;Yıldırım & Türk, 2018) and have an insufficient knowledge base because they have not received professional development courses or induction programs for STEM teaching (El-Deghaidy & Mansour, 2015;Hossain & Michael, 2012;Karademir-Coşkun et al., 2020).These studies also showed that teachers lacked technological pedagogical content knowledge (TPACK) related to STEM education (Karademir-Coşkun et al., 2020;Yanış-Kelleci, 2020).From these findings, there is a need for pre-service and in-service teachers to identify and develop their pedagogical competencies related to STEM.
Given that STEM education is an approach that covers the process from preschool to college (Gonzalez & Kuenzi, 2012), it requires that pre-service and in-service teachers need to have a knowledge base and competencies in all areas of STEM.In addition, pre-service and in-service teachers are expected to implement STEM education in the classroom.From this perspective, it is important to identify and develop the STEM competencies of preservice and in-service teachers in STEM disciplines.For students in the STEM classroom to be successful, creative, critical thinkers, designers, and producers, teachers must have a certain level of knowledge about integrated STEM education and use technology during their classroom instruction.TPACK is teachers' knowledge of incorporating technology into the classroom (Schmidt et al., 2009).Technology is also an integral part of integrated STEM education.From this perspective, STEM and TPACK have a common element of technology (Chai, Jong et al., 2020).In the current research literature, a few studies examined pre-service and in-service teachers' knowledge and skills related to STEM education and their TPACKs from a holistic perspective (Chai, 2019;Chai, Jong et al., 2019;Chai, Jong et al., 2020;Chai, Rahmawati et al., 2020;Çayak, 2019;Rahman et al., 2017;Wang & Fan, 2018).For example, Dadacan (2021) found that the self-efficacy of pre-service science teachers and classroom teachers regarding STEM instruction was better than that of mathematics teachers.The recent studies by Koçak et al. (2019) and Karışan and Bakırcı (2018) found that the orientations of pre-service science teachers toward STEM were better than those of preservice and in-service mathematics teachers.In a recent study, Chai, Jong et al. (2019) found that science teachers performed better than pre-service mathematics teachers.Their results also showed that science teachers had higher scores than mathematics teachers in all factors, including TPMK in TPSK, TPEK, and iSTEM.
All of the knowledge and practices inherent in each STEM discipline have complex structures.The interconnectedness of integrated STEM disciplines and practices makes this structure more complex and dynamic (Chai, Jong et al., 2019;Kelley & Knowles, 2016;Wells, 2016).In this regard, useful, reliable measurement tools that determine teachers' or preservice teachers' TPACKs concerning this complex structure are expected to contribute to the literature.The purpose of this study was to adapt the TP-STEMK survey developed by Chai, Jong et al. (2019) into Turkish language and to determine the self-efficacy level of teachers and pre-service teachers at TP-STEMK.The research questions guiding this study are as follows.
1-) What is the validity and reliability of the adapted TP-STEMK survey in Turkish?2-) Are there statistically significant differences in the subfactors of the TP-STEM knowledge survey according to the program studied by the pre-service teachers?3-) Are there statistically significant differences in the subfactors of the TP-STEM knowledge survey according to the branches of the study of the in-service teachers?

Teachers' Competencies in STEM Education
Teachers' competencies are one of the most important factors affecting the quality of teaching, motivation, and student achievement (Baumert et al., 2010;Park & Oliver, 2008).
Considering the importance of this factor, it is crucial for the implementation of STEM that teachers have a certain level of STEM content knowledge and skills for teaching knowledge.
Research has shown that teachers with insufficient STEM knowledge and skills negatively affect their students ' STEM achievement (Çorlu et al., 2014;Williams, 2011) and have positive attitudes toward STEM (NRC, 2013;Dönmez, 2020).For example, El-Deghaidy and Mansour (2015) found that teachers could not reflect the essence of integrated STEM instruction in their classrooms because they could not adopt the technology concept.In another study, Akgündüz et al. (2015) found that high school students could not successfully use STEM technology because math and science teachers lacked experience and knowledge of STEM education.In addition, Moore and Smith (2014) found that teachers in STEM practices focused on teaching science and mathematics and ignored technology and engineering in their instruction.From this perspective, integrating integrated STEM education in the classroom and teachers' participation in STEM-related professional development activities is very important for the implementation and success of integrated STEM education (NRC, 2013;Srikoom et al., 2017).

Teachers' Self-Efficacies in STEM Education
The concept of self-efficacy was first introduced by Bandura (1977).Bandura defined self-efficacy as a person's individual belief in doing something to be successful.Tschannen-Moran and Woolfolk-Hoy (2001) emphasized that by fostering self-efficacy in teachers, they can be trained to be willing teachers who are successful in their profession.In recent research, Chai, Jong et al. (2019) found that teacher self-efficacy has a multidimensional structure.Teachers' self-efficacy affects their performance, teaching practices, and attitudes toward innovative teaching approaches (Deehan et al., 2017).Studies conducted by researchers show that pre-service and in-service teachers with high self-efficacy are very diligent in training and professional development activities (Usher & Pajares, 2008;Pendergast et al., 2011), open to new ideas and technologies, and willing to use technology in the classroom (Charalambous & Philippou, 2011;Lunenburg, 2011;Smith et al., 2012;Tschannen-Moran & Woolfolk-Hoy, 2001).Hacıömeroğlu (2020) emphasized that teachers with low self-efficacy cannot respond to students' needs.For this reason, it is important to determine teachers' self-efficacy and knowledge levels about integrated STEM instruction (Honey et al., 2014).

STEM Education and Technological Pedagogical Content Knowledge
TPACK is teachers' knowledge of how to integrate technology into the classroom.
Researchers have highlighted that a knowledge base for teaching shapes teachers' instruction while teaching a subject using technological and pedagogical knowledge (Schmidt et al., 2009).The integrated STEM approach reveals an understanding of educating individuals with knowledge and skills by sharing science, design, production, technological tools, and devices (Directorate General of Private Educational Institutions [ÖÖKGM], 2019).
Therefore, integrated STEM is important for understanding teachers' technological knowledge in curriculum and pedagogy (Lin et al., 2021).The role of technology in strengthening STEM education is emphasized by most educators (Chai, Jong et al., 2019).Specifically, integrated STEM is an integrated knowledge base to improve teachers' TPACK awareness and integration of technology in the classroom (Chai, Jong et al., 2019).Thus, TPACK and integrated STEM complement each other in pedagogical goals and STEM disciplines (Parkeret et al., 2015;Milner-Bolotin, 2018;Chai, 2019).integrated STEM education and TPACK highlight the importance of learning in the 21st century (Chai, 2019;Milner-Bolotin, 2018).In addition, researchers have emphasized that TPACK provides a conceptualized knowledge base for instruction (Chai, Koh et al., 2019).This knowledge base requires teachers to know how to use technology to create STEM out-of-school projects, implement strategies to allow students to participate in online activities (TPB), and use subject-specific computer-based activities (TAB).In this context, it is important to identify teacher self-efficacy regarding STEM and TPACK.However, understanding the relationships between STEM and TBAP will facilitate the development of teachers' knowledge, skills, and pedagogical competencies (Chai, Jong et al., 2019).In this regard, technology integration will increase meaningful learning in the STEM classroom.A limited number of research studies have examined teachers' knowledge, skills, and competencies for integrating STEM and TPACK (Chai, 2019;Chai, Jong et al., 2019;Parker et al., 2015).

Teachers' Knowledge of TP-STEMK
In reviewing the literature, qualitative studies have been mainly used to identify STEM-TPACK -related knowledge of pre-service and in-service teachers.Some studies that have used quantitative measurement tools (Çayak, 2019;Phanprom et al., 2021) are limited.
To date, researchers have used different instruments to collect data, including different subfactors of STEM-TPACK measurement.TPACK is a seven-factor model (Mishra & Koehler, 2006).Given that each discipline is in the STEM field and technology, a seven-factor model STEM-TPACK does not seem very practical to use in research.With its practical application, this problem will reduce the usefulness of the data collection tools developed.
In this regard, the four-factor survey Chai, Jong et al. (2019) developed is very useful.In this survey, the factors TPSK (Technological Pedagogical Knowledge in Science), TPMK (Technological Pedagogical Knowledge in Mathematics), and TPEK (Technological Pedagogical Knowledge in Engineering) provide information about teachers' competencies in the technologies they use.The fourth factor was named "integrative STEM" (iSTEM) because integrating these factors contains information about teachers' holistic view of STEM education.The same survey developed by Chai, Jong et al. (2019) has been used by many researchers (Love, & Hughes, 2022;Solina, 2021;Barba-Sánchez et al., 2021).In this study, we aimed to translate the survey developed by Chai, Jong et al. (2019) into Turkish.

Research Model
This research is an adaptation study of a survey.This type of research involves the process of adapting a survey developed in one language to another language by conducting validity and reliability analyzes (Kılıçer & Odabaşı, 2010;Seçer, 2015).In this study, the TP-STEMK survey developed by Chai, Jong et al. (2019) was adapted to the Turkish language.

Participants
The study group was formed using the "appropriate sampling" method, one of the non-probabilistic sampling methods.Appropriate sampling is a technique with voluntary participants that is easily accessible and applicable to researchers due to time, labor, and cost constraints (Canbazoğlu-Bilici, 2019;Teddlie & Yu, 2007).The participants in the study consisted of 657 pre-service and in-service teachers.Of the participants, 54 were involved in the pilot study, and 612 participated in the main study.After identifying missing and extreme values, data was collected from 466 pre-service and in-service teachers.During the adaptation process, one Turkish language expert commented on the first draft of the translated survey, three English language experts, and six experts with a Ph.D. in mathematics and science education.According to the feedback of experts, corrections to the translated survey were done by researchers.The characteristics of the participants are shown in Table 1.

Data Collection Tools TP-STEMK Survey
The TP-STEMK developed by Chai, Jong et al. (2019) was designed to assess preservice and in-service teachers' self-efficacy in integrated STEM education in technological pedagogical content knowledge (TPACK).The original version consisted of four factors and 17 items.Each sub-dimension of the survey, i.e., technological pedagogical knowledge in science (TPSK), technological pedagogical knowledge in engineering (TPEK), and integrative STEM (iSTEM), consisted of four items, while another sub-dimension, technological pedagogical knowledge in mathematics (TPMK), consisted of five items.The survey consists of seven Likert-type items.These items include disagree-1, disagree-2, partially disagree-3, undecided-4, partially agree-5, agree-6, and agree-7.In addition, there is no item in the survey with a negative meaning on the survey TP-STEMK.

Survey's Adaptation to the Turkish Language
After reviewing the relevant literature on STEM education and TPACK, it was determined that there was a need for a data collection tool to assess teacher self-efficacy from a TPACK perspective in the context of holistic education STEM.The researchers searched the literature for the existing instruments based on this finding.After determining the appropriateness of the TP-STEMK self-efficacy survey for research, the researchers asked permission from the authors of the TP-STEMK survey and received permission from Chai, Jong et al. (2019) via email.The researchers initially translated the TP-STEMK developed in English into Turkish.After the first draft of the translation, the researchers received feedback from one Turkish language expert, three English language experts, and two STEM field experts, who formed a six-member translation team and received support.
During the translation assessment, cross-checks were conducted to determine whether differences in the survey were independently translated into Turkish, and a consensus was reached on the common translation.In addition, a Turkish language expert checked the compatibility of the translation with Turkish in terms of spelling and semantic integrity.
During the back translation into the original language, the translation was supported by a bilingual expert and compared and reviewed with the original version.Through the expert's feedback, the items' incomprehensible expressions and expression errors were corrected, and the stages of assessing the survey's structural, content, linguistic, and cultural conformity were completed.The survey was translated into a Likert-type format and presented to 54 pre-service teachers in a pilot study to assess its comprehensibility.The results of the pilot study showed that the factors and item distribution were consistent with those of the original survey.Later, the final version of the translated survey was administered to 612 participants.Participants were asked to answer all items in the survey.
All surveys answered by participants were included in the analysis.No data loss occurred.
The stages of the survey adaptation process to the Turkish language are shown in Figure 1.The following steps were followed in the Turkish adaptation of the TP-STEMK selfefficacy survey:

Data Analysis
The Jamovi program (version 1.6) was used to adapt the survey TP-STEMK (Jamovi Project, 2021).For the survey results administered to the participants, the researchers used the program SPSS 22.0.Regarding the validity of the survey, analyzes were conducted to ensure content validity (content), linguistic validity (linguistic equivalence), and structural validity (concept).For construct validity, confirmatory factor analysis (CFA) was conducted.
The choice between exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) methods in adjustment studies is one of the most common problems.While CFA is used to test the correspondence of items to the defined structure, EFA is used to determine which structures the items cover (Brown, 2015;Karaca et al., 2015;Myers, 2000).In this context, the use of CFA seems appropriate due to the fixed factor structures of the original survey (Byrne & van de Vijver, 2014; Fabrigar et al., 1999;Güngör, 2016).In addition, Cronbach's alpha (α)-coefficient, item-total correlation value, and test-retest reliability coefficient (r) were calculated to test the internal consistency of the factors identified in the original survey, and finally, reliability was tested.As a result of the analysis, it was examined whether the items in the survey could measure the sub-factors.Mahalanobis value and deviant outliers were checked in the data set and removed from the data file.
Since extreme values lead to errors, it is recommended to remove them from the data file (Tabachnick & Fidell, 2013).In addition, the degree of freedom and the chi-square value affect the interpretation of the CFA result as the sample grows (Çokluk et al., 2021).This situation may lead to misinterpretation.Therefore, the fit indices x²/df, CFI, TLI, RMSEA, and SRMR were calculated instead.
Later, the descriptive (arithmetic mean and standard deviation) and inferential statistical data for each variable were calculated.First, we checked whether the distribution was normal by looking at the skewness and kurtosis values (in the range of ±1.96).To reduce the type 1 error and to reveal the significance of the dependent variable (Alpar, 2003), a oneway MANOVA test was used in the analysis.The "Scheffe and Tukey" post hoc test was used as a multiple comparison test.First, the analyzes tested the assumptions of the MANOVA test.For the assumption of the equality of covariance matrices, Box's M test was applied, and in the cases where homogeneity was achieved (Allen et al., 2014;Pallant, 2005), the Wilks-Lambada test was suggested for multivariate test results, and in the cases where homogeneity was not achieved, Pillai's Trace test was recommended (Tabachnick & Fidell, 2013).Assuming equality of Levene error variances, it is assumed that there is no difference between error variances at p >.01 or p >.025 (Tabachnick & Fidell, 2013).In addition, partial eta squared (η2) was used to determine the influence of the independent variable on the dependent variable.Partial Eta squared η2 = .01small effect value, η2 = .06medium effect value, and η2 = .14large effect value (Cohen, 2013).

Ethical considerations
During the study, all guidelines outlined in the Ethics of Scientific Research and Publication in Higher Education Policy were followed, and no actions were taken to the contrary.Faculty members, pre-service, and in-service teachers participated in the study by declaring their voluntary participation via the informed consent form submitted via Google Form.Responsibility for ethical violations in the research rests with the authors.
In this study, all rules stated to be followed within the scope of the "Higher Education Institutions Scientific Research and Publication Ethics Directive" were followed.None of the actions stated under the title "Actions Against Scientific Research and Publication Ethics", which is the second part of the directive, were taken.

Content Validity
To determine the content validity of the adapted survey in Turkish, the researchers obtained feedback from three language experts on the first draft of the translated survey.
The researchers asked these experts to review the translation of each item.The experts reviewed the items and responded to a control form that was given to them.They indicated their feedback on the form as "the item is appropriate or should be corrected as ...".As a result of these feedbacks and reviews, ten items were accepted in their original form.The researchers modified seven items to ensure semantic integrity, linguistic validity, and cultural conformity.For example, the original statement "I can design lessons to integrate interdisciplinary STEM content and technology for student-centered learning appropriately." was translated as "I can design lessons to appropriately integrate holistic STEM content and technology for a student-centered learning approach."translated.
However, in light of expert feedback, the same item was changed to "I can design studentcentered courses that appropriately integrate interdisciplinary STEM content and technology."

Language Validity
After modifying the first draft of the survey based on the experts' feedback, the researchers used the back-translation method to determine the language equivalence of the survey.In the back-translation process, experts who knew the original language of the survey (English) but did not work on the survey (N=3) translated the items from Turkish to English.As a result of this back-translation, consistency and harmony were found between the original survey and the second draft of the survey after the back-translation.Then, the researchers conducted a pilot study with six pre-service and in-service teachers of science, elementary mathematics, and computer.This pilot study showed that the participants had no difficulty understanding the items.

Construct Validity
Since the original survey has a specific factor structure, the use of the CFA seems appropriate (Fabrigar et al., 1999;Güngör, 2016).

Confirmatory Factor Analysis
The purpose of confirmatory factor analysis (CFA) is to test the fit of the model advocated by exploratory factor analysis (EFA) using some values, to test the same model and verify its validity (Brown, 2015;Myers, 2000;Tabachnick & Fidell, 2013;Yaşaroğlu, 2017)  Science and engineering .64;.61 between science and integrative STEM; .72 between mathematics and engineering; .55 between mathematics and integrative STEM; .75 between engineering and integrative STEM.According to Kline (2015), the effect value is 'moderate' around .30 and 'high' from .50 and above.In this context, values above .50mean the effect value between factors is high.

Analysis of the validity of the subfactors
An analysis of Pearson product-moment correlation coefficients for the relationships among the knowledge types formed in the subfactors of the TP-STEMK survey is presented in Table 3. Examination of Table 3 reveals moderate correlation between the survey subfactors (r=.695, r=.650, r=.606 r=.568 and r=.531, r=.497 p < 0.01).In addition, the subfactors that make up the survey are not at extreme values, and the absence of a high correlation level indicates the survey's reliability (Gündüz & Coşkun, 2012).

Internal Consistency
The coefficients of internal consistency (Cronbach's alpha 'α') calculated for a complete survey or its factors are α > 0.70, which means that the survey is reliable (Nunnally & Bernstein, 1994;Büyüköztürk, 2020).The internal consistency coefficients (α) of the subfactors in the original TP-STEMK survey and this study are shown in Table 4.

Analyses for Item-Total Correlation
The overall correlation coefficients of the items calculated for the Turkish version of the survey TP-STEMK ranged from .66 to .91 (Table 5).Researchers recommend that items with an item correlation coefficient of less than .20 should be removed from the measurement instrument (Büyüköztürk, 2020).Since .20 is not a golden value for survey items, the item was not released.Moreover, according to Büyüköztürk (2020), an item with an item correlation coefficient of 0.3 and above is called a good item.So, the results show that all items in the translated survey are good.The results in Table 5 show that the positive correlation between the item-total correlation scores indicates that internal consistency is high and that the items measure similar behaviors (p <.05).

Test-Retest Reliability
To determine the degree of stability of the translated survey, it was administered again to 54 pre-service teachers at four-week intervals.The relationship between the scores obtained on both applications was examined regarding the overall survey, items, and factors, and how stable the final survey was measured.In addition, Büyüköztürk (2020) also indicated that the Pearson moment product correlation coefficient was low between 0.0-0.3,moderate between 0.30 and 0.70, and high between 0.70 and 1.0.The test-retest correlation coefficient regarding the stability level of the survey showed a positive, high, and significant relationship between the two applications [r₍₅₄₎ = .831,p <.01].

Results on Preservice teachers
To answer the research question, "Are there statistically significant differences in the subfactors of the TP-STEM knowledge survey according to the program studied by the preservice teachers?"data were analyzed using a one-way MANOVA.The descriptive results, including the number of data (N), means, and standard deviations (SD) of the variables TPSK, TPMK, TPEK, and iSTEM, depending on the program of study pre-service teachers, are presented in Table 6.Table 6 shows that the mean scores of TPSK, TPMK, TPEK, and iSTEM5 are higher among the pre-service computer teachers than pre-service teachers of science and elementary mathematics.In addition, the skewness and kurtosis values of the TPSK, TPMK, M17 .87 TPEK, and iSTEM subfactors in the pre-service teacher programs show a distribution close to the normal distribution.In this context, the condition of equality of covariance matrices, one of the assumptions of one-factor MANOVA, was tested with Box's test of equality of matrices, and the other assumption, equality of error variances, was tested with Levene's test.In analyzing the Box's M test to assess whether the covariances are equal, Sig.If the pvalue in the line p > is .001, the null hypothesis is accepted (Allen et al., 2014;Pallant, 2005).
This result means that there is no significant difference between the matrices.The fact that there is no significant difference between the covariance matrices of the dependent variable (p= .029,p >.001) indicates that the assumption of one-way MANOVA is met The other postulate of one-way MANOVA was tested using Levene's test to determine if there was a significant difference between the error variances.Levene's test was used in the analysis Sig.If the p-value in the p > row is .01or p > is .025, the null hypothesis is accepted (Tabachnick & Fidell, 2013).This result means that there is no significant difference between the error variances.When testing the error variances of the dependent variables TPSK (p= .213,p >.01), TPMK (p= .056,p >.01), TPEK (p= .943,p >.01), and iSTEM (p= .01).
651, p >.01) showed that the hypothesis "There is no significant difference between the error variants" can be accepted.
The analysis determined that the assumptions of the MANOVA test were met, and a one-way MANOVA analysis was performed.In this context, a multivariate test analysis was performed to determine if there was a difference depending on the program studied.Wilk's Lambada analysis, a type of multivariate analysis (Büyüköztürk, 2020;Green & Salkind, 2005;Tabachnick & Fidell, 2013), was performed.
The Manova results on the TPSK, TPMK, TPEK, and iSTEM responses of pre-service teachers depending on the program they studied showed a significant difference between the dependent variables of the combined TPSK-, TPMK, TPEK, and iSTEM scores depending on the program studied by the pre-service teachers [F₍₈, ₂₉₆₎=9.74,p <.05, Wilk's lambda (Λ)= .627,partial ƞ2=.208].The results of the one-way ANOVA analysis to determine the effects of the independent variables on the dependent variables are shown in Table 7. Table 7 shows a significant difference in the pre-service teachers' TPSK, TPEK, and iSTEM scores according to the course of study (p <.05).It was found that there was no significant difference in TPMK scores according to a degree program (p >.05).In addition, the partial eta squared indicates how much of the change in the dependent variable is explained by the independent variable (Pallant, 2005;Rosenthal & Rosnow, 2008).It can be concluded that the largest effect is for the dependent variables TPEK (partial ƞ2=0.226) and iSTEM (partial ƞ2=0.251).In addition, supplemental analyzes were performed, and a Scheffe posthoc analysis was performed to determine which groups had a significant difference as a function of the program variable examined (Table 8).Table 8 shows that pre-service computer teachers have higher self-efficacy in the TPSK subdimension than pre-service teachers in the primary mathematics program (p <.05).On the TPEK and iSTEM subfactors, pre-service teachers in the science and computer had higher self-efficacy than pre-service teachers in the Primary Mathematics program (p <.05).

Results on In-Service Teachers
A one-way MANOVA analysis was conducted to answer the research question, "Are there statistically significant differences in the subfactors of the TP-STEM knowledge survey by in-service teachers' majors?"The descriptive results, including the number of data (N), mean, and standard deviation (SD) of the TPSK, TPMK, TPEK, and iSTEM variables by teachers' fields of study, are presented in Table 9.Table 9 shows that the TPSK, TPMK, TPEK, and iSTEM scores of computer teachers are descriptively higher than those of science and elementary math teachers.In addition, the skewness and kurtosis values of the TPSK, TPMK, TPEK, and iSTEM scores by teacher branch also show a distribution that is close to the normal distribution.In this context, the condition of equality of the covariance matrices, one of the assumptions of the one-factor MANOVA, was tested with the Box test for equality of the matrices, and the other assumption, equality of the error variances, was tested with Levene's test.
The null hypothesis is accepted in the analysis of Box's M-test to determine if the covariances are equal if the p-value in the line p > is .001(Allen et al., 2014;Pallant, 2005).
This result means that there is no significant difference between the matrices.Accordingly, it was found that there is no significant difference between the covariance matrices of the dependent variable (p= .011,p >.001), and the assumption of one-way MANOVA is satisfied.
The other postulate of one-way MANOVA was tested using Levene's test to determine if there is a significant difference between the error variances.If the p-value in Levene's test analysis is p >.01 or p >.025, the null hypothesis is accepted (Tabachnick & Fidell, 2013).This result means that there is no significant difference between the error variances.Examination of the values for TPSK (p= .566,p >.01), TPMK (p= .036,p >.01), TPEK (p= .057,p >.01), and iSTEM ( p= .200,p >.01) shows that the hypothesis "There is no significant difference between error variances" can be accepted.Based on the analysis, it was determined that the assumptions of MANOVA were met, and a one-way MANOVA analysis was performed.In this context, a multivariate test analysis was performed to determine if there was a difference depending on the program studied.Wilk's Lambada analysis of multivariate analysis of research (Büyüköztürk, 2020;Green & Salkind, 2005;Tabachnick & Fidell, 2013) was conducted.
When the Manova results of teachers' TPSK, TPMK, TPEK, and iSTEM scores were examined together by teacher branch, there was a significant difference in the subfactors of TPSK, TPMK, TPEK, and iSTEM together by teacher branch [F₍₈,₆₁₂₎ = 13.609,p <.05, Wilk's lambda (Λ)= .721,partial ƞ2=.151].Indeed, it was found that teachers' TPSK, TPMK, TPEK, and iSTEM ratings by their branches were not similar when considered together.The results of the one-way analysis ANOVA, which was conducted to determine the effect of the independent variables on the dependent variables, are presented in Table 10.Table 10 shows a significant difference in teachers' TPSK, TPMK, TPEK, and iSTEM scores depending on their branch (p <.05).Teachers' branches significantly impact on all subfactors.In addition, the partial eta squared indicates how much of the change in the dependent variable is explained by the independent variable (Pallant, 2005;Rosenthal & Rosnow, 2008).It can be concluded that the smallest effect is due to the dependent variable TPMK (partial ƞ2=0.047).Furthermore, supplemental analyses were conducted, and a Scheffe post hoc analysis was performed to determine which groups had a significant difference concerning the program variable under study (Table 11).Table 11 shows a significant difference between computer teachers and elementary school mathematics and science teachers on the TPSK, TPMK, and TPEK subfactors in favor of computer teachers (p <.05).Computer teachers were found to have higher self-efficacy in TPSK, TPMK, and TPEK.In addition, science teachers were found to have higher self-efficacy in the TPSK, TPEK, and iSTEM subfactors than elementary mathematics teachers (p <.05).In the iSTEM subdimension, computer teachers were found to have significantly higher self-efficacy than elementary mathematics teachers.

Discussion and conclusion on Survey Adaptation
In this research, an attempt was made to translate the TP-STEMK survey into Turkish by conducting validity and reliability studies.For the construct validity of the questionnaire, the use of DFA was preferred due to the factor structure of the original scale (Fabrigar et al, 1999;Güngör, 2016).The purpose of the DFA is to test the significance and accuracy of the factor structure identified by the AFA and to determine whether the results confirm the model (Brown, 2015;Hair et al, 2010).As a result of the CFA analyses undertaken to ensure the survey's construct validity, it was found that the fit indices (x²=316, x²/df=2.8,CFI=0.961,TLI=0.953,SRMR=0.0346) were fully compatible (Kline, 2015;Meydan & Şeşen, 2015;Tabachnick & Fidell, 2013) and (RMSEA=0.0621)acceptable (Brown, 2015;Meydan & Şeşen, 2015).Values close to the fit indices of the original survey TP-STEMK were obtained [x² = 211.45,x²/df = 1.99,CFI = 0.97, TLI = 0.96, RMSEA = 0.067, SRMR = 0.047, (Chai, Jong et al., 2019)].These values indicate that the survey structure was suitable for the Turkish participants.At the same time, a positive and significant relationship was found between the subfactors of the TP-STEMK survey (see Table 3).
To determine the reliability of the survey, analyses of internal consistency (Cronbach's alpha), item-total correlation, and test-retest reliability were performed.Researchers have pointed out that if the reliability of a survey is low, its scientific value is also low (Ercan & Kan, 2004).The Cronbach's alpha coefficients (α>0.70)calculated for internal reliability show that the reliability value of the survey TP-STEMK is high [see Table 4 (Büyüköztürk, 2020;Nunnally & Bernstein, 1994)].It was found that the item-total correlation coefficients of the survey were positive and statistically significant, and the internal consistency between item values was high.These results indicate that the items measure similar features.The testretest method used to determine the stability of the survey showed a high, positive, and significant relationship between the scores [see Table 7, (Büyüköztürk, 2020)].In light of these results, the researchers conclude that the TP-STEMK self-efficacy survey is a measurement tool used by scholars with high validity and reliability.
Based on the analysis results, it can be concluded that the TP-STEMK survey is useful

Discussion and conclusion on the Application of the Survey
While the TPSK, TPEK, and iSTEM self-efficacy factors differed according to preservice teachers' program variables, no differentiation was found for the TPMK dimension.
It was found that pre-service elementary mathematics teachers' TPSK, TPMK, TPEK, and iSTEM scores were lower than those of other programs.It can be speculated that pre-service elementary mathematics teachers did not receive STEM or interdisciplinary training during their studies at the college may be a reason for this result.Another finding supporting this result is that less than one-fifth of the pre-service elementary mathematics teachers involved in this study reported that they had taken courses or training related to STEM or interdisciplinary teaching (see Table 1).The TPEK and iSTEM factors found that the knowledge levels of pre-service science and pre-service computer teachers were similar.
This result may be due to courses taken or STEM activities in college classes (see Table 1).
Some studies in the literature indicate that pre-service teachers' knowledge and awareness of science and computers in STEM are at a higher level (Yenilmez & Balbağ, 2016;Karışan & Bakırcı, 2018).This result is consistent with the findings of the related research.
When looking at the branches of in-service teachers, there were differences in selfefficacy of TPSK, TPMK, TPEK, and iSTEM.The research results showed that computer teachers performed better than other teachers in all knowledge factors, while elementary mathematics teachers had the lowest self-efficacy level in all knowledge factors.It was also found that science teachers' self-efficacy was better than mathematics teachers in all subfactors.It is suggested that this result may have implications for theoretical and practical training for teaching STEM.Because if we look at the characteristics of the participants, we can find that the percentage of STEM training for computer and science teachers is higher than that of elementary mathematics teachers, who conducted more STEM activities in their courses.
For this reason, they have a better STEM knowledge level than elementary school mathematics teachers (see Table 1.)In the study conducted by Chai, Jong et al. (2019), it was found that the knowledge self-efficacy of science teachers was higher than that of perceptions and orientations toward STEM education.In this regard, the research findings support the findings of previous studies.On the other hand, contrary to the results of this study, some studies in the literature report that the branch variable is ineffective (Demirkol et al., 2022).
When all pre-service and pre-service teachers were compared on the TPSK, TPMK, TPEK, and İSTEM subfactors by branch and program variable, it was found that they had the lowest mean scores on the TPEK and iSTEM factors.Similar to the result in this research, Chai, Jong et al.'s (2019) study also revealed that the engineering knowledge of integrated STEM education teachers was not at the desired level.From this perspective, this result is consistent with the findings of Chai, Jong et al. (2019).Chai, Jong et al. (2019) proposed collaboration between professors of engineering faculties and teacher educators to improve the engineering knowledge of teachers in undergraduate education.In short, it was emphasized that interdisciplinary collaboration is important.The correlation between TPEK and iSTEM in the subfactors is the highest compared to the others.It can be said that training that increases TPEK also will increase iSTEM knowledge.

RECOMMENDATIONS
This study also has some limitations.The first limitation is that the TP-STEMK survey  Öğrencileri disiplinler arası öğrenmeye teşvik eden iyi STEM problemleri tasarlayabilirim.

Figure 1 .
Figure 1.The stages of the survey adaptation process to the Turkish language Ethical review board name: Scientific research and publication ethics committee of science and engineering Instıtute of Alanya Alaaddin Keykubat University Date of ethics review decision: 09/03/2021 Ethics assessment document issue number: 9679 . The path diagram showing the factor structure obtained as a result of CFA is shown in Figure 2. The translated survey has four factors, like the original version developed by Chai, Jong et al. (2019).

Figure 2 .
Figure 2. Analysis of item-structure relationships of the STEM-TPACK self-efficacy survey Looking at Figure 2, the factor loadings of the items on the science dimension of the survey vary from .64 to .79, in the math dimension from .68 to .80, in the engineering dimension from .75 to .88, and in the integrative dimension STEM from .84 to .89.The correlation between the factors is as follows: Science and mathematics dimension .72; because it integrates the seven-factors structure of TPACK advocated byMishra and Koehler (2006) into STEM education.In this study, the TPCK framework was not limited to one subject.It was examined in an interdisciplinary context.As Chai, Rahmawati et al. (2020) stated, this context shows that the survey predicts teacher competencies in integrative STEM instruction.All the research results show that the survey used in this research will contribute to the literature.
mathematics teachers in TPEK, TPSK, and iSTEM factors.The results of this study are similar to those ofChai, Jong et al. (2019).In addition,Chai, Jong et al. (2019) reported that science teachers in China mainly conduct STEM activities.They also pointed out that science teachers' implementation was due to the professional development courses they attended.However,Chai, Jong et al. (2019) concluded that computer teachers' knowledge selfefficacy was higher than that of mathematics teachers on all factors, including the TPMK subdimension.They explained that this result was since the teachers had received computer training and had degrees in technology.In this regard, the study supports our research findings.Chai, Jong et al. (2019) used the original survey to show that mathematics teachers have a lower level of TP-STEM knowledge than science and computer teachers, similar to the result of this research.Özbilen (2018)  found that science teachers had higher STEM educational awareness than mathematics and technology design teachers in elementary school because they do STEM activities in the classroom.Hiğde et al. (2020) found that computer teachers had higher STEM educational awareness than elementary science, mathematics, and classroom teachers.He explained that such a result could be since computer teachers may have learned more about STEM during professional development courses and college.Wang et al. (2011) concluded that the branch variable impacts teachers' used in the study is of Likert type.Based on the results of this research, it can be stated that the data collection instrument, including a test consisting of open or multiple-choice questions, can be developed to assess teachers' knowledge in training TP-STEM.The second limitation is that this study was conducted with a group of participants consisting of mathematics, science, computer science, and instructional technology teachers and preservice teachers.Future studies may be conducted with pre-service teachers and teachers from other disciplines.After having these findings, the researchers believe that providing teachers with professional development courses focusing on STEM teaching by incorporating engineering and technology will improve their self-efficacy related to TP-STEMK.The study found that pre-service teachers' engineering knowledge was lower than their knowledge in the other subfactors.It is believed that courses developed by scientists studying engineering education will help them improve their knowledge level of engineering.Considering the results of this study, the inclusion of STEM in the curriculum will positively impact the TP-STEMK of pre-service teachers.Finally, this study is a crosssectional survey with quantitative data collection.Qualitative or mixed methods research

Table 1
The characteristics of the participants

Table 2
CFA's Compliance values for the TP-STEMK self-efficacy survey

Table 3
Pearson Moments Product Correlation Coefficients for Subfactors of the TP-STEMK Survey

Table 4
Cronbach Alpha (α) internal consistency coefficients for the survey and its factors

Number of Items Cronbach alfa (α) Coefficients for Reliability Cronbach's alpha (α) Coefficients of the Original Survey STEM -TPACK self-efficacy survey
Cronbach alpha (α)-coefficients of the factors of the adapted and the original survey are close.Based on these results, it can be said that the reliability value of the survey is high.

Table 6
Descriptive statistical results of the subfactors of the TP-STEMK survey by pre-service teacher program

Table 7
ANOVA results of TPSK, TPMK, TPEK, and iSTEM scores depend on the pre-service teacher program

Table 8
Post-hoc Scheffe test results of TPSK, TPEK, and iSTEM scores depending on the pre-service teacher program

Table 9
Descriptive statistical results for survey subfactors by teacher branch