Education Policy Analysis Archives

Introducing technologies into national large-scale testing: Are we ready?

Gabriel Cipriano, Susana da Cruz Martins — Tue, 02 Apr 2024 00:00:00 -0700

In recent decades, the potential benefits of introducing technologies into large-scale tests have been much discussed. Yet the path to effective technology use on large-scale testing has fallen short of expectations, especially when these tests have medium or high stakes for students. After a temporary cancelation of external assessment of learning due to the COVID-19 pandemic, the Portuguese government is taking a step forward. With a gradual implementation until 2025, the reintroduction of external assessment of learning in the Portuguese education system foresees all national assessment tests and all national examinations in digital format with the implementation of the External Assessment Dematerialization Project (DAVE). However, assessment reforms, such as DAVE, raise concerns within school communities. To identify and analyze these concerns, we conducted 32 semi-structured interviews with head teachers from mainland Portugal. To understand to what extent these concerns were considered and covered in DAVE’s design and implementation, a supplementary interview was conducted with the president of the Portuguese institute responsible for DAVE. In addition, a survey with 2 673 teachers was carried out to find out their degree of agreement with the implementation of DAVE. Results show that DAVE raises many concerns among head teachers, and it is not fully accepted by the teachers.

Perspectives and discourses on diversity and inclusion in the university community

María Naranjo-Crespo — Mon, 27 May 2024 00:00:00 -0700

The lines of research on diversity and inclusion have increased markedly in recent years. However, few studies integrate the perspective of different agents, and even more scarce are those developed within the epistemological and methodological framework of critical pedagogy. In this sense, the aim of the research has been to analyse the perspectives and discourses on diversity and inclusion in the Faculty of Education of the Complutense University of Madrid from the voices of the university community within three discursive lines or paradigms: integrative, inclusive-canonical, and inclusive-transformative. Forty-seven people from different groups in the university community took part in the study, which used interviews as the data production technique. The results, obtained by coordinating an open and axial coding system, are presented according to three codes: concept of diversity, attitudes towards inclusion, and institutional concept of diversity and inclusion. The conclusions allow us to identify the permanence of a hegemonic model of a banking nature, despite evidence of a positive evolution in the perspectives and discourses on diversity and inclusion.

Beyond the school building: Examining the association between of out-of-school factors and multidimensional school grades

Nandrea Burrell, Erica Harbatkin — Tue, 02 Jul 2024 00:00:00 -0700

Many states report school performance grades as a way to inform the public about school quality. However, past research has shown that when these grades drew largely on proficiency-based measures, they served to capture variation in school and community demographics rather than school quality. We extend this literature by examining whether a multidimensional measure of school quality such as those required under the Every Student Succeeds Act is less confounded by out-of-school factors than the proficiency measures that characterized previous generations of accountability. Drawing on school accountability grades from Florida combined with school and community demographic data, we find that more than half the variation in multidimensional measures of school quality can be explained by observable school- and county-level factors outside the school’s locus of control. Together, our findings show that even school grades that draw on multiple measures misattribute the contribution of demographics and socioeconomics to school quality—but subcomponents based on learning gains perform better than those based on proficiency. We conclude with policy implications and recommend that states focus public reporting on school quality measures that driven less by out-of-school factors and more by the school’s true contribution to student outcomes.

Digital education platforms and schooling: New challenges and alternatives towards educational equity and children’s rights

Pablo Rivera-Vargas, Lluis Parcerisa, Carla Fardella — Tue, 12 Dec 2023 00:00:00 -0800

This article introduces the special issue, which explores and analyses the impact of the development, distribution, and use of digital platforms in educational contexts. Organised in three sections for a comprehensive understanding, the first part outlines the concerns that motivated the guest editors, presenting the conceptual and social framework, historical and current, and focusing on the debate about equity and digital inclusion in education, in a context dominated by large technological corporations. The second section details the eight articles, distributed along two axes: theoretical review and empirical. Finally, the third part synthesises considerations derived from the materials presented, with contributions from diverse geographies and traditions. This collection highlights the tension between digital innovation and social inclusion in education, underlining the need for initiatives that favour the use of digital platforms focused on data governance by educational communities, the protection of users’ privacy and diversity of access to information, and alternatives towards education equity and children’s rights.

Equality and equity in the design of education policies: A case study of the Brazilian National Fund for Education Development (FNDE)

Yasmim Marques de Melo, Sandra Gomes — Tue, 21 May 2024 00:00:00 -0700

This article presents the definitions and concepts associated to meritocracy, equality and equity, and a methodological proposal that allows an empirical application to analyze if and how public policies seek to correct social inequalities. A classification of eight education programs carried out by the Fundo Nacional de Desenvolvimento da Educação was chosen to be analyzed, observing changes that took place across 25 years (1995-2020). The results show that these programs, initially, adopted the principle of equality in their institutional design and, later, an equity principle. However, the introduction of equity principles did not eliminate the equality principles. On the contrary, they were additional criteria adopted incrementally throughout different governments. Principles of meritocracy to allocate this federal fund to subnational governments were not identified.

Statewide standardized exams and the shaping of education policies for assessment: A comparative analysis of Paebes (Brazil) and Exeims-BC (Mexico)

Kézia Alves Moreira Dutra, Heitor Lopes Negreiros, Wagner dos Santos — Tue, 07 May 2024 00:00:00 -0700

This article comparatively analyzes the evaluation policies that establish the creation, organization, implementation, and continuity of the Paebes/Estado Espírito Santo standardized state exams (Brazil) and Exeims-BC/Baja California (Mexico). Specifically, we seek to understand, based on these contexts and the federal organization of the countries, how these assessments and their results influence the creation of new evaluative policies and materialize the impacts of such policies for the individuals involved in this process. It is characterized as a qualitative exploratory research, assuming the comparative method as a theoretical-methodological approach. The sources explored include official government documents, exam papers, and information obtained from competent authorities' websites. The results indicate that the exams are similar regarding the state, national, and international contexts; however, the rise of the private sector in Espírito Santo is more evident through Paebes. It is observed that there is a trend towards the centralization of high-stakes policies with the intensification of accountability, both in Espírito Santo and Baja California. It is evident that this type of assessment influences not only the context of the creation of educational policies but also budget allocation and the creation of new contexts of disputes and the pursuit of school performance.

Research, interrupted: Addressing practical and methodological challenges under turbulent conditions

Tue, 09 Apr 2024 00:00:00 -0700

The COVID-19 pandemic caused tremendous upheaval in schooling. In addition to devasting effects on students, these disruptions had consequences for researchers conducting studies on education programs and policies. Given the likelihood of future large-scale disruptions, it is important for researchers to plan resilient studies and think critically about adaptations when such turbulence arises. In this article, we utilize qualitative analysis of interviews with research study leaders to illuminate practical and methodological challenges, as well as promising practices that arose during the pandemic period. We find that researchers made pivots to address practical challenges and protect the feasibility of their studies. We also find that researchers took precautions, where possible, to understand and bolster internal validity. However, these pivots frequently surfaced additional threats to construct and external validity. We conclude with recommendations for future studies conducted in times of prolonged unplanned school closures or other large-scale disruptions.

The many uses of the Sinae evaluation: The perspectives of Federal University of Bahia’s program coordinators

Vanúbia de Jesus Silva, Roberto Brazileiro Paixão — Tue, 02 Jul 2024 00:00:00 -0700

This research aimed to analyze how the coordinators of undergraduate programs of the Federal University of Bahia use Sinae’s evaluation. Instrumental, conceptual, and symbolic uses, derived from the process or the results, were investigated. We identified 22 categories of instrumental uses (such as “supported analysis and discussion of results”), 13 conceptual uses (such as “changed postures/perceptions”), and 8 symbolic uses (such as “mobilized the community for the assessment”). We found more conceptual and less instrumental uses. Connections between categories might indicate that some uses may stimulate other types.

Socioeconomic adversity and higher education: Is the Quota Law a potential bridge to social mobility in Brazil?

Mon, 27 May 2024 00:00:00 -0700

This article aims to contribute to the debate on the role of the Quota Law regarding the potential social mobility of students with high socioeconomic adversity, using administrative data from a federal university in Brazil. We used Confirmatory Factor Analysis technique to construct an adversity index, composed of variables that may negatively affect access to higher education, such as students' socioeconomic background, parental education, and ethnicity. We classified courses by expected earnings and assessed how background is associated with course choice. Then, we used regression models estimated by the Generalized Method of Moments (GMM) to identify the relative importance of the interaction between quota categories with the adversity index and the score on the admission exam (National High School Examination – Exame Nacional do Ensino Médio - ENEM) to predict course choice and consequent possibility of social mobility. The results showed that quotas can reduce the negative effects of socioeconomic adversity on social mobility. However, they are not completely sufficient to “break” the effects of the low quality of secondary education in relation to ENEM performance and, consequently, social mobility.

Variation in co-authorship networks for publications on student aid, based on the terms used in the research

Lincon Rodrigues Dias Simões, Wagner Bandeira Andriola, Diego de Queiroz Machado — Tue, 27 Feb 2024 00:00:00 -0800

This study investigated the variation in the structure of co-authorship networks for publications on student aid, based on the use of different terms, as well as the variation in the content of these publications. The analysis used the VOSviewer (version 1.6.18) and Ucinet software, to map the relationships between authors, and the Voyant Tools software to analyze the texts of the abstracts. The results revealed that the policies to support students take a variety of forms, to which various terms and expressions correspond, and that the range of results of sociometric studies on student aid varies significantly according to the terms used in the search. Due to this variety, a joint search covering several terms results in many asymmetries among the networks. We conclude that building a panoramic view of co-authorship networks on student aid requires the adoption of several searches using different filters, mainly by country.