Presentations and Publications

Presented at the Competency Based Education Exchange Conference, Colorado Springs, CO - November 2024

Authors:  Castle, C., Moussavi, R., Stockton, J.

Abstract:  This session draws on our experience articulating and measuring process competencies for several graduate programs in education and professional studies. We will describe some of the challenges associated with incorporating process competencies in program design, outline potential approaches to defining and measuring process competencies, and invite participants to share examples from their contexts. 

The main premise of competency-based education (CBE) is that learner progress is determined by their attainment of outcomes. The word “outcome” (often used interchangeably with “competency”) often implies an endpoint in a learner’s development of that knowledge, skill, or ability. The corresponding assumption for CBE assessment is that a learner’s mastery of the outcome can be encapsulated in a final product or demonstration. Most measures of job performance, however, are not focused on a single product nor a single point in time. This is particularly true for the wide range of jobs that emphasize the application of a particular process or approach to solving persistent problems.  

While developing CBE programs for a range of professions, our team has identified a subset of competencies that we call “process competencies.” Process competencies describe an orientation to problem-solving that may be specific to a particular discipline or field. Examples of process competencies include design thinking, critical thinking, decision making, the scientific method, etc. While these process competencies are authentic to the workforce, defining and measuring these competencies is not as straight-forward as with competencies that focus on discrete skills. 

Processes can be defined in a few different ways: a) step-by-step approach: dissecting a process into a list of underlying knowledge, skills, or abilities that are applied in succession; b) a descriptive approach: describing the core characteristics or qualities of a successful application of the process; or c) a product-centered approach: defining a process in terms of what it yields rather than how it occurred. The approach used to define a process competency carries natural implications for how we facilitate learning and measure mastery of that competency. We will share examples of different assessment approaches we’ve taken to measuring process competencies, along with some of the challenges encountered and lessons learned.  

Presented at the Association of Education Communication and Technolgy Conference, Kansas City, MO. - October 2024

Authors: Kang, In Gu, Castle, Courtney, & Pittack, Charles.

Abstract:   Background: Effective professional development (PD) can enhance teaching practices and potentially improve student achievement. However, limited research links teacher PD to improved student outcomes, often due to challenges in transferring knowledge and skills into practice. Transfer of learning is a dynamic process influenced by interactions among teachers, students, and environments, yet it remains understudied. This study explores educators’ perceptions of an online PD program's impact on transfer, their transfer experiences, and factors facilitating or hindering it.

Methods: The Designing for Equity, Justice, and Innovation in Education (DEJIE) pilot program, offered online in spring 2023, focused on competencies like Teaching for Justice, Thinking Like a Designer, and Grounding in the Science of Learning and Development. This 14-week program included synchronous meetings, self-guided modules, and project-based learning. Seven educators participated, and five provided follow-up interviews six months later. Interviews were thematically analyzed using NVivo, with independent coding by two researchers to ensure validity.

Results and Findings: Participants highlighted Teaching for Justice and Thinking Like a Designer as particularly relevant to their practice, enhancing motivation and engagement. The program's learning community fostered collaboration and accountability, while the project cycle encouraged applying skills in real-world contexts. Participants reported using program materials and frameworks in their classrooms and shifting toward student-led, relationship-focused approaches.

Facilitators and Barriers: Supportive work environments, flexibility, and non-tested disciplines facilitated transfer, while time constraints and limited colleague support hindered it. Supervisor support was crucial in adapting learning to individual contexts.

Presented at the American Public Health Association Conference, Minneapolis, MN. - October 2024

Authors:  Kang, I.-G., Kim, N., DeBacker, D., & Mumba, M.

Abstract:   Background: Recent data suggests that early career employees in public health institutions represent the highest-risk subgroups for turnover intention. However, population-level studies examining factors associated with turnover intention in this high-risk groups are lacking. This study examined relationships between demographics, job satisfaction, workplace environments, and turnover intention among early career employees in the US governmental public health workforces.

Methods: Data were drawn from the 2017 and 2021 Public Health Workforce Interest and Needs Survey (PH WINS). The analysis comprised early career employees (weighted N= 41,025) with less than 5 years of experiences in the public health practices, situated non-supervisory level roles in the US governmental public health agencies. Hierarchical multivariable logistic regression models were employed to identify correlates of turnover intention while accounting for complex survey design to yield nationally representative estimates.

Results: Approximately 36.0% of early career employees in public health agencies reported their intention to leave their organization within the next year. Male, more educated, local government working early career employees are more likely to leave their organizations. Higher levels of job satisfaction (including job, organization, pay, and job security) and favorable workplace environments (e.g., adequate assessment of training needs, supervisory support for leadership skill development, and complete involvement in my work) were associated with lower log odds of turnover intention (all ps<.05). Conversely, interpersonal factors such as working relationship with supervisor and respectful treatment showed no significant associations with turnover intention (all ps>.05).

Conclusion: A significant portion of early career employees expressed turnover intention, with certain demographic factors such as male sex, higher educational attainment, and employment in local agencies elevating the risk. Notably, job satisfaction, workplace environment, and interpersonal relationships exert varying degrees of influence on turnover intention among early career employees, underscoring the need of a multifaceted approach in public health workforce management practices.

Presented at the Assocation of Education Communication and Technology, Kansas City, MO - October 2024

Authors:  Stockton, J., Dudek, J., Moussavi, R.

Abstract:  This presentation outlines a program design process for developing competency-based education (CBE) degree programs. Taking a direct-assessment approach to CBE, instead of a course-based approach, has significant implications for the design methods utilized. Implications for the roles served by faculty, instructional designers, and psychometricians throughout this process are also discussed, suggesting tools for their sustained collaboration.  

Presented at the Northeastern Educational Research Association Conference, Trumbull, CT. - October 2024

Authors:  Castle, C.

Abstract:   Traditional assessment design approaches focus heavily on alignment between the construct definition, task specifications, scoring methods, and the intended usage of the results. Accordingly, typical assessment design activities primarily involve academic and psychometric experts who review the construct in question, develop tasks and scoring methods, and consider the validity of the inferences made. This process often does not involve end users until an assessment has already been operationalized. In contrast, design thinking is an approach that focuses heavily on the user experience as a source of input when generating creative solutions to problems. In recent years, design thinking has been applied to educational settings (Razzouk & Schute, 2012; Norton & Hathaway, 2015; Henriksen & Richardson, 2017). This paper will discuss the application of design thinking methodology to assessment development using a tool called the Innovator’s Compass (Ben-Ur, 2016). Using this tool, users are consulted and/or participants in defining a design problem, making related observations, defining design principles, generating ideas, and performing experiments. This process leads to assessment tasks that are more innovative, motivating, and enjoyable for test takers. The approach has potential to improve all forms of assessment but is particularly valuable in developing performance assessments and authentic assessment tasks. The paper will include concrete examples of how design thinking methodology was used in developing assessments in different settings, including a teacher-preparation program, high school STEM classrooms, and identifying cybersecurity talent.  

Presented at the International Society of the Learning Sciences Annual Meeting, Buffalo, NY - June 2024

Authors: Moussavi, R.; Castle, C.; and Dudek, J.

Abstract: This design-based case study follows the design and implementation of Science of Learning and Development (SoLD) competencies across two successive Competency-Based Education (CBE) programs, the first a STEM focused licensure program and the second a program focusing on design, equity, and justice for in-service teachers.  The goal of positioning the Science of Learning and Development as a core competency and integrating it within the two programs was to support educators in becoming well-equipped to support their students’ learning and development through the whole child approach. We offer insights on the conceptualization of an overarching framework and competencies, development of assessments and curricular designs, and strategic supports/learner interactions as a possible model for SoLD–focused teacher preparation/education.  Furthermore, this approach reinforces the importance of SoLD and communicates that SoLD is not distinct from pedagogy which marks a shift from how SoLD is often presented to pre and in service teachers. 

Presented at the International Society of the Learning Sciences Annual Meeting, Buffalo, NY - June 2024

Authors: Castle, C.; Moussavi, R.; Pittack, C.; Chow, C.; Kang, I-G; and Karamcheti, M.

Abstract: This paper presents a pilot program for educators that focuses on designing for equity, justice, and innovation. The program leverages Project-based Learning (PBL) as an approach for effective teacher professional development. This study analyzes the presence of the core features of effective educator professional development—collaboration, inquiry and active learning, and supportive instructional practices—within the pilot program as well as what program attributes support participants’ experience of these features. The findings have implications for designing and developing teacher professional development programs.

Presented at the National Council on Measurement in Education Conference, Philadelphia, PA. - April 2024

Author: Thanos Patelis

Abstract: 

Presented at the National Council on Measurement in Education Conference, Philadelphia, PA. - April 2024

Author: Thanos Patelis

Abstract: 

Presented at the National Council on Measurement in Education Conference, Philadelpha, PA. - April 2024

Author: Thanos Patelis

Abstract: 

Presented at the AAI - C3Be Research Convening, Lawrence, KS. - April 2024

Authors:  In Gu kang., Raha Moussavi, Courtney Castle, & Charles Pittack 

Abstract:   This study develops a meta-framework for competency-based education (CBE) by synthesizing the current theoretical approaches to CBE across various domains including education, engineering, medicine, and others, employing an umbrella review methodology. The creation of such a meta-framework for CBE holds significance in that it lays the foundation for analyzing, designing, developing, implementing, and evaluating effective evidence-based practices in CBE.

Presented at the Competency Based Education Exchange Asia Pacific Conference, Singapore. - February 2024

Authors:  DeBacker, D.

Abstract: 

Presented at the American Public Health Association, Atlanta, GA. - November 2023

Authors:  Kang, I.-Gu., Kim, N., Loh, W.

Abstract:   Purpose: This study aims to utilize machine learning (ML) modeling to identify significant predictors and interactions that stratify subgroups of the US healthcare governmental workforce who are at high risk for job dissatisfaction and turnover intention.

Data: Data were collected from a nationally representative sample of the US governmental workforce (n=43,029, weighted n=75,706) from the 2018 Federal Employee Viewpoint Survey. The ML algorithm-based Generalized, Unbiased, Interaction Detection and Estimation (GUIDE) classification and regression tree modeling was utilized to predict job satisfaction and turnover intention with sampling weights.

Results: The ML analysis identified multiple important tree-splitting variables for job satisfaction or turnover intention. Organizational procedural justice, organizational satisfaction, intrinsic motivation, and talent management predicted job satisfaction while turnover intention was predicted by talent management, job satisfaction, advancement opportunities, and commitment to agency. We identified six at-risk subgroups with low-level job satisfaction and six at-risk subgroups with high-level turnover intention. High-risk subgroups of Job dissatisfaction and turnover intention shared 8 characteristics including organizational satisfaction, employee recognition, talent management, advancement opportunities, performance-oriented culture, and commitment to agency, and employ development.

Conclusion: The results of the ML modeling revealed several significant factors affecting both job satisfaction and turnover intention. Identification of these factors and their complex interactions may inform the design of targeted strategies to reduce the risks of turnover, especially for high-risk subgroups of the healthcare governmental workforce. Public health administrators can use the findings of this study to develop interventions that are tailored to at-risk subgroups while accounting for the shared characteristics.

Presented at Shenandoah University (Virtual). - November 2023

Authors:  DeBacker, D.

Abstract: 

Presented at the Association of Education Communication and Technolgy, Orlando, FL. - October 2023

Authors:  Dudek, Jaclyn

Abstract: 

Presented at the Northeastern Education Research Assocation Conference, Trumbull, CT. - October 2023

Authors:  Chow, C., & Moussavi, R. 

Abstract:   Sense of belonging is an important, yet under-researched element in teacher education programs (Bjorklund et al., 2020). This work adds to research establishing the importance of sense of belonging in teacher education programs. We identify key factors that affected preservice teachers' feelings of belonging and recommendations for teacher education programs

Presented at the Northeastern Education Research Assocation Conference, Trumbull, CT. - October 2023

Authors:  Castle, C. 

Abstract:   This paper provides a general overview of the range of assessments used in a competency-based teacher preparation program, including projects, performance tasks, and a portfolio. These assessments exemplify the types of assessments that align with recommendations for the use of authentic assessments in Competency-Based Education (CBE) programs (Bushway, Dodge, & Long, 2018). The paper also includes a description of the policies and procedures used to guide the assessment process, including rubrics, guidelines, roles, and timelines. The paper will describe a few of the major challenges faced in implementing the competency-based assessment system, including perceived obstacles by students, instructors, and evaluators, as well as challenges in maintaining assessment rigor. One of the challenges described is a lack of consensus understanding among instructors and evaluators, which resulted in sub-optimal interrater reliability. The paper will provide an in-depth description of the nature of this problem and one strategy used to address it: norming (Schoepp, Danaher, & Kranov, 2018; Holmes & Oakleaf, 2013), along with some concrete examples and data from the norming process. The discussion and examples in this paper will provide insight and ideas that may be useful to organizations that are utilizing a competency-based approach to learning and assessment.  

Presented at the Northeastern Education Research Assocation Conference, Trumbull, CT. - October 2023

Authors:  Patelis, Thanos 

Abstract: 

Presented at the Northeastern Education Research Assocation Conference, Trumbull, CT. - October 2023

Authors:  Patelis, Thanos 

Abstract: 

Presented at the Competency Based Education Exchange, Amelia Island, FL. - October 2023

Authors:  DeBacker, D.

Abstract: 

Presented at the Kansas Economic Policy Conference (Panel Presentation), Lawrence, KS. - October 2023

Authors:  DeBacker, D.

Abstract: 

Presented at the Northeastern Education Research Assocation Conference, Trumbull, CT. - October 2023

Authors:  Floyd, Dessaline 

Abstract:   The University of Kansas (KU) Center for Certification and Competency-Based Education (C3BE) recently completed its first pilot of the Designing for Equity, Justice and Innovation in Education (DEJIE) program. The 15-week pilot program included a small cohort of 8 educators and leaders across the United States, all of whom either work directly in K12 schools and districts or in close proximity to K12 systems of teaching and learning. Participants have the option to earn micro-credentials/equivalent graduate credits for completing the pilot program. However, to earn the equivalent credits, participants must agree to submit specific assessment artifacts that serve as evidence of their learning of the DEJIE program’s three (3) core competencies: Teaching for Justice (TfJ), Thinking Like a Designer (TLaD) and the Science of Learning and Development (SoLD). This presents a challenge. By asking participants to submit their assessment artifacts, they are at the same time being asked to showcase evidence of their learning and to put it ‘on display’. This can produce feelings of fear and angst in individuals as much of the content requires rigorous and ongoing self-reflection. Fear can be a barrier to learning and certainly to demonstrations of learning in-practice and alongside others who have their own unique learning trajectories. There is a need for more innovative assessment design approaches that include co-design in partnership with learners themselves. This can alleviate the undo stress and burden some experience that can deter them from having more accurate self-conceptions of their learning as it relates to diversity, equity, inclusion and belonging.

Presented at the University of Kansas - School of Education and Human Services, Lawrence, KS - October 2023

Authors:  Floyd, D., Feldman, K.

Abstract: 

Presented at the Northeastern Education Research Assocation Conference, Trumbull, CT. - October 2023

Authors:  Floyd, Dessaline 

Abstract:   The pursuit of Justice in education is a noble one–it both taxes and rewards the soul. The power to assess one’s individual pursuit must remain in the hands of those who have willingly taken on this cause and for this reason, dynamic assessment guidance is needed to support individuals and teams in assessing their own justice and equity aims. Here are some example question prompts to spark thinking and empower others to take ownership over the changes they want to see in their own lives as it relates to equity, justice and education:

Assessment of SELF

• What happened to me?
• How did it affect me? How does it influence my views of others?
• What is most important to me now?

Assessment of SITE/SITUATION:

• What makes up my ‘environment’?
• What are my environment’s histories? In other words, what events have happened here and what stories are being told about these very same events?

Assessment of SYSTEM:

• What beliefs and belief systems do I hope to change?
• What systems am I interacting with?

Presented at the National Science Foundation Site Visit, Lawrence, KS. - September 2023

Authors:  DeBacker, D.

Abstract: 

Presented at the European Science Education Research Association Conference, Turkey. - August 2023

Authors:  Chow, C., Ofori, C., Godwyll, J., & Jeffrey, P.

Abstract: 

Presented at the Athens Institute of Education and Research, Athens Greece. - May 2023

Authors:  Patelis, Thanos

Abstract: 

Presented at the University of Kansas, Lawrence, KS. - April 2023

Authors:  Chow, C., Ofori, C., Godwyll, J., & Jeffrey, P.

Abstract: 

Presented at the National Council for Measurement in Education Annual Conference, Chicago, IL. - April 2023

Authors:  Patelis, Thanos

Abstract: 

Presented at the Northeastern Education Research Association, Trumbull, CT. - October 22

Authors:  DeBacker, D.

Abstract:

Presented at the Australian Council for Educational Research Conference, Australia. - August 2022

Authors:  DeBacker, D.

Abstract:  This presentation focuses on the need to make learning visible for all constituents – learners, educators, employers, policymakers and future employees – with a focus on the work of the Center for Certification and Competency-Based Education (C3Be) at the University of Kansas (USA). As K–12 and higher education move to a post-pandemic world, it’s imperative we find different ways to identify learner outcomes and evaluate learning. By moving from nouns to verbs; by moving from course titles to competencies; by moving from transcripts to learner outcomes, learning becomes visible. International educator and founding director of C3Be, Dr Diane DeBacker shows how learning is made visible through a proven process of mapping learning outcomes, assessing both new and prior learning using innovative technology, issuing stackable micro-credentials, and working with policymakers and employers to meet workforce demands.

Presented at the 7th International Conference on Higher Education Advances (HEAd'21), Spain. - June 2021

Authors:  Debacker, D., Dudek, J., Kingston, N., Patelis, T.

Abstract:  This paper explores the rapidly changing world of higher education and the need for different ways to identify learner outcomes and evaluate student learning. In recent years, higher education has experienced significant demographic shifts in student populations. These shifts were the result of numerous variables including the increasing cost of higher education, the demand from business and industry to get people into high-demand occupations faster, and the decreasing number of individuals choosing post-secondary education immediately following high school. The year 2020 brought unprecedented challenges to the world with the pandemic caused by the coronavirus known as COVID-19. The pandemic accelerated the change that was already taking place in higher education. From how education was delivered to where it was delivered, higher education was forced to rapidly change a centuries-old model. This paper explores a tier one research university’s response to the changes in higher education by employing a proven process of mapping learning outcomes, assessing both new and prior learning using innovative technology, issuing microcredentials, and working with policymakers and employers to meet workforce demands.

Presented at the Association of Education Communication and Technology Conference, Chicago, IL. - November 2021

Authors:  DeBacker, D.

Abstract:

Presented at the Northeastern Educational Research Association (NERA) Annual Conference, Trumbull, CT.  October 2024

Authors:  Patelis, T. , Castle, C., Stockton, J., Moussavi, R., DeBacker, D.

Abstract:  Within the realm of principled approaches of test development1, we include test users as co-designers.
I will illustrate how this was done in our development of assessments of personal characteristics (attributes) of well-suited individuals for an entry-level occupation in cybersecurity.
The point of this is to show how to embed test users in the assessment development process rather than just getting test user feedback in ad hoc studies.

Test users:

Sponsor of the initiative who is anchored in the business community
Business-led company to accelerate economic growth in Kansas and western Missouri through various strategies and programs
C-level executives in cybersecurity
Managers in cybersecurity
Employees in cybersecurity
Potential candidates
 

Journal:Journal for Applied Instructional Design - October 2024

Author(s):  Dudek, J.,  Patelis, T., Stockton, J.