ATP 2024: PSI Sessions

About:
If you're new to credentialing or starting a new program, understanding the assessment industry can be tricky! This session is designed for those new to the industry or are working on a new professional credential and needing a quick overview of the considerations for a valid, reliable assessment program. Industry experts will provide an overview of the fundamentals of assessment. Participants will gain insights into key concepts such as: the purpose of an exam, exam-delivery platforms, security considerations, assessment construction, and item types. In addition, are you wondering why we are all talking about AI? We will share a general overview of what the implications of AI are in various parts of the assessment development and administration processes. By the end of this session, attendees will be equipped with the basics.

Learning Objectives:

1. Understand the fundamental purpose of assessments and their role in credentialing and program development, including the significance of exam construction and its alignment with program objectives.
2. Explore various exam-delivery platforms and security measures, gaining insights into the importance of safeguarding assessments and ensuring the integrity of the examination process.
3. Grasp the impact of Artificial Intelligence (AI) on assessment development and administration, recognizing its implications across different stages of the assessment process.

Presenters:

•    Emily Schenderlein / Behavior Analyst Certification Board

•    Alex Miranda / Behavior Analyst Certification Board

•    Melissa Nosik / Competency & Credentialing Institute

•    Isabelle Gonthier / PSI Services

About:
Item bank breaches are an unfortunate reality that every testing organization must consider, which is why having a robust, healthy item bank is so imperative. When breaches inevitably occur, web crawling, when coupled with data forensics, is an innovative tool for enchaining item bank health. This presentation seeks to help others better understand the advantages of conducting a test security analysis, recognize the value of routinely screening websites for exam content, and implement strategies for maintaining healthy item banks. We will discuss exam development strategies to help prepare for the inevitable and tools to detect item bank breaches before it is too late. Our goal is to share our insights and our recommendations as test developers for what a healthy item bank looks like and to help increase understanding and awareness of using analytics that enhance item bank health. We invite the audience to share any of their own experiences and insights with these types of analyses.

Presenters:
•    Steven Svoboda / PSI Services LLC
•    Nick Soto / Construction Management Association of America
•    Amy Farmer / American Association of Veterinary State Boards
•    Frank Williams / PSI Services LLC

About:
Both the rise in social media and the launch of remote proctoring have created an expectation to resolve test-taker issues in real time. How does an organization deal with issues, potential false claims, and the visibility to other test takers? How do we protect personally identifiable information in our efforts to resolve issues? Where do we balance our desire for real feedback with too much information? In this Peas in the Pod discussion, we will discuss strategies in use today. In this session, we will solicit conversation around effective and ineffective responses from participants.

Presenters:
•    Paul Bishop / Water Professionals International
•    Heather Richards / PSI Services

About:
The purpose of a credentialing exam is to differentiate individuals with the requisite knowledge for safe, independent, competent practice from those without it. To ensure the validity and legal defensibility of credentialing exams, involvement of subject matter experts (SMEs) is essential across the exam development life cycle. Given the vital nature of SME involvement, processes need to be established and continually evolved to ensure thoughtful recruitment, selection, training, and engagement of SMEs, incorporating considerations for diversity, equity, and inclusion (DEI). Considerations for DEI are not limited to a single timepoint during the exam development life cycle (e.g. SME recruitment, item review). For example, bias in content could originate from various sources, such as content irrelevant language, cultural nuances, or regional differences. Two key challenges for credentialing organizations are identifying and engaging key stakeholders across all exam development activities. In this moderated panel session, credentialing experts will engage in a thoughtful discussion of DEI considerations across the exam development life cycle. By the end of this session, attendees will be able to identify and engage in discussions regarding key considerations related to DEI across the exam development life cycle. Attendees will also be able to consider potential updates to current processes and policies for SME engagement across the exam development life cycle.

Presenters:
•    Pamela Ing Stemmer / PSI Services

•    Shebbra Toussaint / PSI Services

•    Ashlee Griffin / Health Care Compliance Association - Compliance Certification Board

•   Lavina Harless / Association of Social Work Boards

•   Carl Hokanson Association of Social Work Boards

About:
In an era of dynamic change, the assessment industry is undergoing a profound transformation. Hear from a few C-Level leaders in our industry that will discuss the industry's evolution, dissect its trajectory, and confront imminent challenges. These leaders grapple with diverse concerns, including the disruptive influence of emerging AI technologies, the imperative to expand their businesses, and the necessity of reimagining credentialing processes in a global context marked by political and economic fluctuations. Chief among their priorities is the protection against sophisticated security threats. Simultaneously, they prioritize strategic investments to address future needs while maintaining growth commitments. These leaders are committed to addressing the evolving landscape of testing, tackling challenges such as introduction to generative AI, redefining the value of standardized assessments in academic admissions, and striking a balance between user experience and security. The role of online platforms (OP) and test centers is central to their discussions, as they contemplate how to invest in new products and offerings while upholding their growth commitments. In an industry marked by rapid change, these leaders are poised to navigate the future with foresight and innovation. 

Presenters:
•    Brodie Wise / Internet Testing Systems

•    Pat Ward / Internet Testing Systems

•    Janet Garcia / PSI Services

•   Gary Gates / Pearson VUE

•   Tim McClinton / Meazure Learning

•   Stuart Udell / Prometric

About:
The United States and the world are experiencing rapid changes (globalization, automation, immigration, climate change), which are transforming the demographics of students, workers, and society and creating tensions as societies grapple with living in a multicultural environment and sharing power. The assessment industry, like other sectors, is adapting to these changes by embracing diversity, equity, and inclusion (DEI). Organizations are fostering communities of practice (e.g. DEI committees and employee resource groups), where team members come together to discuss and address challenges related to DEI. These communities of practice facilitate the sharing of insights and best practices to navigate complexity. In this session, panelists will discuss the communities of practice established in their organizations and share the lessons learned. They will discuss how these communities of practice have helped develop equitable perspectives, improved collaboration, and fostered a sense of belonging among team members.

Presenters:
•    Joanna Solomon / MHS

•    Erica Babino / NBME

•    Shebbra Toussaint / PSI Services

•   Alex Casillas / ACT

About:
Artificial intelligence, or AI for short, is a hot topic these days—so hot that it generates polarized reactions across the testing and assessment industry. Some reactions are excitement and dreams of enhanced productivity and efficiency in how we develop and deliver tests. Other reactions are of concerns and warnings as to the risks associated with AI tools, especially as they relate to security, data, intellectual property, and cheating concerns. But are there not shades of grey in considering the opportunities and threats AI bring to our industry? Building on the ATP Security Committee white paper on AI, at this panel discussion, we will outline and dissect concerns and opportunities related to the use of AI tools, including generative AI, both from a test development and delivery and security perspective. We will challenge our thinking, discuss pros and cons, and engage the audience in thought-provoking discussions on security considerations related to the use of AI. Our promise: This thought-provoking panel discussion will lead attendees to question their thinking, push their limits, ask the tough questions, and create opportunities for further exchange and conversations during and after the conference.

Presenters:
•    Isabelle Gonthier / PSI Services

•    Marc Weinstein / MJ Weinstein Law

•    Paul Muir / Surpass

•   David Yunger / Vaital