BCBA Integrity Practices: Survey Data

Earlier this year, we embarked on an exciting collaboration with our colleagues at LEARN Behavioral to understand what BCBA supervisors are doing in practice. The mission of this research is clear: We aimed to make procedural integrity not just a standard, but a seamless and natural aspect of every BCBA's practice. To do this, though, we need to know how actively BCBAs are engaging in activities centered around procedural integrity AND if there is a shortfall in engagement, what's holding them back? What challenges are they encountering in their daily practice that might deter them from ensuring procedural integrity? We administered a comprehensive survey to gather this information.

Steps involved in the collection of procedural integrity data

Before we can get into the data, I need to conceptualize the steps involved with the collection of procedural integrity data.

• First an observation must occur, which can be done in the presence of a learner, through role play situations, or a review of permanent product data.
• During the observation, data on how well the protocol of interest should be collected.
• Following the collection of data, it must be tracked so that performance can be analyzed over time,
• and lastly, feedback should be provided to the individual observed.

Check out our other blog posts to learn more about each of the above-outlined steps. 

Survey results

The survey yielded some eye-opening insights.

• An impressive 93% of BCBAs reported actively conducting staff observations, primarily on a weekly or monthly basis. 89% of BCBAs reported consistently providing feedback.
• The results indicate a more varied picture when it comes to procedural integrity data collection, tracking, and analysis though. While many BCBAs report being diligent about observing providers and providing feedback, there are gaps in data management:
  • 42% of BCBAs reported not collecting procedural integrity data at all. Among the 58% who did report collecting data indicated that they only do so monthly, biannually, or annually (which is not frequently enough).
  • Of those that collect data, 57% reported not tracking procedural integrity data.
  • And 44% reported not analyzing the collected procedural integrity data.

These findings are especially concerning considering the BACB mandates procedural integrity data collection. Beyond the obvious ethical implications, not having regular and comprehensive procedural integrity data hinders our ability to assess the true impact of our services and the areas of improvement.

In essence, while our industry shows a strong commitment to observations and feedback, we have a considerable opportunity to enhance our procedural integrity data practices, ensuring we're best positioned to drive positive client outcomes.

If objective data are not being collected, tracked, and analyzed over time then supervisors do not have all the information they need to best support their providers in providing high-quality services. That is, without this information supervisors cannot be sure that their programs are delivered accurately, and that provider performance is not impacting clinical progress. They simply do not have all of the information needed to ensure their team is delivering top-tier services. From an organizational perspective, they do not know the quality of services their organization is providing and do not have the insights into where they need to allocate organizational resources.

So how can we fix this? Potentially we can address the barriers that were reported most frequently. Understanding the barriers is key to unlocking the solution. Our survey pinpointed a recurring theme among BCBAs, which revolves around the challenges of time management and prioritization.

• Many BCBAs reported that there are not enough hours in the day to focus on procedural integrity data. The absence of dedicated time slots in their schedules for these activities adds to this sentiment. With a plethora of responsibilities, often, other tasks rise to the forefront, inadvertently sidelining the collection and analysis of procedural integrity data.

So, what's the takeaway? For organizations to truly champion procedural integrity, these activities need to be woven seamlessly into supervisors' everyday processes. Dedicated time slots must be carved out, ensuring these crucial activities aren’t side-lined. Engaging in this process is not a big time commitment for supervisors; rather, it can and should be embedded into their everyday supervision process.