Audit Trail Review in Viedoc

  • Published by Viedoc System 2021-06-11
  • Print

Objective

The regulatory authorities have outlined the role of regular Audit Trail Reviews (ATRs) in ensuring data integrity during clinical research. ATRs can help identify systematic or significant issues that arise during the data collection phase of a clinical or medical device trial, so that these issues can be addressed and mitigated.

Sponsors are ultimately responsible for the data integrity and reliability of the trial results and should therefore determine which ATRs to implement in their trials.

This document describes the reports and features available in Viedoc to aid sponsors and their representatives when performing ATRs of data collected in Viedoc.

References

The eClinical Forum (eCF) and the Society for Clinical Data Management (SCDM) have published a joint position paper describing ATRs and outlining a collection of standard use cases that can be used by sponsors when planning the ATRs for their studies. These use cases are utilised as the basis of this document, exemplifying how standard Viedoc functionality helps the sponsor fulfil their obligations.

Viedoc features and functionality

The sponsor should perform a risk-based assessment to identify critical data for their study, but EDC and eCOA data as collected in Viedoc are usually central to the trial being performed. Collection of that data needs to ensure that data integrity is maintained.

From the very beginning Viedoc has been designed with the goal of making standard clinical trial operations easy for site staff and for the sponsor/CRO staff working in the study. The users in the study are in control of the study, there are no “backend” changes made to the study or the collected data that require the intervention of Viedoc staff.

Some of the more important features from an audit trail point of view are:

  • Of course, a full audit trail of all data entry and changes in compliance with 21 CFR Part 11 and other regulations and guidances.
  • User management is performed by the Study Manager (for study personnel) and the Site Manager (for site personnel), ensuring that users are added to the study and maintained by the people who know the most about the users in that study. In addition, each user must accept the role that the Study or Site Manager has allocated to them, and if they feel it is the wrong role, they can flag it immediately to the Study or Site Manager. There are no anonymous helpdesks or backend system administrators managing the study users.
  • All access to the data is controlled by user roles, with the privileges for each user role being customisable for every study. This enables the sponsor to define exactly who can do what in their study. There are no universal administrator roles that give Viedoc staff backend access to the studies or the data – only the roles created and defined by the sponsor can view or change the data in the study in accordance with the privileges defined for each role.
  • Audit trails can be viewed online and as exported files (in PDF and MS Excel formats).
  • There are many different audit trails covering the metadata of the study, examples of these are:
    • Data audit trail – in accordance with 21 CFR Part 11 (date and time, user, reason for change)
    • User logs showing all users in the study and the roles they have had, including the privileges assigned to each role. All changes made during the study to both users and roles are logged and listed.
    • User login history.

The use cases

The use cases identified and defined in the eCF/SCDM position paper were derived from the MHRA “GXP Data Integrity Guidance” document. See the eCF/SCDM position paper for more information.

Use case 1 – Access concerns

Unauthorised user access

In Viedoc users are added to studies and to sites by the Study Manager and the Site Manager respectively. These are the people who know the most about who should be involved in the study, and what their roles are. The users themselves also check that they have been allocated the correct role in the study when they accept the invitation to join. There are no middlemen involved – no help desk that adds the users, no system administrator at Viedoc doing the work. This eliminates many of the error sources that are common in studies:

  • There is no copying of lists of users from one media to another which can give rise to transcription errors
  • The person adding the user to the study/site knows who their users are and which role they should have in the study – if there is an error in the list they are working from it will normally be picked up at this stage, before adding the user to the study in Viedoc.
  • The Study Manager/Site Manager know when people leave the study/site and can deactivate the user without needing a separate notification first.
  • The Study Manager/Site Manager know if a user changes their role in the study and can make the appropriate changes.
  • Finally if a mistake should be made in spite of having the people with the best knowledge managing their respective users, this can be detected by reviewing the User Logs available in Viedoc at any time – logs available in PDF and MS Excel formats.

Access without training

This is entirely within the control of the Study Manager and the Site Manager – they ensure that their users have been trained before they grant them access to Viedoc. The Study Manager can mark certain documents and eLearning sessions as mandatory for a given user role, and users with that role then need to mark that they have read and understood those before they are able to commence using the study in Viedoc.

User login and activity

One concern raised in the ATR position paper is that site users are not accessing the EDC systems in a regular fashion to enter or review data. The user logs can be reviewed directly in Viedoc, or they can be exported as reports for detailed analysis and review. It is possible to monitor the login history of all users. The logs also include information on the latest data entry/change made by users that have are able to perform data entry or make data changes.

System login and activity

There are no Viedoc system administrator roles that can access your studies or your data. Only sponsor roles can do that, and if you want Viedoc staff to support you in your study you must first invite them into the role you select for them in your study. And when a Viedoc staff member is granted a role in your study all their accesses are logged in the same way and the same place as for all other users in the study (see the previous section).

Another role that is sometimes missed is system roles, used for example when transferring data into Viedoc from external systems. These are also defined as standard study roles and can be seen in the user logs with their login history.

Use case 2 – Changes

Incomplete data

Incomplete data is flagged in the study dashboard for action by the relevant user (i.e., investigator for the site, or another role depending on the study settings).

Deleted data

In Viedoc only study users with the privilege to do so (assigned by the Study Manager or Site Manager) can delete data. When they do so the data is marked as “deleted” but is not actually physically removed. All study data can be exported for review, included deleted items. The full audit trail includes records of all data deletions.

Data changes

All data changes are recorded in the data audit trail. The audit trail can be exported in MS Excel format to aid ATR to identify frequent changes to critical data, such as multiple changes to data that affect inclusion/exclusion criteria.

Data changes after study lock

Viedoc does not allow data changes after the study has been locked. As there are no Viedoc administrator roles that have access to either view or change the study data, only the regular study roles setup by the Study Manager and the Site Manager can change data. And none of these roles can change data after the study has been locked.

To change data after the study has been locked the study needs to be unlocked first, and then the data can be changed by users with the required privileges. The study unlocking and the data changes are all captured in the audit trail.

Excessive changes of critical data

This use case covers several similar examples, including:

  • Multiple changes of critical data on a given form
  • High number of data change at one site compared to others
  • Multiple changes in patient reported data

All changes of this type must be made by site staff, and they are recorded in the audit trail allowing for easy detection and monitoring.

Data changes after long time

Changes to data a long time after the initial entry should be explained. These can be easily identified in the Viedoc audit trail.

Use case 3 – Data collection concerns

Sponsors can define windows in Viedoc during which EDC and eCOA data should be entered, as defined in the study protocol. The sponsor can further decide whether to allow data entry outside of these windows, in which case it can be seen in the audit trail. This applies to the more detailed use cases listed below.

Data not collected per protocol timing, or not contemporaneously

The audit trail lists the data and time for events, and this can be compared to the visits as defined in the study protocol.

Data collected at odd times, and the time taken for data entry varies

The audit trail report lists the date and time for all events in local time as well as UTC.

Missing data

Metrics concerning missing values, visits and investigator signatures are all presented to Viedoc users on their dashboard when they first log on to the study in question.

Use of edit checks before submitting data

In Viedoc the sponsor defines whether they want to use edit checks that are fired as the data is entered before submission to the database, or whether to use derived variables or check questions to fire edit checks after the data has been submitted. The audit trail captures all changes after the data has been submitted, the practice of site staff trying to influence patient inclusion by changing inclusion criteria data at time of entry can be avoided by not using edit checks that fire before the data is submitted.

Patient altering data before submitting it

When using ViedocMe patients are allowed to alter their answers in the present form before submitting it, but once it has been submitted the data is locked. The sponsor uses this when deciding whether they want to allow the patient to make data changes before submission or not. If data that should not be changed (for example a pain score) is collected in a form of its own and submitted then it can no longer be altered, and data collection from the patient can continue in the following form(s).

Use case 4 – Reporting concerns

The use cases defined in the position paper do not apply to Viedoc, as Viedoc uses validated computer functionality to perform all data migrations. Data can be exported in multiple formats, including MS Excel, CSV, CDISC ODM and SAS.

Use case 5 – Device concerns

Device malfunction

The device concerns listed in the position paper do not apply to ViedocMe, as ViedocMe is a Bring Your Own Device (BYOD) solution and does not rely on sourced devices that can malfunction. ViedocMe is a web-based app-like implementation that is not device dependent, there is zero footprint on the device used by the patient and the patient can use multiple devices to enter their data without any installation or migration from one device to the other. This allows for seamless reporting as patients upgrade their devices during the study.

Database synchronisation issues

ViedocMe data is not stored in a separate eCOA database and does not require synchronisation with the EDC database. All ViedocMe data is directly stored in the EDC database from the very start.