AHRM Blog

What is Data Management?

Data Management is a term encompassing various functions and applicable within several industries.   Within the field of research it is often referred to more specifically as Clinical Data Management.   Data Management is an integral part of doing research and can best be described as a process for collecting, validating and reporting the data produced during a clinical trial or other type of research study.   Highly effective Data Management is crucial for the generation of reproducible and reliable study results.     The degree of Data Management will vary from one research effort to another, but all research efforts will require some level of data management prior to the data being analyzed and published.  How the data is to be collected, validated and reported are precisely outlined in a document called the Data Management Plan.   This helps to ensure that the way in which data is reported and collected is consistent among all sites participating in the research effort, as well as the consistency in which the data is analyzed. 

What is the role of a data manager?

A data manager is an important member of the research team, whose main priority, is to ensure the integrity of the data that is generated for use during a clinical trial or other research effort.   They can be employed by pharmaceutical or medical device firms, as well as by contract research organizations.  Some large hospitals or clinics may also hire data managers if their involvement in research is great enough to support position(s).  Often data managers at hospitals and clinics have other responsibilities as well; including direct patient care.  For the remainder or this section, we will focus on the responsibilities of data managers employed by either pharmaceutical/medical device firms or by contract research organizations.  

The data manager is charged with a variety of tasks related to developing the processes and procedures for the collection; validation and reporting of the data generated for use during a clinical trial other research effort.   Among these responsibilities, are the development and processing of case report forms (CRF), the identification and generation of necessary logic checks and writing and resolving queries.  Because much of a data manager’s time is spent performing one of these three tasks, we will take a deeper look at each of these. 

Case Report Forms (CRFs) are forms used during a clinical trial or other research effort to collect and report the required subject level data.  They can be in either electronic (eCRF) or paper (CRF) format.   The number of CRFs used for a given research effort will depend on how much data is being collected, what types of data are being collected and how often data is being collected on each subject.  It will also depend on how each form is designed and how much information is included on a single form.  Sometimes a CRF may be designed where data related to Medical History and the results of a Physical exam are combined, while other CRFs separate this information onto two separate forms. 

Logic Checks, as indicated by their name, look for a logical pattern to the data that has been reported on the CRF/eCRF.   They are also referred to as edit checks and are generated to identify errors in the data.  The errors range from simple identification of missing data to more complex issues, like lack of consistency between a data point reported on Form A, for example, and a related data point reported on Form B.  An example of this is that on Form A, the subject’s gender is reported as Male, but on Form B, the results of the screening pregnancy test are reported as negative, instead of not applicable.   They also identify data that is out of range, which means that the value is abnormal and not likely to be true.  An example to this would be a subject who is reported as being 152 years old. 

Writing and resolving queries is often the most time intensive part of a data manager’s job; especially early on in the data collection process.  It is not abnormal for several queries to be written on each CRF submitted for the first several subjects.  Generally speaking, as the research effort progresses, the queries begin to decrease in number because the necessary corrections are made in the collection and reporting process.   Although queries cannot be completely eliminated by the use of eCRFs, they do significantly decrease the number of queries generated, by disallowing certain data to be entered, such as an age of 152 years old, and also requiring that all mandatory fields have been completed in real-time.  

Querying advice for the data manager

When writing queries for research conducted using paper CRFs, double check the Site Number, Subject Number, CRF Title and Question Number before sending the query.  Often there is an error in one of these which makes it unlikely or impossible for the site to answer.  If they are able to determine the correct Site Number, Subject Number, CRF Title or Question Number, it will require unnecessary time that most site staff do not have the luxury of.   These errors will also most likely result in having to send an additional query to the site, for which they will have to again spend time to resolve.   Other helpful tips include:

-          Do not combine queries on separate CRFs into a single query.

-          If a response is to be reported as a predefined value, list the options for the response, such as negative, positive, or not applicable. 

The best advice you can give to a participating hospital or clinic regarding queries, is to complete queries as soon as possible after receiving them.  This will alert them to errors/omissions sooner rather than later, and prevent them from continuing to complete additional forms in a manner that will continue to generate the same queries.  

Pros and Cons: Observational and Simulation Economic Studies

Over the last several years, our company has been commissioned to carry out studies whose primary purpose is to provide evidence to support reimbursement of a specific healthcare technology (either pharmaceutical or device).  In general, for both the US and abroad, there has been an increase in health economic literature because there has been a greater demand for this type of information from the reimbursement agencies, whether they are public or private.  In our experience, these studies have typically taken one of two forms: simulation modeling or an observational study.  This blog explores some of the trends in terms of publications; as well as the pros and cons of the respective approaches. 

Over the past five years, there has been an increase in publications associated with the health economics in general.  For instance, an advanced search (i.e. All Fields and Date of Publication) of PubMed for “cost” or “cost-utility” from 2007 to 2013 increased from 23,919, to 37,624 (57% increase) and from 151 to 294 (95% increase) [1].  Similarly, for the same time period, there has been an increase in studies associated with the keyword “observational” from 4,915 to 13,622.  Although a crude method of categorizing these types of studies, the approach indicates an increase of 57% for cost studies and an increase of 177% for observational studies which suggests a greater increase in observational studies.

This trend has paralleled some of our own experience over the past few years.  The limitations of cost-effectiveness analyses (i.e. simulation modeling) have been often mentioned [2].   The modeling effort itself depends upon data from previously published studies, and the data quality is an important factor to consider.  While RCTs are generally considered the gold standard for data quality there are other factors to consider when utilizing the efficacy results from RCTs for subsequent economic modeling efforts.  The similarity in study design, completeness and consistency of reporting, and protocol driven resource utilization are all factors to consider when compiling the evidence which will be the foundation of the simulation model. 

Under “good circumstances”, there may be several high quality studies available for each intervention of interest such as the reported outcomes (e.g. treatment success) are very similar for the set of studies.  Definition of the patient populations should be as similar as possible.  Also, if a cost-utility analysis is desired, it is necessary that generic health preference scores be available, or there be a mapping algorithm from a disease specific instrument [3].

However, in our experience, as well as the experience of others, there are many limitations in this type of approach [2].  In many cases, there are significant differences with respect to study design for the studies used to inform the models.  In many cases the QALY information is not ideal.  While it is becoming more common to consider quality of life information earlier in the research process, very often the generic health preferences are not available from the studies which were used to inform the efficacy parameters.  In some cases, it is necessary to map scores from one instrument to another, such as a mapping from the SF-12 to the EQ-5D [4].

While the above discussion covers some of the key limitations associated with economic modeling (i.e. simulation modeling), there are potentially additional limitations.  In summary, there are situations where the data quality is high, and the required data is available, and the similarity between the key studies is high, then the resulting economic model has a high likelihood of being both reasonably accurate and persuasive.  However, more often than not, the actual situation is more of a mixed bag of higher quality data and other data which is not ideal or from small samples.  In these cases, when the full set of assumptions and limitations are considered, the resulting analysis may not be as persuasive as would be desirable.

While observational studies have limitations as well, some of the limitations which are part and parcel of simulation modeling can be overcome.  Since the studies are generally prospective, there is the opportunity to design the study and capture all of the necessary data elements.  Also, since the study is indeed “observational”, the limitations associated with protocol driven resource utilization (i.e. from RCTs) is overcome.  The other factor to consider is that the resulting dataset is an original dataset, as opposed to an amalgamation of data elements from a series of previously published set of studies; this factor increases the interest and publishability of the work. 

In summary, while it is somewhat more expensive and time-consuming to conduct a prospective observational study, the benefits of this type of effort may be well worth it.  The trends in publication numbers indicate that more sponsors are likely choosing to shift from health economic modeling toward observational studies, in order to support the reimbursement of their technologies.

References:

1. http://www.ncbi.nlm.nih.gov/pubmed/
2. Weintraub WS, Cohen DJ.  The limits of cost-effectiveness analysis.  Circulation: Cardiovascular Quality and Outcomes. 2009; 2:55-58.
3. Parker M, Haycox A, Graves J. Estimating the relationship between preference-based generic utility instruments and disease-specific quality-of-life measures in severe chronic constipation: challenges in practice. Pharmacoeconomics. 2011 Aug; 29(8):719-30.
4. Sullivan PW, Ghushchyan V. Mapping the EQ-5D index from the SF-12: US general population preferences in a nationally representative sample. Med Decis Making. 2006 Jul-Aug; 26(4):401-9.