Evaluating the Reliability of Health Data
Bart Laws 2002

1. Background
2. Minority Health a new Concept
3. Legislative Acts Concerning Minority Health
4. How is the Data Collected and How Accurate is it?
5. Who Collects Data?
6. Major Data Collectors
7. Other major Data Collectors
8. Data Sources Used by the Indian Health Service
9. Data Sources Used to Address Access to Care Issues:
10. Proposed Methods for Achieving Accurate Data Collection
11. NHS Support for Science – Peer Review of Data Submissions and Information Systems



Background

Currently available data is not sufficient for planning and targeting public health programs to meet the needs of disadvantaged minority populations. Nevertheless, the limited data that is available shows persistent, distressing disparities in key health indicators among minority populations relative to the overall population. Compared to the white population, minority populations in many instances suffer higher rates of morbidity and mortality.

For example:1

• Infant mortality rates are 2.5 times higher for African Americans and 1.5 times higher for American Indians/Alaska Natives.
• African American men under 65 suffer from prostate cancer at nearly twice that of white Americans.
• Asian Americans and Pacific Islanders have the highest rates of tuberculosis.
• Hispanic Americans have two to three times the rate of stomach cancer.
• American Indians/Alaska Natives suffer from diabetes at nearly three times the average rate, while African Americans suffer 70 percent higher rates than white Americans.
• More than 75 percent of AIDS cases among women and children occur among racial/ethnic minorities, primarily African American and Hispanic American.

However, commonly used categories – Black, Asian, Hispanic, American Indian/Alaska Native – are actually heterogeneous, and mask substantial internal variation in health status and social determinants of health. Furthermore, even these categories are not always applied consistently or reliably.

Minority Health – A New Concept

As a result of such discrepancies in health data of racially and ethnically diverse communities, the concept of minority health is a new and rapidly evolving field of study shaped by changing demographics and the multicultural nature of our society. Current focus includes:

• Addressing the multiple complex issues related to eliminating health disparities for racial/ethnic minorities.
• Facilitating an understanding of the benefits of culturally competent health care, thereby reducing the potential for misdiagnosis of clients and inappropriate treatment.
• Promoting an understanding of racial and ethnic differences in response to drugs.
• Stimulating the development of strategies to overcome racial biases in the delivery of health care.
• Fostering the integration of culturally-related health factors into the design of intervention programs.
• Supporting the adoption of policies and research initiatives that enhance health outcomes for underserved minority populations and directly address the design and delivery of health care systems that respond to the specific needs of racial/ethnic minorities.3

Legislative Acts Concerning Minority Health

• The Disadvantaged Minority Health Improvement Act of 1990 authorized the National Center for Health Statistics (NCHS), and Center for Disease Control (CDC) to establish a grants program to improve minority health statistics.
• Some of the Main Objectives:
  1. Re-evaluate survey questions for wording, accuracy of measurement, and cultural appropriateness.
  2. Increase the pool of minority researchers, especially from communities where research is being conducted.
  3. Make existing national databases more accessible to those interested in minority health statistics research.
  4. Develop methods to reduce linguistic barriers between minorities seeking health care and the health provider.
• The Minority Health and Health Disparities Research and Education Act of 2000 -- Focuses on evaluating the methodology for data collection relating to race and ethnicity.
• Discrepancy Between the Public & Private Sectors: Private sector health plans purchase information and expect standardized performance measures as a result. The public sector (public health) is faced with decreased emphasis on the direct provision of medical services and increased emphasis on the purchase of quality care, assessment of health status and community health needs, and development of timely, informed public health policy. Changes are needed to conjoin the interests of public and private health arenas, driven by a changed purchasing environment focused on health information systems. The public sector wants to craft the purchasing demands and performance measures, which influence private sector information systems.
• Assessment: Standardization of health information platforms in both the private and public sectors is necessary for uniform efficiency when addressing the challenges of health issues; fostering increased interest in sharing health information between health plans & employers, and private and public health agencies.4
• View the Minority Health Database Catalog: http://www.minority.unc.edu

How is the Data Collected and How Accurate is it?

Current national surveys were developed according to dominant, white, middle class standards. Therefore, methodologies used to collect and analyze data have resulted in the misrepresentation of minority populations. There is a need to develop survey standards utilizing culturally sensitive data collection tools in order to properly represent and reflect the health needs of a culturally diverse and increasingly heterogeneous populace.

Self-Reported vs. Administrative Data – When self-reported data collection methods were observed, it was noted that many individuals choose multiple race/ethnicity categories. This caused the sample size to be falsely increased and therefore skewing the survey results. Administrative data collection methods have also resulted in sampling discrepancies, misrepresentations, and even omissions of many individuals because at times the race/ethnicity categories listed did not apply to the individual participating in the survey. This would force the participant to choose an incorrect category, leave the category blank (coded as “missing”), or make up a new category unknown to the administrative database. It is evident from such discrepancies and disagreements between race/ethnicity categories offered and chosen, that the source of the data is a key factor in the ongoing efforts of eliminating health disparities among racial and ethnic minorities.5


Robin Weinick, Ph.D., and Director of Intramural Research, Center for Primary Care Research at AHRQ presented a framework for thinking about access to health care services. The framework consists of six major components to consider in determining how one gets the necessary care in order to produce good health outcomes. She described the measures associated with each component and provided examples of related research on disparities in access to care. The major components include:6

• Health insurance coverage
• Access to care
• Access to health and health care information
• Utilization
• Quality of care
• Discrimination, language, trust, and acculturation (the modification of the culture of a group or individual as a result of contact with a different culture)

Who Collects Data?

• AHRQ (Agency for Healthcare Research & Quality) has a list of data sources for studying disparities:7
  1. Medical Expenditure Panel Survey (MEPS) – information about health care use and expenditure at the national level. MEPS is the only national survey that provides a foundation for estimating the impact of changes in sources of payment and insurance coverage on different economic groups or special populations of interest, such as the poor, elderly, families, veterans, the uninsured, and racial and ethnic minorities. Data from AHRQ's (MEPS) reveal that:
  • About 30 percent of Hispanic and 20 percent of black Americans lack a usual source of health care compared with less than 16 percent of whites.
  • Hispanic children are nearly three times as likely as non-Hispanic white children to have no usual source of health care.
  • African Americans and Hispanic Americans are far more likely to rely on hospitals or clinics for their usual source of care than are white Americans (16 and 13 percent, respectively, v. 8 percent).8
  2. Healthcare Cost and Utilization Project (HCUP) – a family of health care databases and related software tools and products developed through a Federal-State-industry partnership. It provides hospital discharge data for 29 states. AHRQ sponsors HCUP.
  3. HCUP Nationwide Inpatient Sample (NIS) – a database of hospital inpatient stays. Researchers and policymakers use the NIS to identify, track, and analyze national trends in health care utilization, access, charges, quality, and outcomes.
  4. Consumer Assessment of Health Plans (CAHPS) – provides information to help consumers and purchasers assess and choose among health plans. CAHPS focuses on gathering data on the experiences of users of health care services and reporting the results in a way that is meaningful to consumers. By providing consumers with standardized data and presenting it in a way that is easy to understand and use, CAHPS is intended to help people make decisions that support better health care and better health. This emphasis on the consumers' point of view is what differentiates CAHPS reports from other sources of information about clinical measures of quality.9
  5. State Inpatient Databases (SID) are one in a family of databases and software tools developed as part of HCUP to inform decisionmaking at the national, State, and community levels. Researchers and policymakers use SID to investigate questions unique to one State; to compare data from two or more States; to conduct market area research or small area variation analyses; and to identify State-specific trends in inpatient care utilization, access, charges, and outcomes.

Major Data Collectors

• Department of Defense
• Department of Health and Human Services (Office of Minority Health)
  • Agency for Health Care Policy and Research
  • Centers for Disease Control and Prevention and Agency for Toxic Substances and Disease Registry
  • National Center for Health Statistics
  • Food and Drug Administration
  • Health Care Financing Administration
  • Health Resources and Services Administration
  • Indian Health Services
  • National Institutes of Health
  • National Cancer Center- International Cancer Information Center
  • National Library of Medicine
  • Substance Abuse and Mental Health Administration
• Department of Transportation
  • Office of Emergency Medical Service
• Department of Veterans Affairs
• Federal Emergency Management Administration
• Office of Personnel Management 10

Other Major Data Sources for Health Status Indicators 11

• National Vital Statistics System—Principle source of data on births and deaths.
• National Health Interview Survey (NHIS)—Principle source of information on the health of the civilian non-institutionalized population of the U.S.
• Behavioral Risk Factor Surveillance System (BRFSS)—Tracks risks among adults associated with premature morbidity and mortality.
• National Health and Nutrition Examination Survey (NHANES)—Collects information about the health and diet of people in the United States.
• National Hospital Discharge Survey—Provides information on characteristics about inpatients discharged from non-Federal, short stay hospitals in the U.S.
• Youth Behavioral Risk Surveillance System (YRBSS)—School-based survey to assess health-risk behavior among youth and young adults.
• Linked births/Infant Death Data Set—Used to overcome discrepant reporting of race, especially for minority categories other than Black.
• Compressed Mortality File—County-level national mortality and populations database that provides counts and rates of death.

Data Sources Used by the Indian Health Service (IHS):

• 1990 Decennial Census Data—Used to calculate vital events data, develop budget justifications, estimate service area access to IHS services, and analyze the growth rate of the service populations.
• NCHS Vital Events Files—Used to project the service population for IHS service areas and prepare vital event statistics on AI/AN.
• IHS Patient Care Database—Used to produce user population estimates and patient data on leading causes of hospitalization, ambulatory medical visits, clinical impressions, and for Government Performance Reporting Act (GPRA) reporting.

Data Sources Used to Address Access to Care Issues:

• National Health Care Survey—Provides a wide range of data on the health care field, including utilization, medical technology and quality of care.
• U.S. Census 2000 data—Provides population and housing data for every housing unit in the U.S. in 2000.
• U.S. Census Current Population Surveys—Provides up-to-date information on social, economic, and demographic characteristics of Americans.

Directory of Health and Human Services Data Resources: http://aspe.hhs.gov/datacncl/datadir/index.shtml

Proposed Methods for Achieving Accurate Data Collection

It should be recognized that health data usage is for researchers, public policy and health officials to analyze and comprehend reasons for trends in minority health disparities, to develop methods to eliminate those disparities, to monitor increasing diversity, and to improve the health and welfare of communities. Therefore, collection methods and data must be formatted and worded such as to capture the knowledge and understanding of how a culture views itself within a community. This is an important factor of identifying trends of an increasingly diverse nation and the impetus that accompanies it. How can this be done effectively? By bridging the gap! By strengthening the relationship of data collection efforts with CBO’s and empowering members of a community to contribute feedback and develop alternatives to design and implement their own research to benefit their community.

Identifying the appropriate language and terms comprehensible to members of a community is a necessary means of building the capacity of these CBO’s. The recognition of these collaborative efforts will assist in creating a succinct and sustainable level of autonomy for a community by allowing it to become involved in future research methods that reflect the specific needs of their community. Thereby maximizing the benefits and services provided to members of a said community. By recognizing and building on this important standard of “bridging the gap”, these efforts will ultimately improve the overall capacity of minority CBO’s and the many services they provide to their communities.

As stated earlier, current national surveys were developed according to dominant, white, middle class standards. Therefore, the methodologies used to collect and analyze data failed to yield a sufficient representation of racial and ethnic minority populations. Here are some recognized changes that need to be implemented in order to eliminate this reoccurring problem:13

• Use of cognitive methods (Northern California Cancer Center or NCCC and the Survey Research Laboratory)
• Focus groups
• Think-Alouds
• Sampling techniques – small group disorders go undetected (or represented)
• Sample-resample methodology – (similar to capture-recapture) method in ecology. Examine intersection of different sources listing individual diagnoses.
• Data linkages – standardization to avoid occurrence of racial/ethnic misclassification (one data set can fill the gaps of another data set)
• Make current national databases more accessible to researchers of minority health.
• Consider Minority Health Database Catalog at UNC-CH as an example.

NHS Support for Science – Peer Review of Data Submissions and Information Systems

• Revamp standards of survey categories for data collection by:
  1. Identifying methods of collection and evaluate those methods by statistical data (regression models: what works, what doesn’t and why?)
  2. Identifying glitches and problem areas
  3. Identify indistinguishable or ambiguous areas
  4. Clarify definitions and test for comprehension and accuracy of intended outcome
  5. Note “best practices”
• Identify and Focus on Outcomes of Peer Reviews:
  1. Emphasis placed on improving respondent methods
  2. Analyze reasons for low respondents (funeral directors, health clinics, public health agencies) **see attached
  • Easier access methods (download, upload, what works? what is least cumbersome?)
  • Recognition of different target respondents, thereby using appropriate formatting methods for that target
  • Maintain consistency (note areas/reasons for inconsistency)
  • Standardization of data terms a hybrid methods for statistical analysis (links to other sources of information) – Maintain consistency!
  • Mandatory response methods needed to increase the sample size (or reduce sampling error)
  • National Electronic Disease Surveillance System (NEDSS)15 – a need to integrate surveillance systems, which transfer health, laboratory, and clinical data securely and efficiently over the Internet.
    1. Facilitate electronic transfer of health/clinical info systems to public health departments
    2. Reduce provider burden in provision of health information (perhaps a key factor in low respondents and inconsistencies)
    3. Improve the timeliness and quality of the information provide and access to that information
• Issues: 1. Data gaps were left unfilled by national surveys inhibiting researchers to draw reliable conclusions about minority health issues. 2. The surveys were not designed to collect data from racially and ethnically diverse communities.
• Assessments: 1. There is a need to develop ways of improving cross-cultural health data research by collaborating with other projects that use surveys on minority health (ex. Pathways to Early Cancer Detection for Hispanics, Vietnamese Americans, African Americans, and Chinese Americans, and Breast & Cervical Cancer Intervention Study (BACCIS) 2. Develop ways to study the effects of culture and socioeconomic status on methodological problems when evaluating minority health research.