Validation of Paper and Web-based Calcium Assessment Tools: Are they accurate tools for health professionals’ use?

Background

Research continues to substantiate the health benefits of milk and dairy foods, showing a protective effect against osteoporosis, certain types of cancer, hypertension, type II diabetes, and most recently, weight management efforts. Much of the research on milk and dairy foods shows benefits at levels consistent with or slightly higher than the current recommendation of 3 cups per day.

Dairy products continue to be an important source of calcium, providing over half the calcium intake for most individuals ¹. Addressing calcium needs will help to optimize bone health by improving deposition in early adolescent and teenage patients, maintaining bone density in adults, and minimizing bone loss in older patients. Educational interventions promoting adequate calcium intake with tools estimating dietary calcium that can be easily utilized by consumers would be a valuable asset to promoting skeletal health. Validation of these tools against standard dietary assessment methods enhances their utility and influence.

The goal of a food frequency questionnaire (FFQ) is to provide an instrument that can assess dietary intake in a way that is valid, easy and inexpensive to administer in research and clinical environments ². An FFQ may be inexpensive and easy to administer but it must also be valid. Nutrient data from diet records and FFQ are usually reasonably correlated, with correlations ranging from 0.5-0.7 ^3,4.

This project was undertaken to validate two Dairy Council of California calcium questionnaires: the online Calcium Quiz and the calcium questionnaire in the publication Calcium Connection compared to the calcium intake recorded in a three-day food record. The University of California at San Francisco’s Mt. Zion Osteoporosis Clinic research team was contracted by Dairy Council of California to conduct this study.

Methodology

To maximize the generalizability of the results, exclusion criteria were minimal, focusing on individuals with health conditions that minimize their intake of calcium containing foods, inability to keep a diet record, active gastrointestinal disease that limits or restricts diet, or inability to understand and provide informed consent.

The primary goal of the analysis is to validate the dietary calcium questionnaires against a three-day food record. Data from the diet records were entered into a nutrient analysis program. The two dietary calcium questionnaires were entered into a database and analyzed. The overall data analysis was conducted using Pearson correlation coefficients and the Bland-Altman test to determine if the two dietary calcium questionnaires are valid when compared to a three-day diet record. Data were analyzed for the entire group and sub-analyses conducted by ethnicity to determine if the questionnaires are equally valid in women of all ethnic backgrounds. The sample size was calculated to validate the questionnaires if they correctly estimate dietary calcium intake within 115 mg of the food record in the whole group (n required=150), using the standard deviation of dietary calcium intake from NHANES data. Goals to recruit additional non-Caucasian subjects were set to ensure the ability to validate the questionnaires with a margin of error in each ethnicity of less than 200 mg (adds an additional 23 participants).

Results

From advertisements, 209 women passed telephone screening and were scheduled for participation. One hundred sixty four women completed the study visit and completed both questionnaires, and were provided materials for the three-day food record to be completed at home. Five women reported sending in a food record, but no record was received. An additional 19 women did not return a three-day food record despite several reminder calls. Thus, complete data (two calcium questionnaires and three days of food records) are available for 140 women.

The ethnicity distribution among the 140 study participants for whom complete data are available was 102 Caucasian, 12 African American, 16 Asian, and 10 Latin women. The average age (mean ± SD) was 49 ± 15 years (mean ± SD) with a range of 22.7 to 89.9 years. There were 74 women in the 19-50 years old age category, 54 women in the 51-70 years old age category, and 12 women over age 70 years.

The average daily calcium intake from the questionnaires was 834.4 ± 554.1 mg/d for the online Calcium Quiz, 719.5 ± 521.9 mg/d for the Calcium Connection printed questionnaire, and 901.8 ± 355.9 mg/d for the food record.

The two Dairy Council of California questionnaires were highly correlated with an r=0.82, an expected result as they were administered on the same day, although they were administered in random order. Correlations between each questionnaire and each day of the food record ranged from 0.21 to 0.39, with a nearly identical correlation between each questionnaire and the three day calcium intake average from the food records of 0.37. The correlations with the average intake derived from the food records were statistically significant at p<0.001.

Correlations appeared to differ by ethnicity. Among the ethic subgroups, the correlations between the questionnaires and the food records were highest for African American women, followed by Caucasian women. Correlations were poor between both questionnaires and the food records in Asian women while the Latin women showed disparate results with a reasonable correlation between their food records and the Calcium Connection questionnaire, but a low correlation with the online Calcium Quiz.

There was also a trend toward higher correlations in younger women (Table 1). In the youngest quartile of age, the correlations between the three-day diet record and the Calcium Quiz and the Calcium Connection questionnaire were 0.43 and 0.50 respectively. Average age in this quartile was 30.5 ± 4.0. In quartile 2, the average age was 42.3 ± 3.9; correlations between the three-day diet record and the Calcium Quiz and the Calcium Connection questionnaire were 0.60 and 0.45 respectively. In quartile 3, the average age was 54.9 ± 2.7; correlations between the three-day diet record and the Calcium Quiz and the Calcium Connection questionnaire were 0.26 and 0.39 respectively. In quartile 4, the average age was 68.1 ± 7.3; correlations between the three-day diet record and the Calcium Quiz and the Calcium Connection questionnaire were 0.13 and 0.02 respectively.

Summary and Conclusions

The overall correlations between the two Dairy Council of California calcium questionnaires and the three-day food records are similar to the ranges of correlations seen in previous studies of limited item calcium food frequency questionnaires. Previous studies in other populations have shown correlations from 0.31 to 0.73 with most values in 0.4 to 0.5 range. Studies with higher correlations have utilized questionnaires with more food items (up to 209 items), more days of dietary records (up to 11 days), or repeated administration of the food frequency questionnaire ^5-13. Examination of the subgroups and outlying values has revealed some observations that may help improve the questionnaires.

• It has been previously demonstrated in teenagers that calcium questionnaires perform less well in some ethnic subgroups ⁹. Examining the diet records of the non-Caucasian subjects in this study showed that foods that were unique to these ethnic/racial groups were listed as individual items in the questionnaires, but not as a mixed dish as they are often consumed. For example, Chinese greens and/or tofu as part of a stir fry or beans, cheese, and tortillas as part of an enchilada. If a mixed dish is prepared at a restaurant or by another individual the study participant may not realize or remember the components of the dish. They may also underestimate the amount of that ingredient that was in the food. In contrast, when the participant keeps a diet record they are required to document the components of the mixed dish item that they are consuming and to measure what those components are. The process of measuring food may have altered the participant’s intake. They may have chosen to eat at home where measuring the food was easier or chose items on a restaurant menu that were easier for them to assess and measure.
• Latin women had a better correlation between the Calcium Connection and their food records than the Calcium Quiz questionnaire. This appeared to be at least partly due to refried beans appearing as an individual item on the Calcium Connection.
• In Asian diets dairy products contribute close to 25% of calcium in the diets and western diets 50% which could explain the low correlation between the FFQ and diet records in the Asian population.
• Both questionnaires ask women to fill them out considering what they ate the day before. While not measured directly, the women completing the questionnaires repeatedly commented that their consumption the previous day was not typical for their usual intake. The diet records may more accurately represent their usual intake and thus, have a poor correlation with the one day represented by the questionnaires.

In conclusion, the two questionnaires performed reasonably well. They are considerably shorter than other calcium questionnaires that have shown a higher correlation with food records. As the goal of a public health questionnaire such as these is to be simple and widely utilized, a shorter questionnaire would be preferable although the correlation to the gold standard food record may be somewhat lower. In their current form, the questionnaires provide valuable information; however, this project has identified several areas that may improve the questionnaires and further enhance their utility.

View Calcium Connection booklet materials.

Reference List