Generalizability: this is the term frequently use in research to describe the ability of a particular study outcome (say, benefits of high intensity interval training, or HIIT) to apply to the population as a whole. In research, generalizability is necessary , since it is impossible to sample EVERY single individual (although an n=300,000,000 looks nice) in the population and randomize them to an experimental arm. Ideally, in a scientists utopia in which this could happen, we wouldn’t even need small studies, since we could know how every single person responds to our intervention. Unfortunately, due to limited funds, resources, and time, we simply can’t sample everyone.
So, we take a “sample” — a group of individuals (perhaps 20–60 for a given study) who we assume to be more or less a representation of the population. Random sampling attempts to make our sample even more representative, since we aren’t “hand picking” participants, but even then, we only have a handful of individuals out of a larger whole — far from encompassing all of the variation in a place like the United States.
What’s more — studies often define inclusion (what you need to be “in” the study) and exclusion (what prevents you from being in the study) criteria; age range, normal blood pressure, being a male — to name a few. These criteria, while making the study internally valid (since all of our participants are similar), subsequently makes the generalizability fairly low. 20 year old males respond differently than do 20 year old females than do 70 year old males or females.
Sure, there are studies which specifically target a population (hypertensives, for instance) with the goal of providing recommendations, based on study outcomes, for hypertensives. This is valuable research and is largely why many successful interventions in medicine have been developed.
Back to generalizability and our HIIT example. What can we conclude when we see improved cardiovascular function in 20 year old males after 12-weeks of training?You guessed it — that 12 weeks of HIIT improves cardiovascular function in 20 year old males. What we can’t (or shouldn’t) do is use our results to generalize the efficacy of HIIT to everyone outside our population. While it may be true that HIIT benefits a variety of populations, our hypothetical study only shows benefits for our sample and others akin to our sample.
This is where health and physiology research has failed in the past. I don’t want to talk about prior failings in this post — but the theme, historically, is that most research has been done in healthy, college-aged male populations — This has been done, as we will see, largely for reasons of convenience. Offer up free pizza on a college-campus and you are sure to find a cohort of males eager to participate in whatever experiment you wish.The results of studies are presented as a male/female combo and “clumped” as if they were one, homogenous sample.
Homogenous we are not. Recent studies suggest that males and females (surprise!) are vastly different when it come to how we respond to exercise both in the short term and the long term. It is now known that the menstrual cycle and the accompanying change in hormones (namely, estrogen) greatly influences the response to exercise, among other things. Age further complicates the matter. Pre-menopausal women have been shown to respond quite differently than post-menopausal women to exercise training. While it seems intuitive that these surface differences exist, it is only recently that scientists have understood just how these sex differences translate into physiological disparities when it comes to experimentation and drawing conclusions.
It is no longer acceptable, given our knowledge, to exclude women from physiology research. Not only is it not acceptable, there is now a priority to study specifcally, women. We aren’t searching for our lovely term “generalizability” anmore. We need studies that acknowledge that sex differences exist and need to be studied, in isolation.
Indeed, the National Institutes of Health (NIH) as of November 2017, updated their guidelines on the inclusion of women and minority groups as subjects in clinical research.
Amendment: NIH Policy and Guidelines on the Inclusion of Women and Minorities as Subjects in Clinical Research
Notice Number: NOT-OD-18–014
Release Date: November 28, 2017
If a study of only one gender or minority group or subpopulation is proposed, there must be a scientific justification for limiting the diversity of the study population, such as high prevalence of the condition, unique disease characteristics, or gaps in knowledge in the select population. The review committee will include the adequacy and scientific appropriateness of a justification in its determination of the priority score. Women of childbearing potential should not be routinely excluded from participation in clinical research.
Luckily, research is being done, and labs are finding fascinating results.
For instance, a recent paper by D’Urza et al sought to determine in fluctuations in estrogen throughout the menstrual cycle had any effect on the blood flow response to exercise. Because the increase in blood flow to an exercise stimulus (in this study they used handgrip exercise) is a marker of vascular function (how well your blood vessels work), the results would indicate whether estrogen plays an important role in exercise responses in young females. One of estrogen’s many roles is to act as a vasodilator, causing an increase in blood vessel diameter to stimulate blood flow and lower blood pressure, acting through a molecule known as nitric oxide (NO). Interestingly, they study revealed differences in the blood flow response despite changes in estrogen (high estrogen; late follicular phase vs. low estrogen; early follicular phase), implying that for this type of exercise, menstrual phase may not play a role in young, healthy women.
Do women adapt to exercise training similarly to men of the same age? This question is important, given the fact that we know (unequivocally) that exercise protects against a variety of different cardiovascular diseases and dysfunction — especially in aging adults. Recently, it was shown that estrogen-deficient post-menopausal women, despite being considered aerobic exercise “trained,” had no better vascular function (a marker of cardiovascular disease risk) than their sedentary peers (also post-menopausal women). The study also compared vascular function of post-menopausal exercise training women to pre-menopausal women who didn’t exercise (sedentary group) and found that the pre-menopausal sample had considerably better vascular function. Thus, estrogen may play a major role in vascular function with aging. Results are in contrast to what is observed in healthy middle-aged and older aerobic exercise-trained men — who show responses to training of similar intensity.
What does this tell us? Is exercise “useless” in post-menopausal women? Surely not — given that exercise has numerous benefits outside of improving vascular health. Indeed, in the same study, post-menopausal exercise trained women had lower C-reactive protein and oxidized LDL, which are both associated with greater risk for coronary heart disease (CHD). However, research like this provides insight into the mechanisms responsible for different effects of aging in men and women, and may allow researchers to tailor interventions to fit the need of women.
Even more recently, matters of menstrual health have been considered in terms of athletic performance. Secondary functional hypothalamic amennorhea (SFHA) is reported to be a common finding in female athletes — and those in weight-sensitive sports like endurance running are said to be more susceptible, since many of these activities inevitably lead to low energy availability which may have consequences for female reproductive health and function. Tornberg et al studied exercise capacity, knee muscular strength (KMS), knee muscular endurance (KME), and reaction time (RT) in 16 eumennorheic (normal) female athletes and 14 females with reported SFHA (amennorheic). Not only were estrogen and thyroid lower, and cortisol and glucose higher in amennorheic athletes, but reaction time was found to be slower by 7%, knee muscular endurance lower by 11%, and knee muscular strength lower by 20%. This is one of the first studies to directly measure performance in amennorheic females and compare them to eumennorheic peers. We already knew that presence of amennorhea effects general health and well being — and now further insight is provided into how athletic performance may be directly linked to reproductive health.
The point of this story isn’t to highligh tthe specific findings of a few hand-picked studies, but to illustrate the changing paradigm in health research. As the updated NIH policies reflect, scientists and society demand a change in how we design and publish research. Once, n=28 (48% female) was okay. As long as researchers reported the number or percentage of female sincluded in the study, then they could report combined male and female conclusions as a whole, and this was considered normal and acceptable.
As responsible scientists and consumers of science, we can no longer accept research like this. What are the two alternatives?
The first — separate study co-horts (if sex differences are found to exist after the statistics are run) should be presented independently; data for males and data for females. Many studies now even report individual data in order for other researchers to see the variability in response among participants.
Option number two — just do more research in women, and only women. The only real caveat to this is inconvenience — since we know that estrogen and the menstrual cycle may play a role in response to an intervention, we must “schedule” experiments to occur at the same time of month, reducing inter-participant variability and thus increasing our generalizability.
Will minor inconvenience and possibly a larger study budget prevent us from conducting rigorous science, science that will allow us to better understand the vast-unknown of human physiology? Our lab and others are determined to design quality interventions that control for female differences and influences of hormonal fluctuations, and how these differences may effect the response to exercise. The future is bright for studying female physiology — we are heading into a new area of research that will allow us to improve the health of females and realize that our inherent differences are present, and are fascinating.
No generalizations to be made.