Brian D. Ray, Ph.D.*

     The purpose of this paper is to place in context and evaluate the efforts of many of those who negatively criticize homeschooling, homeschool researchers, or particular research studies on parent-led home-based education. What is “good research” and how should one understand negative criticisms of homeschool research?

Elizabeth Bartholet’s recent criticism of homeschool research and researchers, which originally appeared in the Arizona Law Review, will serve as a point of contextualization for this article.[i]  Professor Bartholet called for a “presumptive ban” on homeschooling in the United States, such that parents would have to petition the civil government to get permission to home educate any child. Under her regime, the government would become the controller of all children’s educational lives, and parents would need government approval for their children to be allowed to have something different than an institutional state-run education. A significant portion of Professor Bartholet’s argument involved negative criticism of much homeschool research and the scholars who have conducted the studies. I will address Professor Bartholet’s claims in the second portion of this piece.

What is “Good” Research? Research Methods 101

Social science research (e.g., education research) almost never sees “the perfect study.” While teaching research methods at the university graduate and undergraduate levels across seven years, I explained to my students that many studies in education are exploratory, descriptive, and correlational in design.[ii] Some studies include designs that allow for more conclusions that address cause and effect. But researchers are not allowed (morally or legally) to execute a cause-and-effect design like this:randomly assigning one-third of 1,000 students in New Orleans to state or public schooling, one-third to private institutional schooling, and one-third to private homeschooling, letting things roll for 13 years to see how all the students “come out in the end” on academic achievement or amount of abuse experienced at the hands of adults and peers,  then making a bunch of causal claims. Researchers are not allowed, by law or ethics, to force parents to put their children into these three types of schooling (i.e., educational treatment) and so may not conduct this type of study.

37 Years Ago

When I began doing research—about 37 years ago—on homeschool families, students, and groups, other scholars and I had a lot of questions. I wanted to know why parents chose homeschooling. What were their demographics? And how were the children performing academically?

When I first set out to measure homeschool educational attainment back in 1985, there were several challenges. How do I study a “hard-to-reach population?” What is a way to gain participation when there is no master list from which to randomly select families or students? How do I ethically engage their participation? How can I statistically control for background demographic traits so that reasonable comparisons can be made to public school student norms? How will I get test scores when there is no research budget for new testing? How can I do this on a shoestring of a budget and with very little manpower to assist me? (And especially when thousands of old-fashioned, multipage survey questionnaires would be going out and coming back via the US mail and all the data would be entered by hand, one question and click at a time, into a spreadsheet.)

As in any study, there were problems and challenges to executing my studies the ideal way. With hard-to-reach populations, there are typically even more obstacles. To learn something rather than nothing, researchers must come up with solutions and compromise between the ideal approach and the realistic or practical approach. I had to do that; just as most other social scientists have had to do. I had a lot of conversations and brainstorms with other scholars, leaders of homeschool organizations, academic achievement testing services, government officials, and homeschool families. It was and is complicated. It tries the mind. It pits the curious scientist against reality while trying to get questions answered. It was exhausting at times. It was thrilling at times. I thoroughly enjoyed it (and still do).

The Perfect Study

It is highly unlikely that an investigator will ever get to do a “random-assignment true experiment” regarding the “effects” of schooling type (e.g., public institutional school compared to private institutional school compared to homeschooling). Research opportunities and life are just not that way. Research scholars must work within budgetary, timeline, and ethical constraints to design and execute the best study that they can to meet their objectives. So, for example, since a scholar may not do a “true experiment,” the alternative might be some kind of “quasi-experimental” or non-experimental design, such as a matched-pair study (or a retrospective, explanatory study; or a cross-sectional, predictive study; or a cross-sectional, explanatory study).[iii]

Every study related to homeschooling is based on a particular research design, with an underlying theoretical framework; and each specific study has limitations and delimitations, based on its methods (which are often based on the aforementioned factors: budgetary, timeline, and ethical constraints), regarding what it can and cannot reveal and to what extent its findings can inform understanding and policy considerations. Every researcher should clearly state the study’s purpose and objectives; lay out a review of related literature; present a theoretical framework and explain methods; and, finally, present the findings and analysis. A researcher should also note the limitations of his study and provide a discussion of conclusions and interpretations. The conclusions should follow from and be coherent with the study’s methods and limitations.[iv]

The study is sound research if the researcher takes all these steps. If things are clearly stated so that they are conceivably reproducible and the conclusions that the scholar states reasonably follow from the methods, findings, and limitations, then it is sound, beneficial, or “good” research.

It is a bogus approach for any observer, evaluator, or critic to say that a research study is “bad” or “flawed” because it (a) does not meet all of the objectives that the critic wished it had included or met, (b) cannot lead to causal statements, or (c) led the investigator to suggest policies or ideas that the critic does not like. If a researcher is clear about the study’s theoretical framework, methods, conclusions, and limitations, and her conclusions and recommendations generally fit the findings and overall framework, then it is a decent study and people can learn from it. It can inform the field of inquiry.

Big Picture of Some Homeschool Research

Overall, research has found many positive things connected to homeschool students and families. .....

    To continue with Dr. Ray's unique article, click here.

The National Home Education Research Institute is celebrating 30 years of service to homeschoolers, researchers, and the general public. Donate to the non-profit NHERI here.