Does Self Improvement Work?

Data on Effectiveness of Changing Personal Environments to Achieve Behavior Change

Executive summary

         Since 1999, nearly 300  graduate students in health professions have done personal improvements in three universities (George Mason, Vanderbilt and Case Western Reserve universities).  Among these, 129 were at George Mason University.  They signed a consent form that allowed us to collect information from them about their experience.  On average, these participants had attempted but failed 2.96 times to achieve their resolutions.  After a brief introduction to Continuous Self Improvement, participants were asked to make ecological changes that would facilitate bringing about lasting improvements.  Participants were asked to collect objective data on their improvements and to test the statistical significance of changes they bring about.  Their class grade did not depend on their success in improvement.  Eight to fifteen weeks later. they reported their data.  20.66% reported they had made a measured and 69.42% reported they had made a measured and statistically significant improvement in their lives.  Additional analysis showed that participants who relied less on their own motivation and more on environmental changes were more likely to succeed. 

          Participants, who made environmental changes, increased their odds of success by 1.5 times.  In contrast, participants, who relied on their own motivation, changed their odds of success by 0.58 times.  Changing the environment improved the odds of success by 2.59 times more than relying on one's motivation. 

          These findings may not generalize to other settings because it summarizes the experiences of graduate students and not the general public, we relied on data gathered by the participants, we did not follow participants for a long period and we did not randomly assign participants to control and experimental groups.  Despite these limitations, the data show the value of making personal improvements through systemic changes in the environment. 

Introduction

To help people make systemic changes in their lives we suggested the following seven steps:

1. Make a resolution that is realistic and that you care about.
2. Put together a team of "process owners," people who live with you and who help you carry out daily living activities.
3. Together describe life processes and ecological factors that affect your habits.
4. As a team, list possible environmental changes & simultaneously make several changes.
5. Monitor progress.
6. Engage in cycles of improvement. Plan for, do, check and act again.
7. Publicly post your story so that all team members have access to your progress. 

People who follow these steps, i.e. people who seek ecological and systemic solutions to their habits, are more likely to achieve their resolutions.  From this perspective will power is not a characteristic of the person but a feature of the environment. When the individual modifies his environment, he changes his ability to stay the course and succeed, he engineers his own will power.    Of course, will power is elusive.  Eventually it waivers and people sometimes fail to achieve their resolutions.  We claim that people who follow our advice are more likely to stay the course.

The central idea behind our proposed method is that one can change the environment to accomplish personal goals.  For example, a person can reduce the room temperature to increase energy consumption.  Another person may join a car pool in order to make it home in time to cook.  Still another person may commute to work by bicycling.  Yet another person may raise his work desk and work standing up.  A large number of studies support the role of environment in habit formation.  Most of these studies are in maintaining recovery from drug abuse.  Our own earlier study provides data on a diverse set of personal resolutions.  In this study of a few graduate students, we reported that 83% of participants achieved their resolutions within 15 weeks. 

No one can guarantee success.  Some people follow our advice and do not accomplish their goals.  Others readily succeed, even when they are not following our advice.  The empirical question is whether our advice improves the likelihood of success.  It is a game of chance and the question is whether the probability of success is improved when people follow our advice.  The answer to this empirical question is not simple.  Some people do not follow our advice even when exposed to its logic.  For example, we asked participants to modify their environment but they may have misunderstood us; or even when they understood us they may have been unable or unwilling to do so.  The very definition of success is murky.  Some people will consider modest weight loss as success, others or the same person at a different time, will consider such weight loss as inadequate.  We asked participants to set their own goals and define what they consider success.  Obviously, the probability of success depends on how difficult their goals were and how long a time they had to achieve it.  Short and long follow up periods affect the probability of success.   

Methods

Nearly 300 graduate students in George Mason, Vanderbilt, and Case Western Reserve Universities have participated in our improvement efforts.   Among these we collected data from 129 students mostly at George Mason University.  These students came from a diverse background including nursing, medicine, nutrition and health system management.  The students were told that their grade will not depend on their success or failure.  All students who were asked participated.   

We began collecting data on 12/5/2001 and ended data collection on 6/16/2008.  During the course of this time, the instructors as well as the content of what was taught changed.  Later classes put a bigger emphasis on systemic changes as opposed to increased motivation or effort.  Later classes also put more emphasis on use of control charts to test if apparent improvements could be due to random chance variations. 

It is natural that any intervention delivered over several years and several sites, may have differences in emphasis.  The changes in the intervention make the interpretation of the findings difficult.  At the same time, these changes provide preliminary data on impact of components of the intervention. 

Participating students had  3.65 years of education post high school graduation (number of respondents= 121, standard deviation=5.77), which is significantly more than the general population. 

We followed the students for 8 to 15 weeks later and asked them to complete a private self evaluation regarding what their resolution was, whether they succeeded and whether they had data supporting their claim of success.   All were required to hand in a storyboard containing their personal goals, their analysis of their lifestyle and their evidence that changes they had introduced had led to improvement.

Results

What did participants do?

Of the 129 participants, 129 reported their resolution.  Among these, 25.58% (n= 33) had resolved to reduce their weight or change their diet,  42.64% (n=55) resolved to exercise and become more active and 37.98% (n=49) had made other resolutions (e.g. better time management, or reducing stress in their lives.). 

Not all of the 129 participants kept up with our recommendations.  Given the variety of instructors involved, the diversity of participants backgrounds and resolutions, and changes in the nature of the intervention, it is not surprising that participants followed some but not all of the advice given:

• We recommend that participants should make a resolution on a topic that is important to them so that they can sustain their effort over time.   Out of 129 participants, 3 were not asked, did not answer or were not sure of their response to the importance of the resolution.  Among responders,  86.89% reported that they chosen a personal improvement that was important or very important to them.  The rest, 13.11%, reported that they made a resolution about a topic that was somewhat or not at all important to them.
• We recommend to participants that they should display their progress publicly and share it with their improvement team.  Our expectation is that such displays will remind the participant to remain on task.  We asked the participants whether they publicly displayed their storyboard.  Of the 129 participants, 2 were not asked the question.  Among responders, 34.65% created a storyboard only at the end of their effort and therefore did not benefit from public display of the storyboard throughout their effort.   Among responders, 59.06% followed our recommendation and display their story publicly throughout their effort.
• Participants were asked to collect data and report how many data points were collected.  Of the 129 participants, 8 were not asked the question, did not answer or were not sure of their response.  Among responders, 19.49% collected 1-5 data points,  7.63% collected 6 to 10 data points,  13.56% collected 11 to 20 data points and 59.32% collected  more than 20 data points.  Participants were asked to analyze the data they had collected using control charts.  Control charts help distinguish between random variations and real improvements. Among responders, 111 analyzed their data using a control chart, 3 made scatter plots, 13 made a histogram of their data.
•  We encouraged participants to make systemic changes in their lives and rely less on their own motivation.  We asked the participants whether they relied on their motivation or on changes in their environment.  Out of 129 participants, 129 were asked and responded.  Among responders to the question,  61.4% followed our advice.  The remaining group relied on their motivation to stay the course and achieve their objectives.  
• We encourage participants to engage other process owners (people living with them who share an environment with them) in the effort.  Most participants are familiar with going through a resolution with someone else with similar objectives.  Our advice is to move away from getting help from a buddy or friend to getting help from people who live with the participant and share their environment.  Out of 129 participants, 4 were not asked, did not answer or were not sure of their response to the question.  Among the responders, 61.48% engaged process owners,  31.97% asked for no one's help, and 6.56% joined with another buddy with a similar resolution.
• We encourage participants to study their lifestyle using flow charts and lists.  Out of 129 participants, 82 made a flow chart of their lives,  85 made a list of factors that contribute to their habit,  and 24 made a Fish Diagram (a visual display of stratified list of causes),

In the real world, advice on self improvement is given but not always followed. It is important to evaluate whether participants were able to succeed even though they did not follow all of advice given.

Did it work? 

Of the 129 participants,  123 were asked how often they had been unsuccessful in changing their habit.  In the past, respondents had tried  2.96 times (Standard deviation = 3.29)  but failed to accomplish their resolution.  Against this backdrop of repeated failure, success would come as a surprise.  Nevertheless, using the new method, participants reported high success rates:

We asked from the 129 participants if they had made an improvement in their lives and accomplished their resolution.  5 did not answer or were not sure.  Among the responders, 4.13% did not measure their outcomes; 5.79% reported that they had not seen any improvement yet, 20.66% reported that they made an improvement but it was not significant or they did not check for its statistical significance;  69.42% reported they had made a measured and significant improvements in their lives.

We asked from the 129 participants if the advice we had given to them was helpful.  4 were not asked, did not answer or were not sure.  Among the responders, 82.79% said it was helpful or "a lot" helpful to them, the remainder (13.11%) said it was somewhat helpful or not helpful at all.    

We also assessed participants' intent to continue with the improvement effort past the 15-weeks study period.  Of the 129 participants, 11 did not answer the question, were not asked the question or were not sure about their response.  Among the responders, 98.33% planned to continue with their efforts.

One of the central themes in our proposed approach is that resolutions are more likely to be accomplished if one brings about changes in the environment as opposed to simply relying on motivation.  To address this question in more detail, we ask participants to describe the nature of the change they undertook .  If our claim is correct, then success must be more likely among people who bring about environmental changes.   Table 2 shows the relationship between nature of the intervention and success rates.

Data in Table 2 can be used to calculate likelihood of success for different types of interventions.  For example, the likelihood ratio of success for environmental intervention was measured by dividing proportion of environmental  interventions in successful cases by proportion of environmental interventions among failed resolutions.  A likelihood ratio higher than one shows that participants were more likely to succeed than fail; a ratio less than one shows the reverse.   Analysis of the data in Table 1 shows that participants, who made environmental changes, had a likelihood ratio of 1.5; they increased their odds of success by 1.5 times.  In contrast, participants, who relied on their own motivation, changed their odds of success by 0.58 times.  Changing the environment increased the odds of success by 2.59 times more than relying on one's motivation.  These data support our claim of the importance of ecological change in bringing about significant improvements. 

Discussion

The data we have presented has a number of limitations.  First, it is collected from graduate students.  Their experiences and skills may not generalize to the larger population.  The selection of graduate students to test our ideas may be reasonable because we do not recommend our approach to everyone.  To successfully implement this approach, the participants need skills in problem solving, data collection, and data analysis.  Therefore, our approach is recommended to groups in the population that are comparable in skills and training to the participating students.

The study is also limited because there was no control group and no random assignment.  Students who analyzed their data using control charts, act as their own controls.  But not all students used control charts and therefore lack of randomization and controls could lead to a number of problems.  The observed changes in behavior may be due to self selection.  For example, students more likely to change their behavior may have joined the classes (even though in many instances the class was a required course).  The observed improvements may be due to other causes besides our advice.  For example, during the course of the study years there have been increased emphasize on dieting and exercise in the media.  While these limitations are there, keep in mind that the course was taught in three different locations and over several years by different instructors.  The variety of the environments reduce the chance that findings are an artifact of something else.  Furthermore, keep in mind that participants were repeatedly making resolutions and failing before they started the course.  Hence, their success in achieving their resolutions is more meaningful when compared to their earlier failures.  .

The study is also limited because it is based on data collected by participants.  It is possible that individuals may alter their data in order to look good to their colleagues and even to themselves.  We tried to reduce this possibility by asking respondents to measure their success or failure on an objective criterion.  We also clearly emphasized that the student's grade did not depend on success.  Furthermore, we tried to reduce the possibility of reading too much into small improvements by asking participants to test whether the improvement they had observed was statistically significant.

Despite the limitations, the study established several facts.  First that participants did not follow all advice given.  Second, that despite variations in how the advice was adopted, participants reported high success rates.  Furthermore, success rates were higher for those participants who did follow our advice of relying on environmental changes.  

In our view, the success of the method is not due to any one component but the combination.  Certainly, when we emphasized system change, the success rate improved, but these may be an artifact of numerous other changes we simultaneously introduced.  The idea that environment can reinforce a person's motivation has been around for some time.  We have shown how these ideas can be used to accomplish personal resolutions. 

Our proposed method is not inconsistent with other methods, theories, or popular books about weight change and exercise.  For some, the process evaluated here could be a useful addition to their chosen method of weight loss (e.g. Atkins diet, Weight Watchers, Meditation, etc.) or exercise.   Obviously, success rate depends in part on the reasonableness of the resolution and the method of accomplishing it.  We can help people stay their course, but they, themselves, must select what they want to do. 

Further research is needed to confirm our preliminary findings.  In particular, research should be based on random assignment of participants to intervention and control groups.  It should more rigorously control for individual characteristics of participants and it should go beyond participants report of objective data.  Finally, our data do not compare the performance of our recommended actions to other methods of behavior change.  Additional comparative data are needed.  Until then, the data we have presented suggests that the proposed approach works well and helps increase the odds of success.