How satisfaction surveys should be organized  Slides►  Listen► Paper► Analysis of complaints  More►   Recently asked:  "do we need to have feedback from our sponsor given in the final report of the Survey Proj.? or just feedback from the peer evaluations? " See answer to this and other questions.

Satisfaction Survey Planning Project

This section describes the Satisfaction Survey Planning Project in the course on Process Improvement. 

Objectives

At the conclusion of this project participants should be able to:

• Apply quality improvement tools and techniques.
• Plan efforts to assess patients' satisfaction or health status.
• Construct a quality improvement storyboard.

The purpose of this project is to help you design and put in place satisfaction surveys.  The purpose is to plan for the surveys but not to do them. 

Steps by Step Guide to Project Completion

Here are the steps you can take to accomplish this project:

1. Team up with a classmate that is working within an organization and can obtain permission from the organization for you to plan a satisfaction survey to address their needs. 

2. Obtain a letter of introduction from the instructor and send it to the sponsor.  Download►

3. Read about how satisfaction surveys are done and review the literature.  Review►  Questionnaires►  Response rate►  H-CAPH► Abstracts►  Full text►  Library►  Google►  

4. Make an initial plan
    

   • Decide on the survey tool you will use.  We have developed a tool for active solicitation of complaints and praise which we call “Minute Evaluation” to emphasize the short nature of the form.  Short surveys are expected to increase the client's response rate and reduce cost of conducting satisfaction surveys.  The response rate to current satisfaction surveys is relatively low.  It takes repeated call backs and other reminders to improve the response rate.  Naturally, the cost of conducting satisfaction surveys is high.  One way of improving response rate is to make it easier for people to respond.  The following form reduces the number of questions asked from the patients to a bare minimum.
     
     
     
     Note that patients respond to two questions.  The first question asks the patient to rate their experience.  The second question provides the explanation for the rating.  In addition, the provider enters the information on visit code and location code prior to providing the postcard to the patient.  The question on the visit code establishes the sequence of the data collected and is important in analysis of time (number of visits) to dissatisfied customer.  The question on location relates the rating to a specific unit within the health care institution and allows the provider to focus on evaluation of specific units.  The Minute Evaluation of Satisfaction is typically printed on a postcard.  Preferably, patients should drop their response in a box before leaving the organization.  Some, may wish to mail the postcard at a later time.  The postcard contains no personal information and can be mailed without cover. 

     Sometimes, when organizations want to understand more details about the reason behind the patient's complaint or praise, additional survey tools are used to gather more details.  Patients who rate their experience in the two extremes may be contacted to complete a more detailed survey tool.  In this manner, most patients are spared the burden of completing long and detailed survey tools.  Patients find the Minute Evaluation of Satisfaction more convenient than other longer tools, response rate improves and cost of collecting patient's satisfaction with care is reduced.
      

   • Decide on the frequency of the survey.  Discuss sampling strategies (convenient sample, sampling the next n patients, two stage sampling, inverse sampling or sampling until two dissatisfied patients are found, adaptive sample or sampling more intensively when a dissatisfied patient is found.  More► 
      

   • Decide on who will do what, when and how.  Decide survey procedures (e.g. list of patients, sampling steps, cover note, consent form, questionnaire distribution, collection of questionnaire, data entry, timing of data analysis, timing of reports, recipients of reports.)   Consider various options for who will hand out the survey (mail, receptionist at start, receptionist exit, etc.)  Give details and discuss these details with the organizations.  Estimate resources needed to carryout the survey.  Estimate the start and end times.  More►
      

   • Do not collect data.  You are expected to only plan for data collection and not actually collect the data.  Use the data provided by the instructor (or make up your own data) to demonstrate your project plans.  Analyze the data and show how the findings will be reported.  In your presentation make sure it is clear that the tables and graphs are not based on real data.   Download►

   
    

   • Describe who will use the data and why.  Specify the improvement team that is waiting for the data or plan the selection and maintenance of the team.

5. Engage members of the organization in your plans and get their reactions. 

   • Prepare a narrated slide show that provides an over view of your plans.  The intent is to prepare a short narrative that is effective in selling your plan to others within the organization (use customer's voice, use vivid images or videos, use short slogans, give the advantages of the plan, why it is easy, why it is effective, etc.).  See guide on how to post narrated videos to the web.  Video►  SWF►  You Tube► 

6. Prepare a final report using the following headings:

   • Executive summary:  A one page summary of the entire plan

   • Time to dissatisfied customer:  Review the literature on satisfaction surveys (at least 2-3 pages) and report the low rate of response to satisfaction surveys.  Discuss what is known to improve response rate and the importance of examining time to dissatisfied customers. 

   • Minute survey:  Describe different questionnaires available for measuring satisfaction of patients.  Contrast these with the minute evaluation form.

   • Survey timing:  Describe how often patients will be surveyed and why.  Describe the two stage sampling plan and why detailed information is not collected in the first stage.

   • Plan details:  Describe who, what, where, when the survey will be done

   • Pilot test:  Using a sample data, show how the data will be analyzed.   You can obtain sample data from the instructor.  Analyze the data and show the results that could be obtained from the data collection.

   • Plan acceptance:  Describe reaction of people who have reviewed the plan or participated in the planning process

The following resources may be of use: 

1. Listen to the instructor's view on how satisfaction surveys should be organized.  Slides►  Listen►
2. Read the paper on analysis of complaints.  Download►   More►
3. Read paper on advantages of Minute Survey of satisfaction.  More►

Examples

Here are some examples of the work done by a student.  Please note these examples are provided without judgment of quality of the work.  Furthermore, note that the instructions for the project have changed since these assignments were completed.   Example 1►  Example 2►, Example 3►and Example 4►

The following are two examples of narrated slides prepared by students about their plans:

Here is a comic example of media students put together to present their work:

Here is another narrated example on use of Minute Survey.  Note that in this example the use of Time-between Control Chart is not appropriate, as multiple surveys are available after each class.  You Tube 3►   
 

Peer Evaluation

We ask students to evaluate each other’s work because they often have fresh perspectives that are absent when only instructor provides feedback.  In addition, when students teach an idea, they learn it in much more detail.  The exercise ends up helping the evaluator more than the person being evaluated.   When providing peer evaluation of group projects, students can use the following rubric to guide their evaluation: 

1)       Date:  The document should include the date the draft was received and the date evaluation was sent back.

2)       Praise:  Student evaluators should start with what worked well.  They should use many adjectives and be clear about what was done well.

3)       Presentation:  Comment on effectiveness of the presentation.  Students will do their colleagues a service by setting a very high standard for effective presentation as that is what the instructor expects.  There is no specific prescription but asking the following questions may help:

• Is it too much text and too little visual guides?  

• Is it written poorly? 

• Are pages numbered and titled clearly? 

• Is the document fun?  Are there any cartoons to break the boredom? 

• Are there images to help make the points more succinctly? 

        No one is required to do any of these elements but these are given as examples of thoughtful presentations.

4)       Improvement:  Student evaluators should describe what needs more improvement within each section (literature review, survey selection, survey timing, plan details, pilot test and plan acceptance).  They should make at least one comment in each section.  They should not use any adjectives in their comments.  They should give details of the improvements and avoid general statements.  They should be honest and clear.

5)       Completeness:  Student evaluators should comment on whether all sections of the report were present, including sections waiting for input or data: 

a)       Executive summary (all reports should have this or have a placeholder for it)

b)       Literature review  (there should be a reasonable review attached)

c)       Survey (there should be several different surveys discussed and contrasted)

d)       Survey timing (two stage sampling or other sampling strategies should be discussed)

e)       Plan details (it should be clear who is doing what and when)

f)        Pilot test. (some data should be analyzed and presented or plans to do this should be clear)

g)       Plan acceptance (plans to solicit and report acceptance should be described)

6)       Student evaluators should finish by describing in detail what they have learned from the review of the draft project.  They should give details of how they plan to change their own project as a consequence of what they have learned.

Students should not provide a grade.  Email the evaluation to all members of the group as well as to the instructor.  Please send one email copied to the relevant people.

Recently Asked Questions

Ask your question and we will post your answer here.  Alternatively read answers to questions asked by others. 

Question: do we need to have feedback from our sponsor given in the final report of the Survey Proj.? or just feedback from the peer evaluations?  Answer: Feedback is needed from your sponsor and you can take extra time to obtain it if you have not yet done so.  This question was asked on 4/16/2008 3:37:56 PM and answered on 4/16/2008 8:01:37 PM.

Question: If we have not already receieved feedback from the satisfaction survey when should the groups be expecting to hear back from you or Bathsheba?  Answer: You are supposed to hear from your colleagues and not from us at this point. If you have not received your peer evaluation then please contact the instructor otherwise continue implementing the recommended changes. If you need more feedback, please set a time to talk with the instructor.   This question was asked on 3/24/2008 11:14:47 PM and answered on 3/24/2008 11:16:58 PM.

Question: After sending and receiving comments on the survey project, there appears to be two different versions of the rubric. There is one on the Satisfaction Survey Planning Project page and one linked to the syllabus for 26 Feb. Which is the correct version??  Answer: Either one is ok but I will examine the duplicate information and remove one  This question was asked on 3/12/2008 8:09:35 PM and answered on 3/12/2008 8:27:54 PM.

Question: So what do you label the "not sure" response as? "Yes= Satisfied" or "N0=Dissatified?"  Answer: It depends on what you are analyzing. If you are analyzing people who said that they were satisfied, then not sure is put together with the remaining group. If you are analyzing people who said that they were dissatisfied, then no sure is put with the remaining categories which includes satisfied people.   This question was asked on 3/1/2008 3:11:35 PM and answered on 3/1/2008 5:32:19 PM.

Question: What are the expectations for the number of pages total for the report?   Answer: There are no minimums or maximums.   This question was asked on 2/27/2008 8:19:55 PM and answered on 2/27/2008 8:32:12 PM.

Question: When reviewing the suggested format for the project report, I notice that you use end notes instead of the APA format. Is this the general rule of thumb for citation format in this paper? What is your suggestion?  Answer: Use the format that you think is appropriate. I use end notes because I copy the article citation from PubMed. You should do the method that is easiest for you but stay consistent.   This question was asked on 2/25/2008 8:48:22 PM and answered on 2/25/2008 11:41:53 PM.

Question: I understand that inverse sampling involves sampling until a predetermined # of instances of a particular event occur. Is there a particular why you suggest 2, as the predetermined # of events?  Answer: Two is the minimum and therefore if we are interested in reducing the cost, it may be the minimum we need to succeed.   This question was asked on 2/24/2008 8:47:31 PM and answered on 2/24/2008 9:39:02 PM.

Question: For the literature review of different types of surveys, are we supposed to include health care surveys in general, ,or make it specific to the type of organization we are using for our survey? For example, we are using an organization that has a application/registration process for doctors who want to participate in cancer research and clinical trials. So we want to ask their satisfaction w/the application/registration process. I have found a few surveys out there for satisfaction w/the application process. Should I just include those? Or also include CAHps and other health care surveys that measure patient satisfaction w/care received?  Answer: The idea is to talk about possible options for your organization and show that what you have selected makes sense.   This question was asked on 2/23/2008 1:52:44 PM and answered on 2/23/2008 2:12:04 PM.

Question: Is the data we need to analyze located in the download link that is provided on this page or will you be sending us sample data?  Answer: Yes that is the right data.   This question was asked on 2/23/2008 10:37:34 AM and answered on 2/23/2008 2:10:54 PM.

Question: In the 2 minute survey, how do you statistically incorporate the response of the second question into your data chart, graph?  Answer: You do not. The second question is for helping improvement teams understand reasons for dissatisfaction. The first question is to measure rate of dissatisfaction.   This question was asked on 2/23/2008 10:33:50 AM and answered on 2/23/2008 2:10:12 PM.

This page is part of the course on Quality / Process Improvement, the section on "About the course."  It was first created in 1996.  It was last edited on 08/29/2007 by Farrokh Alemi, Ph.D.  © Copyright protected.