黑料网

Efficacy of Hyaluronidase Preceding Subcutaneous Opioid Infusions for the Treatment of Pain among Hospice Patients

¹Chapters Health System, Inc., Temple Terrace, Florida, USA

²Chapters Health Pharmacy, LLC, Tampa, Florida, USA

³LifePath Hospice Inc., Tampa, Florida, USA

⁴Tidewell Hospice, Sarasota, Florida, USA

⁵Legacy Meridian Park Medical Center, Tualatin, Oregon, USA

*Corresponding Author:

Dr. Ronald S. Schonwetter
Chapters Health System, Inc.,
12973 Telecom Parkway Suite 100
Temple Terrace, FL 33637, USA
Tel: (813) 871-8001
Fax: (813) 383-5044
E-mail: schonwetterr@chaptershealth.org

Received date December 07, 2011; Accepted date January 18, 2012; Published date January 26, 2012

Citation: Schonwetter RS, Kim S, Quinn MJ, Boehm D, Patel D, et al. (2012) Efficacy of Hyaluronidase Preceding Subcutaneous Opioid Infusions for the Treatment of Pain among Hospice Patients. J Palliative Care Med 2:105. doi:10.4172/2165-7386.1000105

Copyright: ©2012 Schonwetter RS, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Visit for more related articles at Journal of Palliative Care & Medicine

Abstract

Background: Morphine or hydromorphone are often administered by the subcutaneous (SC) route when oral and/or venous access is difficult to achieve or higher doses of opiates are needed. Human recombinant hyaluronidase is believed to increase the permeability of SC connective tissues by degrading hyaluronan and increasing the dispersion and absorption of coadministered molecules.

Objectives:To determine whether hyaluronidase coadministered with SC morphine or hydromorphone to hospice patients results in 1) faster reduction in pain level; 2) self-perceived pain relief, 3) improved pain-related distress, 4) reduction in the frequency of opiate bolus attempts utilized, and 5) more favorable outcomes on these measures over an eight-hour period.

Methods:This double blind, randomized, placebo-controlled study compared patients given SC hyaluronidase (n=25) and those receiving normal saline (n=29) both immediately prior to SC infusions with morphine or hydromorphone.

Results:Hyaluronidase was found to facilitate reduced pain level (p=0.059) and augmented pain relief (p=0.047) during the first 15 minutes following co-infusion of hyaluronidase when compared to patients in the control group who received usual pain control medication involving SC morphine or hydromorphone. In subsequent periods of observation up to eight hours, the intensity of pain level and the magnitude of pain relief realized showed no difference between the groups. There were no differences between the groups in pain-related distress (p≥0.657) and in the frequency of nfusion bolus attempts during the entire period under observation (p=0.173).

Conclusion: Hyaluronidase may be an effective adjunct during the first 15 minutes of drug administration in reducing pain intensity and enhancing pain relief when administered at the onset of a subcutaneous opioid infusion among hospice patients.

Keywords

Hyaluronidase; Pain level/relief; Subcutaneous infusion

Introduction

In the United States, 73% of all hospices use continuous subcutaneous (SC) infusions. Ninety-five percent of hospices use the SC routes for pharmacological management, 56% for poor intravenous (IV) access, 38% due to no oral intake, 8% for dehydration, 2% for infusion of saline, and 1% for nutrition [1]. The most common indication for SC was opioid infusion for pain control. The most frequently used medications were morphine (97%) and hydromorphone (60%). In hospice settings, SC infusions allow the administration of palliative medications in symptom crises when oral medications have not been effective or when IV access is difficult to achieve. Due to the frequent need to administer medications via a non-oral route in the hospice population, Mercadante [2] suggested that hospices that do not offer SC infusion might be missing an opportunity to optimize symptom control.

Recombinant hyaluronidase human injection (hyaluronidase), is a spreading or diffusing substance that temporarily increases the permeability of connective tissues and enhances dispersion and absorption of co-injected fluids and medications. The restoration of the dermal barrier removed by hyaluronidase is incomplete at 24 hours, but at 48 hours, the barrier is completely restored in all pretreated areas [3]. Concomitant SC administration of hyaluronidase has been reported to be safe and effective in enhancing the absorption rate of morphine compared to SC morphine with placebo [4]. Another study demonstrated that the mean time to reach maximum morphine plasma concentration was 13.8 minutes with a median time of 15 minutes when SC morphine was administered with placebo. When SC morphine was coadministered with 150 U of recombinant human hyaluronidase, known as rHuPH20, the mean time to reach the maximum plasma concentration was 9.2 minutes with a median of 5 minutes[5]. This study, however, is a study of pharmacokinetics rather than therapeutic response in terms of the pain relief actualized on the part of treated patients. Hyaluronidase was approved by the U.S. Food and Drug Administration (FDA, NDA21-859, 2005) to be used as a spreading agent to increase the absorption and dispersion of other injected drugs and for subcutaneous hydration. Hyaluronidase is supplied as a sterile, clear, colorless, non-preserved, ready-to-use injection.

According to Herndon and Fisk, continuous SC injection is a safe, effective method of medication administration and has several advantages over IV administration[1]. If hyaluronidase enhances the dispersion and absorption of SC opioids, it is reasonable to hypothesize that the preceded co-injection of hyaluronidase with these medications will shorten the time needed to realize pain control in hospice patients experiencing uncontrolled pain. Although there are studies indicating hyaluronidase could enhance the pharmacokinetics of SC morphine [4,5], there exists a paucity of data surrounding the clinical use of hyaluronidase co-injected with opioids in realizing pain reduction. Ultimately, patients remain the best judge of their pain levels, and any evaluative judgments on pain levels by means other than patient reports may lessen the accuracy of the pain levels achieved.

The overall aim of this study is to determine whether SC infusion of opioid medication preceded by hyaluronidase SC injection improves patients’ realization of pain reduction along several dimensions of pain perception when compared to patients given SC opioid medication without hyaluronidase. Specifically, the aims of this study are: To determine whether hyaluronidase coadministered with SC morphine or hydromorphone to hospice patients results in 1) faster reduction in pain level; 2) self-perceived pain relief; 3) improved pain-related distress; 4) reduction in the frequency of opiate bolus attempts utilized and 5) more favorable outcomes on these measures over an eight-hour period. The focus of this study is to test the efficacy of hyaluronidase in the context of standard hospice practice in pain management.

Methods

Setting and the patient profile

The study subjects were recruited from three inpatient facilities from two large hospices located in the State of Florida. In 2009, the two hospices served 11,066 patients and the daily census averaged 1,910 patients. The patient inclusion criteria included: 1) non-pregnant, nonlactating adults 18+ years; 2) ability to provide informed consent; 3) hospice patients whose Short Portable Mental Status Questionnaire (SPMSQ) [6] score is 6 or greater; 4) English-speaking; 5) pain not satisfactorily controlled with current medications-oral, topical or rectal; 6) pain level of 4 or greater at inpatient admission on a 0-10 numeric pain intensity scale using Modified Brief Pain Inventory (mBPI) [7]; 7) able to self-administer bolus dose or ask someone to hit bolus button; 8) estimated life expectancy of three days or more; and 9) patients appropriate for continuous SC infusion with either morphine or hydromorphone. The exclusion criteria were: 1) patients with a history of allergy or hypersensitivity to hyaluronidase or any components of the product; 2) patients on infusion therapy for pain management up to 14 days prior to entering the inpatient facilities; 3) patients who are actively dying identified by any of the following physical signs and symptoms: a) non-communicative or unresponsive, b) confusion/ disorientation about time, place and person, c) significant chest congestion with gurgling sounds, d) restless and repetitive motions, e) little or no food or fluid intake, f) minimal urine output, and g) different breathing pattern, i.e., shallow rapid breaths with periods of apnea. Informed consent was secured with consent procedures following the standard operating procedures of the hospices by signing an Informed Consent Form and the HIPAA-required Research Authorization Form before beginning the study.

Experimental design

The study design employed was a randomized, double-blind, placebo-controlled experiment. Hospice patients who initially met patient inclusion and exclusion criteria were consecutively asked to participate in the study. Once patients consented to participate in the study, they were randomly assigned to either an experimental or control group, using a table of random numbers. The patients in the treatment group received 1 ml of hyaluronidase, plus the usual SC infusion medication for pain control. The pain medication was administered as ordered by the attending physician. Patients were able to utilize as needed doses of their opioid analgesics as ordered by their attending physician. Subjects in the control group received 1ml of normal saline, equal volume to that of the hyaluronidase solution, followed by the usual SC infusion medication for pain management in an identical manner used for the patients in the treatment group. Each ml of hyaluronidase contained 150 USP units of recombinant human hyaluronidase. A 24” subcutaneous non-Di (2-ethylhexyl) phthalate infusion set with a 27Ga × ½” needle was used for the administration of the hyaluronidase or placebo, followed by the SC infusion administration. The priming volume of the set was approximately 0.2ml. Prior to insertion into the subcutaneous tissue, the set was primed with the study drug or placebo. The SC set was placed into the subcutaneous tissue and the hyaluronidase or placebo was manually injected through the administration set and into the subcutaneous tissue. Following the administration of the study drug or placebo, a 0.2ml loading dose of the opioid utilized was administered with the infusion pump to assure complete clearance of the study drug from the administration set. Historical data from the Hospira Gemstar patientcontrolled analgesic (PCA) pumps were downloaded and printed for inclusion in the study. The data were inclusive of loading dose volume, basal rate of doses in both the groups, bolus dose, lockout time, and bolus demands.

Outcome variables and measurement intervals

Four scales embedded in the Pain/Distress Assessment Instrument were used as the outcome variables for the study. Patient responses were collected and recorded by research nurses. The outcome variables were related to four dimensions of pain: 1) self-reported pain intensity level - What is your overall pain now - using an 11-point numeric rating scale (0=no pain to 10= pain as bad as you can imagine); 2) pain relief realized- At this time, how much relief are you getting from your pain? - using a 5-point scale (1=no relief at all; 2=a little bit of relief; 3=some relief; 4= quite a bit of relief; 5=complete relief); 3) painrelated distress- How much does your pain bother (distress) you now? using a 5-point scale (1=not at all; 2=a little bit; 3=somewhat; 4=quite a bit; 5=very much); and 4) the frequency count of bolus attempts made by the patient and as recorded by the PCA printer. The frequencies of bolus dose attempts in hourly intervals were used to generate infusion pump history. These data were registered by automated printers from start to finish, which allowed time-tagged infusion history of each patient. The outcome variables were measured eight times during the eight-hour period: just before the SC injection/infusion (baseline), 15 minutes from start, 30 minutes, 45 minutes, one hour, two hours, four hours and the eight hours after the initial placebo or hyaluronidase was given. Of the four dimensions of pain, the pain-level-right-now was considered as the primary outcome measure in assessing changes in pain level due its conceptual clarity reacted by the patient here and “right now” rather than constructs such as “relief” and “distress” that may require additional frame of reference or implied retrospective recollection on one’s perception of pain. Placebo saline or hyaluronidase was administered only once at the start of study.

Data reduction strategy

Demographic profiles included: age, gender, caregiver number at home, percent white, black and Hispanic. Pain-related profiles at t₀ were: pain now, acceptable pain level at baseline, intensity of pain distress and the degree to which the patients are getting pain relief at baseline. The dose-related medication profiles at baseline include: morphine-equivalent rate of basal rate of routine order (mg/hr). Since the objective of the baseline comparison was to detect any difference between the groups at baseline, two-tailed analysis of variance (ANOVA) tests were applied.

In order to reduce the effects of baseline differences that may have impacted the outcome variables included in the study, a decision was been made to apply statistical controls by employing analysis of covariance (ANCOVA) method [8]. The covariates selected were: gender, number of family caregivers at home, acceptable pain level, pain level now, intensity of pain-related distress all observed at baseline. The selected covariates were pre-existing patient profiles that were either assumed or known as having impacted the outcome but had random relationship with the independent variable, i.e., group status. It was hypothesized that patients in the treatment group will have lower pain levels, greater pain relief, lower distress and a smaller number of bolus attempts compared to those in the placebo group for any given interval and for the entire observation period. Since all outcome hypotheses were directional, the one-tailed ANCOVA tests were conducted with the Type I error set at the 0.05. Methods for using one-tailed tests in ANCOVA setting is available elsewhere [9].

Statistical power analysis

Seventy patients verbally volunteered to participate in the clinical trial. Of these, five withdrew from the study during the written consent procedure as shown in (Figure 1) of the remaining 65 subjects, 11 patients were declared ineligible during the screening process. The final sample included 54 subjects at baseline: 25 patients in the treatment and 29 patients in the control group, resulting in an overall attrition rate of 17% (=11/65). Using one-tailed ANCOVA test with five covariates (w=5), comparing two groups (u=2-1=1) while setting the effect size of hyaluronidase intervention at the medium level (f2=0.15) with a sample size of n=25 in each group, the noncentrality parameter (L) of ANCOVA test is estimated as 6.45 [10]. The resulting statistical power computed for the experimental design is 0.72, whereas the ANOVA yields statistical power at 0.76 due to the absence of covariates in ANOVA setting.

Figure 1: Randomized patient assignment process used for two arms of the trial.

Results

Sample

Table 1 shows the distributions of demographic profiles, disease types, pain-related scores, and dose-related medication, comparing patients in the treatment and control groups at baseline (i.e., at t0). As shown in the table, all fourteen F test scores indicated that there was no significant difference between the groups at baseline. All patients in the treatment group received morphine except two patients whose hydromorphone dose was converted to morphine-equivalent units using the standard conversion rate of 6.667 [11].

Table 1: Patient Profiles Observed at Baseline, comparing Patients in Treatment and Placebo Groups

Pain level now

Table 2 shows that the differences of pain scores realized at each interval by group from t₁ to t₇. During the first interval, the mean pain score reduction observed from treatment group was -1.48 while the mean pain score reduction observed from control group was -1.00. The ANCOVA test results indicated that there was a trend in the difference in pain scores (F=2.55, n=54; p=0.059). There were no subsequent differences in the pain reduction scores between the groups after the 15th minute. The last column indicates the sum of differences of pain scores realized for all intervals. The sum of pain reduction realized by the treatment group was larger than that realized by the control group (-3.76 vs. -3.31). However, the overall difference was not significant (p=0.321).

Table 2: Difference of Pain Score Realized at Each Interval from the Previous Observations by Group Status.

Pain relief realized

A significant difference in the magnitude of pain relief was found during the first 15-minutes comparing patients in the treatment and those in the control groups as shown in (Table 3) (p=0.047). During the interval, pain relief realized by the patients in the hyaluronidase group was greater than that observed from the control group by 0.388 units (=0.560-0.172) along the 5-point scale. This observation was consistent with the mean pain score reduction previously observed from the hyaluronidase group during the first 15 minutes shown in (Table 2). There were no differences in pain relief realized after this time period. The last column indicates the sum of differences of pain relief realized for all intervals. The sum of pain relief realized by the treatment group over the entire interval showed a trend in greater pain relief in the treatment group (1.667 vs. 1.000; p=0.083).

Table 3: Difference of Pain Relief Realized at Each Interval from the Previous Observations by Group Status.

Distress level

For each of the seven intervals, there were no differences in distress levels noted between the two groups as shown in Table 4. The last column indicates sum of differences of distress levels experienced for all seven intervals. The sum of distress realized by the treatment group was larger than that realized by the control group (i.e., 1.667 vs. 1.483). However, the overall difference was not significant (p=0.657).

Table 4: Difference of Distress Level Observed at Each Interval from the Previous Observations by Group Status.

Bolus attempts made

It is interesting to note that the frequency of bolus attempts made by patients in the hyaluronidase group was lower for all but one of the time periods examined as shown in (Table 5). For the total 8 hour period, the mean of all bolus attempts made by the control group was approximately 31 compared to the mean of approximately 23 attempts made by the patients in hyaluronidase group. However, none of the differences in bolus attempts made were found to be significant at the 0.05 level.

Table 5: Frequency of Infusion Bolus Pump Attempts Registered at Each Interval by Group Status.

Adverse events

In this study and as shown in (Table 6), patients receiving hyaluronidase had a minimal number of reported adverse events and none was statistically different from those who received placebo. None of these adverse events required discontinuation of the test article. One patient had redness at the hyaluronidase infusion site, which may have been secondary to sensitivity to the hyaluronidase medication or to other etiologies, but this did not require additional intervention. Based on the instructions given to the research nurses to collect all adverse events during the 8-hour trial period, the following events and their frequencies were observed from the hyaluronidase group: four patients (16.0%) with agitation, three (12.0%) with anxiety, and one (4.0%) for each of the remaining events: anemia, insomnia, lethargy, nausea, respiratory infection, and dehydration. These adverse events were all minor in severity and they were equally distributed between the groups. A similar generalization concerning the distribution of adverse events involving hyaluronidase and a control group was reported elsewhere [5]. It is also possible that these adverse events were from disease progression, other medications or other causes.

Table 6: Incidence of Adverse Events Observed by Group Status^a.

Discussion

Hyaluronidase may be an effective adjunct in reducing pain level (p=0.059) and relieving pain perception (p=0.047), when given prior to initiating subcutaneous opioid infusions during the first 15 minutes among hospice patients. However, hyaluronidase had an insignificant impact in relieving distress (p=0.451) during the same time period. This may be attributable to the fact that the word “distress” is more subtle and diffuse, perhaps with multiple latent constructs rather than a single construct measured as in the case of “pain score right now” or “pain relief you are getting right now”. In addition, the concept of “distress” and “pain-related distress” may have been co-mingled with emotional distress, such as depression and anxiety, that are often found among many end-of-life patients, where patients with emotional suffering among cancer patients are found to have higher level overall pain than those without these symptoms [12,13]. In subsequent intervals of pain observation after the first fifteen minutes, the pain level and the painrelief realized was not affected by hyaluronidase. There was a trend in the total eight-hour period for greater pain relief in the treatment group (p=0.083). While there was no statistically significant difference of bolus attempts made with the use of hyaluronidase, the total count of bolus attempts by the control group was higher than those of the patients in the hyaluronidase group (31.28 vs. 22.63) but insignificant (p=0.173).

One may question whether the reduction in pain score observed by a difference of 0.48 between the groups during the first 15 minutes with a p value of 0.059 could lead to an interpretation that hyaluronidase may be effective. Almost all Type I p-values can be found statistically significant if the sample size was very large, statistically speaking. But, it must be noted that this study was under-powered, making it difficult to reject the null hypothesis of no difference between the groups. Thus, the study was based on a rather conservative approach to generating a new finding, realizing that p-values are to be used as a guideline for making individual decisions based on risks and benefits and regrets associated with a decision. At this point, one may further ask whether the clinical findings observed during the first 15 minutes of the clinical trial are clinically meaningful. Some may argue that the small differences of pain severity scores found from our two-arm study may be statistically significant but nevertheless clinically trivial [14-18]. Related to this issue, one earlier study eluded that the minimal clinically significant difference in pain severity is 13 mm when measured with a 100 mm visual analog scale among emergency room patients with acute pain resulting from trauma [14]. Others noted pain reduction of 6.2 mm among patients with rheumatoid arthritis [17], while another study using the same numeric rating scale indicated 23 mm on a 100 mm visual analog scale as the minimal difference associated with detectable changes in patient satisfaction among patients under emergency department care [19]. These reports and variations of this type of findings, however, are limited and have inherent problems, and they have little to contribute to the advancement of science in the measurement of clinically significant reduction of pain [20]. These problems occur as the estimated magnitude varies depending on the score distribution, external standard one may use, how the scale values are assigned to anchor points, the direction of change/difference, and where you locate the baseline value.

An alternative strategy which overcomes these problems is to determine the change in scores between two points in time or difference of means from two or more groups and then evaluate the magnitude of difference or change in the context of the overall dispersion of scores found within a population under investigation, where the score dispersion is approximated by sample standard deviation. In this study, this is accomplished by determining whether the difference obtained is greater than 1/3 of a standard deviation of the pooled sampled population, given the sample size is adequate, e.g., n≈30 or larger. Under the standard normal curve, 1/3 of one standard deviation is translated as a z-score of 0.033. A difference in means between two groups compared, which is greater than 1/3 of one z-score may then be treated as clinically significant or a meaningful difference. In this study, the difference score on pain-relief during the first 15 minutes is 0.388 (Table 2) and the associated standard deviation of pain-relief score for all patients is 0.97 (Table 1). Accordingly, the critical ratio computed is 0.40 (=0.388/0.97). It is greater than the 1/3 of one standard deviation. Therefore, the amount of pain-relief observed may be viewed as clinically significant. In the case of mean difference score on painright- now during the first 15 minutes, the amount of pain reduction observed is -0.48 (Table 2) and the associated standard deviation for all patients is 1.48 (Table 1). Again, applying the 1/3-of-one-standarddeviation criterion, the ratio computed is 0.32 (=0.48/1.48), which is 0.01 point shy of 1/3 of one standard deviation within rounding errors. The evidences originating from the differences in pain-reduction and pain-relief are consistent in terms of their directions and they seem to mutually reinforce one another. Accordingly, it may be generalized that the changes observed in pain-relief and pain-reduction during the first fifteen minutes are clinically meaningful. These findings are consistent with the pharmacokinetic properties of hyaluronidase [5].

One may raise a question why the data reduction strategy employed interval-based difference score (i.e., difference of post from pre measurement for each interval) instead of using a scale score on a point in time, i.e., point estimate. This is based on the fact that scale value measured on point estimate is, in general, less reliable than changed score observed within a time interval [21]. The fact that the test article was observed as having almost no beneficial effect on pain perception after the first fifteen minutes compared to patients in the placebo group for the same time periods attests to the fact that the usual method of delivering opioid subcutaneously without the use of test article was able to achieve reduction in pain levels comparable to the pain levels felt among patients in the treatment group after the fifteenth minute. One may not have expected ongoing efficacy of hyaluronidase in such a study design as patients were able to utilize as needed rescue doses to manage their pain effectively, thus limiting the chance of demonstrating further efficacy of hyaluronidase.

One of the important limitations of this study resides in the fact that the sample size drawn was relatively small, resulting in a statistical power of 0.72. The latter was somewhat smaller than the conventional statistical power usually set at 0.80. This means that the final sample size employed was small, leading the investigators to reach more conservative findings. This study also employed a mixed sample of both cancer and non-cancer patients. Due to small sample size, the study design did not allow an elaboration of the impact of different diagnoses on the perception of pain scores and the pain relief realized by the treatment status. Another limitation of this study is that patients with different diagnoses and different types of pain may have varying responses to treatment. Furthermore, the sample was drawn from two hospices and, accordingly, this may limit the generalizability to other patient populations and to clinical settings. This study also did not evaluate the cost of utilizing subcutaneous hyaluronidase. In addition, determining the optimal time at which improved pain relief occurred during the first fifteen minutes of medication administration would have added to the study. However, the timing of improved pain level and reduction in pain did correspond with the prior pharmacokinetic studies conducted involving hyaluronidase [4,5]. Lastly, while hydromorphone may be slightly more lipophilic than morphine resulting in faster onset of pain relief, the number of cases utilizing hydromorphone was so small that it could not have influenced the results of the study.

It is widely shared among clinicians that patients are the best judges of their pain, and the clinicians practicing hospice and palliative medicine rely primarily on patient symptom response data. The main strength of this study is the evaluation of the efficacy of hyaluronidase for pain management comparing patients provided SC hyaluronidase plus the standard SC opioid medication to those receiving standard SC pain management protocols practiced at a hospice without the benefit of hyaluronidase in a natural clinical setting. It appears that hyaluronidase may be beneficial to hospice patients requiring acute pain control in reducing pain intensity and enhancing self-perceived pain relief in the first fifteen minutes following its administration with a SC opiate analgesic.

Acknowledgements

The authors would like to thank the hospice patients who participated in the study at a difficult time in their lives.

Author Disclosure Statement

This investigator-initiated study was supported by Baxter Healthcare Corporation. The study design, data collection, analysis, and interpretations of the findings are conducted entirely by the authors in this study and the sponsors of this study did not have any role in these activities. The authors have no financial relationships with the study sponsor and no competing financial interests exist.

References

1. Herndon CM, Fike DS (2001)
2. Mercadante SG (1998)
3. Baxter Pharmaceutical Solutions (2006) HYLENEX Recombinant: Clinical Pharmacology, Bloomington, IN, March 2006: 1-7.
4. Thomas JR, Wallace MS, Yocum RC, Vaughn DE, Haller MF, et al. (2009)
5. Thomas JR, Yocum RC, Haller MF, von Gunten CF (2007)
6. Pfeiffer E (1975)
7. Mendoza T, Mayne T, Rublee D, Cleeland C (2006)
8. SPSS V18.0 (2002)
9. Knapp TR, Schafer WD (2009)
10. Cohen J (1977) Statistical Power Analysis for the Behavioral Sciences. (2ndedn), Academic Press New York, USA.
11. Hallenbeck JL (2003)
12. Utne I, Miaskowski C, Bjordal K, Paul SM, Rustoen T (2010)
13. Taylor CR, LeMone P, Lillis C Lynn P (2008) Fundamental of Nursing: The Art of and Science of Nursing Care. (6th edn), Wolters Kluwer, Philadelphia, USA.
14. Sackett DL, Hanes RB, Guyatt GH, Tugwell P (1991)
15. Todd KH, Funk KG, Funk JP, Bonacci R (1996)
16. Jaeschke R, Singer J, Guyatt GH (1989)
17. Wells GA, Tugwell P, Kraag GR, Baker PR, Groh J, et al. (1993)
18. Redelmeier DA, Lorig K (1993)
19. Wright JM, Price SD, Watson WA (1994)
20. Hays RD, Woolley JM (2000)
21. Anderson S, Auquier A, Hauck WW, Oakes D, Vandaele W, et al. (1980)