Testing a Computerised Systematic Observation Tool in the Clinical Setting
Testing a Computerised Systematic Observation Tool in the Clinical Setting
Fifty two observations of intrapartum care were observed during the four month data collection period. One hundred and eleven hours of direct intrapartum observation were undertaken. The average length of each observation was 127.7 minutes (range 45.8 minutes – 318 minutes). Data were lost in three observations due to user error. Full data were recorded and analysed for forty nine observations, 104.3 hours of observation. Observations were often shorter than the planned three hour period as women progressed quickly through labour and to the birth of their baby in one third of the observations. A summary is provided in Table 3 .
Verbal feedback was sought from all of the observers about their experience of being involved in the study and any problems they encountered in carrying out the direct observations in the clinical setting. All three of the volunteer observers reported feeling very positive about their involvement in the study, being surprised with the ease with which they were able to gain consent from participants and how accepted and welcomed they felt by staff.
There were no problems experienced during the data collection period in gaining access to the ward areas or to midwifery staff to explain the study and to ask them to discuss participation with women. The midwives were generally enthusiastic and positive about the study and interested in the outcomes. Following the end of the data collection period, the local collaborators at each of the sites were asked to talk informally to staff about the study and identify any problems or concerns that staff had not felt able to share with the research team. All of the feedback given to the local collaborators was positive and no concerns or problems were raised.
Data were not routinely collected of numbers and reasons for women, birth partners and midwives not wishing to participate. Verbal and written information about the study was given to all midwives providing labour care to eligible women on a shift when an observer was present. Information was provided and consent sought from the woman and birth partner either by the midwife caring for her in labour, the midwife caring for her in the triage unit or the coordinator of the labour ward. Once consent had been provided by a woman and her birth partner to participate in the study, the researcher undertook the consent procedure with the midwife caring for her.
The midwife and woman participants responded positively to a postnatal question about the presence of an observer during the labour. Twenty nine midwives (64.4%) agreed with a statement that they felt 'fine, enjoyed it', with sixteen (35.6%) stating that the experience of being observed was 'OK'. No midwives chose the negative options of 'distressed, very or mildly uncomfortable'. Forty four of the midwives stated that they would participate again. Thirty nine women (88.6%) felt that having an observer was 'fine' and 11.4% chose 'OK'. All of the women involved in the study said they would be happy to participate again in a similar study.
The responses in the postnatal data collection sheet demonstrated that the SMILI had content validity with only a few problems and gaps identified in the earlier observations. In 28 observations (57.2%) the observer felt that the SMILI enabled them to record the midwifery support they observed 'fully' and in 19 observations (38.8%) 'very well'. In two observations the observers stated that the SMILI was inadequate, these were occasions when the programme temporarily crashed. Additions were made as a result of the comments and no further problems were identified after these changes were made. Problems with the programme were rare during the 111 hours of observation and overall the programme proved very reliable and usable in the clinical setting.
Internal reliability was found to be acceptable to good, with the exception of the negative variables of the woman and partner which show a weaker correlation. The results are summarised below in Table 4 .
Most midwives (92%) were in the room for more than 80% of the observation, with around one quarter of midwives present for 98% of the observation.
The overall results for the forty nine observations were:
The quantity of supportive behaviours varied considerably between the midwives observed. The most frequently observed category of support was emotional support, with a study mean of 395.5% (that is an average of approximately four emotionally supportive behaviours displayed at each observation point across the study as a whole). The lowest frequency that emotional support was observed was 98.9% in one observation (an average of just below one emotionally supportive behaviour at each observation point) and the highest frequency demonstrated was 629.7% (that is more than six emotionally supportive behaviours observed at each observation point). The second most frequently observed category was informational support, followed by tangible support and then partner support. These results are summarised in Table 5 .
Both emotional and informational support were relatively normally distributed. Advocacy, tangible and partner support were not distributed normally and were skewed in frequency to the less frequent.
The quantities of neutral/professional and negative behaviours also varied significantly between the midwives in the study. The frequency data for individual midwives revealed some patterns of behaviour. There were nine midwives who had below and two above study average frequency of all of the neutral and negative behaviours. In a similar manner to positive support behaviours, the majority of midwives (n = 37, 75.5%) showed a mixture of behaviours, displaying some neutral or negative behaviours. The quantities of neutral/professional and negative behaviours by the midwives are further summarised in Table 6 . Overall, negative behaviours were seen infrequently.
These results not only relate to the quantity of supportive and non-supportive behaviours observed and recorded but also contribute to the analysis of the quality of the support observed.
This was calculated using the Spearman's Rho correlation coefficient test.
The postnatal 'SCIB' questionnaire was completed fully for 42 of the 49 observations (86%) Where a woman was observed being cared for by two different midwives she was asked to complete two SCIB questionnaires. Women generally reported feeling very well supported in the questionnaire. The mean score for the study was 4.6 out of a possible total of 5.
The observers' global assessments of the quality and quantity of care observed were also generally skewed to the positive ( Table 7 ).
Spearman's Rho correlation coefficients were calculated between rates of negative behaviours by the midwife recorded in the SMILI and the assessments of support recorded by the woman in the SCIB and the observer in the global ratings questionnaire. Negative behaviours and inattentiveness by the midwife showed significant moderate inverse correlation with the assessment of the midwifery support by women and observers ( Table 8 ).
One of the key elements of support derived from the literature is the importance of the continual physical presence of the midwife to woman's feelings of being supported. The results of the analysis to test the association between these elements are given below. This analysis sought to test whether the quantity of presence or attendance by the midwife is an element in the assessment of the quality of the support. These results (presented in Table 8 below) show a significant strong inverse correlation between the amount of time that the midwife spent out of the room and the woman and observer's assessments of support. The higher the proportion of the observation that the midwife was out of the room, the lower the assessment of the support offered.
Correlation coefficients were calculated between positive midwifery support behaviours and the assessments of support by the woman and observer, in order to test whether the quantity of positive midwifery support behaviours is a key element in the measurement of the quality of midwifery support. These results show a significant moderate correlation between women's assessment of the support they received and the overall measurement of emotional support and support of the partner ( Table 9 ). Analysis of the data for the sub-categories of emotional support found that the most significant element of emotional support for women appeared to be rapport building (Spearman's Rho correlation with SCIB score .432**, p = .002).
The main method to test the construct validity of the instrument is to identify whether women's views of the support correlate significantly with the data collected using the SMILI. These links have been clearly identified in the previous section, with significant correlations between women's views and the quantity of positive and negative behaviours recorded.
However, women's views of the support they received may not only be influenced by the supportive and non-supportive behaviours recorded using the SMILI, but may be also significantly influenced by other elements of the experience. For example, a woman may describe the support she received less positively if she has had a more difficult labour and birth experience in terms of more medical interventions. These other possible influences and their impact on women's and observers' views of the support provided were examined.
This analysis found that there were no correlations between women's views of the support they received and their parity, allocated care pathway, analgesia used, number of medical interventions, type of birth, amount of non-support care and assessment activities, maternity unit and number of years the midwife had been qualified. This may be seen as further evidence to support the construct validity of the SMILI.
Results
Fifty two observations of intrapartum care were observed during the four month data collection period. One hundred and eleven hours of direct intrapartum observation were undertaken. The average length of each observation was 127.7 minutes (range 45.8 minutes – 318 minutes). Data were lost in three observations due to user error. Full data were recorded and analysed for forty nine observations, 104.3 hours of observation. Observations were often shorter than the planned three hour period as women progressed quickly through labour and to the birth of their baby in one third of the observations. A summary is provided in Table 3 .
Feasibility and Acceptability Data
Verbal feedback was sought from all of the observers about their experience of being involved in the study and any problems they encountered in carrying out the direct observations in the clinical setting. All three of the volunteer observers reported feeling very positive about their involvement in the study, being surprised with the ease with which they were able to gain consent from participants and how accepted and welcomed they felt by staff.
There were no problems experienced during the data collection period in gaining access to the ward areas or to midwifery staff to explain the study and to ask them to discuss participation with women. The midwives were generally enthusiastic and positive about the study and interested in the outcomes. Following the end of the data collection period, the local collaborators at each of the sites were asked to talk informally to staff about the study and identify any problems or concerns that staff had not felt able to share with the research team. All of the feedback given to the local collaborators was positive and no concerns or problems were raised.
Data were not routinely collected of numbers and reasons for women, birth partners and midwives not wishing to participate. Verbal and written information about the study was given to all midwives providing labour care to eligible women on a shift when an observer was present. Information was provided and consent sought from the woman and birth partner either by the midwife caring for her in labour, the midwife caring for her in the triage unit or the coordinator of the labour ward. Once consent had been provided by a woman and her birth partner to participate in the study, the researcher undertook the consent procedure with the midwife caring for her.
The midwife and woman participants responded positively to a postnatal question about the presence of an observer during the labour. Twenty nine midwives (64.4%) agreed with a statement that they felt 'fine, enjoyed it', with sixteen (35.6%) stating that the experience of being observed was 'OK'. No midwives chose the negative options of 'distressed, very or mildly uncomfortable'. Forty four of the midwives stated that they would participate again. Thirty nine women (88.6%) felt that having an observer was 'fine' and 11.4% chose 'OK'. All of the women involved in the study said they would be happy to participate again in a similar study.
Face and Content Validity and Usability of the SMILI in the Clinical Setting
The responses in the postnatal data collection sheet demonstrated that the SMILI had content validity with only a few problems and gaps identified in the earlier observations. In 28 observations (57.2%) the observer felt that the SMILI enabled them to record the midwifery support they observed 'fully' and in 19 observations (38.8%) 'very well'. In two observations the observers stated that the SMILI was inadequate, these were occasions when the programme temporarily crashed. Additions were made as a result of the comments and no further problems were identified after these changes were made. Problems with the programme were rare during the 111 hours of observation and overall the programme proved very reliable and usable in the clinical setting.
Internal Reliability of the SMILI
Internal reliability was found to be acceptable to good, with the exception of the negative variables of the woman and partner which show a weaker correlation. The results are summarised below in Table 4 .
Content and Construct Validity Testing: Using the SMILI to Measure the Quantity of Support
Most midwives (92%) were in the room for more than 80% of the observation, with around one quarter of midwives present for 98% of the observation.
The overall results for the forty nine observations were:
The total observation time with complete data was 104.3 hours.
The mean length of each observation was 127.7 minutes. Observations ranged from 45.8 minutes to 318 minutes in length.
The mean length of time that a midwife was out of the room excluding breaks was 11.5 minutes, which is 9.3% of the observation time. This ranged from 0% to 33.8% of the observation.
The average number of times that the midwife left the room was six.
Midwives left the room on average every 25.7 minutes, with a range from every 9.6 minutes to 165 minutes without leaving the room.
The Quantity of Positive Support Behaviours
The quantity of supportive behaviours varied considerably between the midwives observed. The most frequently observed category of support was emotional support, with a study mean of 395.5% (that is an average of approximately four emotionally supportive behaviours displayed at each observation point across the study as a whole). The lowest frequency that emotional support was observed was 98.9% in one observation (an average of just below one emotionally supportive behaviour at each observation point) and the highest frequency demonstrated was 629.7% (that is more than six emotionally supportive behaviours observed at each observation point). The second most frequently observed category was informational support, followed by tangible support and then partner support. These results are summarised in Table 5 .
Both emotional and informational support were relatively normally distributed. Advocacy, tangible and partner support were not distributed normally and were skewed in frequency to the less frequent.
The Quantities of Neutral/Professional and Negative Behaviour Categories
The quantities of neutral/professional and negative behaviours also varied significantly between the midwives in the study. The frequency data for individual midwives revealed some patterns of behaviour. There were nine midwives who had below and two above study average frequency of all of the neutral and negative behaviours. In a similar manner to positive support behaviours, the majority of midwives (n = 37, 75.5%) showed a mixture of behaviours, displaying some neutral or negative behaviours. The quantities of neutral/professional and negative behaviours by the midwives are further summarised in Table 6 . Overall, negative behaviours were seen infrequently.
These results not only relate to the quantity of supportive and non-supportive behaviours observed and recorded but also contribute to the analysis of the quality of the support observed.
Content and Construct Validity Testing: The Measurement of the Quality of Intrapartum Midwifery Support
This was calculated using the Spearman's Rho correlation coefficient test.
The postnatal 'SCIB' questionnaire was completed fully for 42 of the 49 observations (86%) Where a woman was observed being cared for by two different midwives she was asked to complete two SCIB questionnaires. Women generally reported feeling very well supported in the questionnaire. The mean score for the study was 4.6 out of a possible total of 5.
The observers' global assessments of the quality and quantity of care observed were also generally skewed to the positive ( Table 7 ).
Spearman's Rho correlation coefficients were calculated between rates of negative behaviours by the midwife recorded in the SMILI and the assessments of support recorded by the woman in the SCIB and the observer in the global ratings questionnaire. Negative behaviours and inattentiveness by the midwife showed significant moderate inverse correlation with the assessment of the midwifery support by women and observers ( Table 8 ).
One of the key elements of support derived from the literature is the importance of the continual physical presence of the midwife to woman's feelings of being supported. The results of the analysis to test the association between these elements are given below. This analysis sought to test whether the quantity of presence or attendance by the midwife is an element in the assessment of the quality of the support. These results (presented in Table 8 below) show a significant strong inverse correlation between the amount of time that the midwife spent out of the room and the woman and observer's assessments of support. The higher the proportion of the observation that the midwife was out of the room, the lower the assessment of the support offered.
Correlation coefficients were calculated between positive midwifery support behaviours and the assessments of support by the woman and observer, in order to test whether the quantity of positive midwifery support behaviours is a key element in the measurement of the quality of midwifery support. These results show a significant moderate correlation between women's assessment of the support they received and the overall measurement of emotional support and support of the partner ( Table 9 ). Analysis of the data for the sub-categories of emotional support found that the most significant element of emotional support for women appeared to be rapport building (Spearman's Rho correlation with SCIB score .432**, p = .002).
Construct Validity of the SMILI
The main method to test the construct validity of the instrument is to identify whether women's views of the support correlate significantly with the data collected using the SMILI. These links have been clearly identified in the previous section, with significant correlations between women's views and the quantity of positive and negative behaviours recorded.
However, women's views of the support they received may not only be influenced by the supportive and non-supportive behaviours recorded using the SMILI, but may be also significantly influenced by other elements of the experience. For example, a woman may describe the support she received less positively if she has had a more difficult labour and birth experience in terms of more medical interventions. These other possible influences and their impact on women's and observers' views of the support provided were examined.
This analysis found that there were no correlations between women's views of the support they received and their parity, allocated care pathway, analgesia used, number of medical interventions, type of birth, amount of non-support care and assessment activities, maternity unit and number of years the midwife had been qualified. This may be seen as further evidence to support the construct validity of the SMILI.
Source...