Recent DP incident onboard a DSV

A member has reported the following DP incident onboard a diving support vessel (DSV). The company is still awaiting information from the DP equipment supplier regarding the onboard ‘black box’ but is keen to see this safety flash being distributed to warn of the potential problem with this mode in the system and the uncontrolled change to software.

Background

At the time of the incident, the vessel was conducting open water diving operations at 90 msw on a suction anchor. The starboard diving bell was deployed and two divers were deployed working on the suction anchor, the bell being set above the subsea structure.

The vessel was on a heading of 215 degrees with three generators online and with all thrusters running and selected into the DP system. Reference systems in use were 3 x DGPS and 2 x moonpool tautwires. Weather at the time was winds of 35 to 40 knots from 220 degrees with occasional gusts to 50 knots.

The DP watch was about to be handed over.

Sequence of Events

• A series (estimated to be three) of larger than previously experienced waves was felt to move the vessel off in the stern direction (the vessel was heading into the prevailing weather);
• Over an eight second period all five reference systems were rejected by the DP system on ‘prediction error’;
• At 00 04 15, the DP operator (DPO) deselected then reselected DGPS1 which recalibrated but then immediately rejected again;
• At 00 04 17, Taut Wire 1 out of limits;
• At 00 04 25, Alarm for ‘position dropout’ (30 seconds after last reference rejected);
• At 00 04 26, Taut Wire 2 out of limits;
• 00 04 27, DGPS1 deselected then reselected by DPO. DGPS1 calibrated and was taken as reference origin by controller A;
• At 00 04 28, auto switch from controller A to controller B;
• Around this time (40 seconds from initial wave impact) dive control called the bridge to ask if the vessel had suffered a position loss. The DPO observed that the vessel had now moved approximately 40m astern according to the survey display;
• At 00 04 33 to 36, DGPS2, DGPS3 and Taut Wire 1 calibrated back in, but only to controller A;
• At 00 04 49 DGPS3 accepted as reference origin to controller B;
• Now that the online controller had accepted a valid reference system (DGPS3), the DP system increased thrust to maintain the set point. Position was stabilised. Moves were initiated back to the original vessel position and the divers recovered to the bell;
• At 00 29 00, the bell was on the surface. Taut wires were recovered and the vessel remained on DP while investigations into the incident were initiated.

Initial Investigation Findings

1. The initial investigation concentrated on the potential causes of the simultaneous rejection of five reference systems on ‘prediction error’. The DP equipment supplier gave a description of the prediction test feature. The prediction test detects sudden jumps in the measured values (of reference systems) and immediately rejects those that lie outside the limits. The test will also sooner or later reject a system that is drifting away from the other reference systems as the measurement data is compared to the DP model. The prediction rejection limit is established as a function of the estimated position in the mathematical model and the actual measurement accuracy. This feature is useful for quickly rejecting for example a DGPS which takes a sudden ‘jump’ or a taut wire which ‘slides’. The downside to this feature is that if the vessel moves further or faster than the model anticipates (due to rough weather, for example), then the DP system will assume that the reference systems are incorrect and will reject them all.
2. The software basis of the vessel’s DP system had been upgraded in April 2005 from version 3 to version 5. The prediction rejection feature had been included in version 3 of the software and therefore had been present on the vessel for around four years without causing any problems. However, the version 3 software did not incorporate any operator-set variables. The version 5 software had three settings for the prediction rejection limit. These were ‘narrow’, ‘normal’ and ‘wide’. Although these limits were variable, depending on the rate of change of the reference system position, they could be said to equate to distances of 8 metres for narrow, 11 m for normal and 25 m for wide. It is important to note that in the previous software revision (version 3), the ‘normal’ setting (which was the default and could not be altered by the operator) equated to 15 m.
3. Shortly after the software upgrade was installed on the vessel, the DPOs had become aware of a problem with seemingly healthy reference systems being rejected in rough weather. The DP equipment supplier had been consulted and the vessel was advised to change the reference system acceptance limits (prediction error) from ‘narrow’, which was the default setting, to ‘normal’. No specific advice was given as to the weather conditions which would require a change of setting from ‘narrow’ to ‘normal’ or ‘wide’. Since then, the DP system had always been operated with the acceptance limits set to ‘normal’. This was controlled by checklist and the ‘normal’ setting was in use at the time of the incident.
4. In the hour leading up to the incident, the vessel was reported to be moving within a footprint of between 3 m and 5 m with occasional excursions in excess of 5 m. There were also two reported excursions up to 10 m. The position alarm limit was set at 3 m and the DP system printout suggests that this limit was being breached on average every three to five minutes. During this time there were no position reference dropouts.
5. Although the weather conditions were heavy, the vessel was still operating within the limits of her DP Class 2 capability. The position excursions were being reported to dive control, which was satisfied that the vessel motions were within the safe working limits for this open water location.
6. From the alarm printouts, it is likely that the DGPS systems were rejected after the vessel was moved astern 6 m in a very short space of time (outside of the limits for the ‘normal’ setting). This was followed seven seconds later by the rejection of the taut wires, at which time it is estimated that the vessel had lost position by 20 m.
7. After rejection of the reference systems, the vessel would have attempted to maintain position using the information contained within the mathematical model. In this mode, the DP system would apply the average thrust vector that had been necessary to maintain position prior to the loss of the position references. Allowance would also be made for changing wind speeds which could, of course, still be measured by the wind sensors. Position keeping on the model under these circumstances (being moved rapidly off set point by abnormal wave action before losing reference systems) would not be particularly accurate.
8. Once vessel motion slowed sufficiently for the reference systems to be accepted into the DP system again, the vessel applied the necessary thrust to maintain the new set point as evidenced by the high thrust alarm (stern azimuths) printouts.

Results of the Investigation into Procedural and Human Factors

1. The Master and DPOs were all appropriately qualified and very experienced on this vessel. The actions taken during the incident were reasonable.
2. The Master and the DPOs were asked about their familiarity with the reference prediction feature of the DP software. Until the problems experienced shortly after the software revision and the subsequent to DP equipment supplier advice, the Master and DPOs were unaware of the feature. Subsequent to the DP equipment supplier advice, they were aware of the reason for the feature (detection and rejection of sudden jumps in reference systems) and were aware of the need to change the setting to ‘normal’ from the default ‘narrow’ setting. However, they were unaware of any numerical values attached to each setting or any guidance regarding the weather conditions which require a change between the various settings. It is evident that they had not considered the possibility of all (in this case five) reference systems being simultaneously rejected by this software feature in heavy weather.
3. The DP equipment supplier’s operator manual supplied to the vessel states the following with regard to the reference system acceptance limits:”The acceptance limit for the Prediction Test and indirectly also the Median Test can be changed.NarrowNarrow limit. Corresponds to a Minimum Prediction Error circle with a small radius. Recommended for drill ships/large vessels. For example, drive off due to thruster errors will be detected.NormalMedium limit. Systematic drift of the position measurements will increase the prediction error limit. Example of use is on smaller vessels with moderate position keeping requirements. For example transitory position drift off due to rough sea will be allowed.WideWide limit. Systematic drift of the position measurements will increase the prediction error limit. Suitable, for example, for sailing in Mixed/Joystick mode at high speed.”This guidance, while no doubt technically accurate, is not easily interpreted. The terms ‘large vessel’, ‘small vessel’ and ‘rough sea’ are not defined. The possibility of simultaneous rejection of all position references due to vessel motion, while perhaps implied, is not highlighted.
4. An amendment to the DP equipment supplier’s operator manual was produced in February 2005 in order to “provide a more comprehensive description of the acceptance limits for the reference system prediction test”.”The acceptance limit for the Prediction Test and indirectly also the Median Test can be changed.NarrowNarrow limit. Corresponds to a Minimum Prediction Error circle with a small radius. The radius may still increase due to increased noise in the position reference system. Narrow is recommended when operating in calm weather and with requirements for accurate station keeping. For example, drive off due to thruster errors will be detected at an early stage, resulting in rejection of all position reference systems and a ‘position dropout’ alarm. If all available (or the dominating) position reference exhibit an erroneous drift in position, the system(s) will also be rejected at an early stage before the vessel is significantly affected by the wrong measurements.NormalMedium limit. The same Minimum Prediction Error limit as for Narrow is used. There is an additional feedback mechanism where the actual deviation from the model is used to increase the Prediction Error limit up to a maximum of 2 to 3 times the smallest radius. Normal should be applied in situations where there is a chance that the DP model does not follow the actual movement of the vessel. This is especially relevant when operating in rough sea. It is also applicable for a vessel operating with another vessel alongside. A negative side effect of this setting is that the DP system will to a larger extent than with the Narrow setting, tend to follow drifting position reference systems.WideWide limit. A Minimum Prediction Error circle with an increased radius compared to the other two settings is used. The same feedback mechanism as for Normal is used, and the maximum value of the Prediction Error is also increased. Wide is suitable, for example, for sailing in Mixed/Joystick mode at high speed.”This revised guidance is marginally better, but was not available on the vessel.
5. The DP operator reported that he observed a 40m ‘jump’ astern in the position of the vessel on the survey display after being asked by dive control if the vessel was suffering a loss in position. The dive shift supervisor who was observing the same survey data reported that the position loss was seen to be progressive with no ‘jump’. Under the circumstances, with so many almost simultaneous occurrences, it is understandable that the DPO may have seen the position loss as instantaneous, as he was unable to continually monitor the survey screen. There is no other evidence to suggest a DGPS jump as a contributory factor to this incident, but this will be confirmed when the DP equipment supplier has completed further analysis of the DP data logger files.A limited enquiry through the industry has indicated that there may be a widespread ignorance of the potential hazards due to mismanagement of the reference prediction error rejection feature of DP equipment supplier’s DP systems.Conclusions and Recommendations

   The company’s conclusions were as follows:

Immediate Causes

• The immediate cause of the loss of position was the simultaneous rejection of all five reference systems following a rapid loss of position due to exceptional wave action. The references were rejected on ‘prediction error’, because the vessel movement (speed and distance) was outside of the limits expected by the DP system mathematical model. The vessel then attempted to maintain position using the mathematical model until position references were re-established and the position stabilised.

Underlying Causes

• When the DP software was upgraded from version 3 to version 5, the rejection settings for the prediction error were tightened up in the ‘normal’ setting from 15 m to 11 m. The vessel reported that DP operations had been conducted in worse weather conditions than those experienced during the incident without any rejection of reference systems using the previous software revision. This was the worst weather experienced since the upgrade and it was apparently sufficient to trigger the ‘prediction error’ rejection criteria. The vessel crew were unaware that they were operating with a less tolerant rejection limit than previously.
• DP operators, although aware of the ‘prediction error’ feature and its ability to reject an errant reference system, had not considered the possibility of all reference systems being simultaneously rejected on prediction error limits due to the motion of the vessel in heavy weather.
• When the software upgrade was installed, no formal briefing had been given by the DP equipment supplier regarding new or revised features.
• The amendment to the DP equipment supplier’s DP operator manual published in February 2005 giving expanded guidance on the prediction error feature was not available on the vessel. This despite the fact that the new manual had been delivered to the vessel in April 2005.

Recommendations – the following recommendations are made following the investigation of this incident.Immediate

• A shipboard standing order was produced by the Master stating that “the maximum value for the prediction error is 11 m when the ‘normal’ limit is selected. Note the innovation factor has increased from 0.2 to 0.4 (which makes the risk of simultaneous rejection of references in rough weather far less pronounced).”
• The DP equipment supplier’s Field Engineer has adjusted the ‘innovation factor’ to 0.4 from its previous setting of 0.2. The innovation factor is used by the DP system in its calculation of the prediction error limits. In effect, this takes the system’s limits back to the settings that were in effect with the previous software revision.
• Note. The DP equipment supplier has subsequently remarked that the ‘wide’ setting should not be used in rough weather for critical DP operations in conjunction with the change in innovation factor that the Field Engineer made as the rejection limits are too wide.
• A toolbox talk should be conducted to ensure that all DPOs are fully familiar with the reference prediction error function. DPOs are to be aware of the benefits and potential hazards of the function.
• DPOs should be made aware of the advice contained in an email from the DP equipment supplier that if all position references are rejected, then they should all be de-selected before attempts are made to re-select them (to prevent a single reference being calibrated against an incorrect model).

Medium/Long Term

• Discussions should be held between the DP equipment supplier and vessel operator to establish whether the innovation factor is correctly set for the DSV. More detailed guidelines should be established regarding the vessel motions which would necessitate an operator change in the reference system acceptance limits from narrow to normal or wide.
• The above exercise should be repeated for other vessels in the fleet.
• A fleet safety bulletin or equivalent should be promulgated to other fleet vessels advising of this issue.