Following the anomaly that occurred during the December Boeing Starliner Orbital Flight Test, NASA and Boeing formed a joint investigation team tasked with examining the primary issues, which occurred during that test.
Those issues included three specific concerns revealed during flight:
1. An error with the Mission Elapsed Timer, which incorrectly polled time from the Atlas V booster nearly 11 hours prior to launch.
2. A software issue within the Service Module Disposal Sequence, which incorrectly translated the SM disposal sequence into the SM Integrated Propulsion Controller.
3. An Intermittent Space-to-Ground forward link issue, which impeded the Flight Control team’s ability to command and control the vehicle.
The joint investigation team convened in early January and has now identified the direct causes and preliminary corrective actions for the first two anomalies.
The intermittent communications issues still are under investigation. NASA reviewed these results on Jan. 31 along with multiple suggested corrective actions recommended by the team.
While NASA was satisfied that the team had properly identified the technical root cause of the two anomalies, they requested the team to perform a more in-depth analysis as to why the anomalies occurred, including an analysis of whether the issues were indicative of weak internal software processes or failure in applying those processes. The team is in the process of performing this additional analysis, as well as continuing the investigation of the intermittent communications issues. NASA briefed the Aerospace Safety Advisory Panel on the status of the investigation this week.
Regarding the first two anomalies, the team found the two critical software defects were not detected ahead of flight despite multiple safeguards. Ground intervention prevented loss of vehicle in both cases. Breakdowns in the design and code phase inserted the original defects. Additionally, breakdowns in the test and verification phase failed to identify the defects preflight despite their detectability. While both errors could have led to risk of spacecraft loss, the actions of the NASA-Boeing team were able to correct the issues and return the Starliner spacecraft safely to Earth.
There was no simple cause of the two software defects making it into flight. Software defects, particularly in complex spacecraft code, are not unexpected. However, there were numerous instances where the Boeing software quality processes either should have or could have uncovered the defects. Due to these breakdowns found in design, code and test of the software, they will require systemic corrective actions. The team has already identified a robust set of 11 top-priority corrective actions. More will be identified after the team completes its additional work.
The joint team made excellent progress for this stage of the investigation. However, it’s still too early to definitively share the root causes and full set of corrective actions needed for the Starliner system. What is expected is to have those results at the end of February, as was the initial plan, as well as making sure there is a comprehensive understanding of what happened so that one can fully explain the root causes and better assess future work that will be needed. Most critically, the necessary steps need to be completely understood prior to determining the plan for future flights. Separate from the anomaly investigation, NASA also is still reviewing the data collected during the flight test to help determine that future plan. NASA expects a decision on this review to be complete in the next several weeks.
In addition to these reviews, NASA is planning to perform an Organizational Safety Assessment of Boeing’s work related to the Commercial Crew Program. The comprehensive safety review will include individual employee interviews with a sampling from a cross section of personnel, including senior managers, mid-level management and supervision, and engineers and technicians at multiple sites. The review would be added to the company’s Commercial Crew Transportation Capability contract. NASA previously completed a more limited review of the company. The goal of the Organizational Safety Assessment will be to examine the workplace culture with the commercial crew provider ahead of a mission with astronauts.
Boeing’s Orbital Flight test launched on Dec. 20, on a United Launch Alliance Atlas V rocket from Space Launch Complex 41 at Cape Canaveral Air Force Station in Florida. The mission successfully landed two days later on Dec. 22, completing an abbreviated test that performed several mission objectives before returning to Earth as the first orbital land touchdown of a human-rated capsule in U.S. history.