We are delighted to be publishing today the Safety Documents created to support the safe testing of the Automated Valet Parking function using the StreetDrone test vehicle in a car park.
This is the result of months of effort by Connected Places Catapult and Parkopedia. Following the post on Systems Engineering these documents now support the case made to drive under autonomous control within a car park.
These safety documents encompass system safety and operational safety.
The Safety Plan and the Requirements are live documents (they have been updated for testing in a car park and will be updated for future milestones)
Any entity seeking to conduct autonomous vehicle trials will need to develop and publish a safety case specific to their own trials (as specified by the government’s Centre for Connected & Autonomous Vehicles (CCAV) Code of Practice for Automated Vehicle Trialling) and gain permission to do so.
You know how it can be a real challenge to understand the complexity in emerging multi-modal and automated transport systems?
By harnessing Systems Engineering best practices, our Systems Engineers at the Connected Places Catapult (CPC) use their expertise in systems requirements gathering, safety management, and verification and validation, ensuring that all aspects and interactions in the AVP system are clearly defined, integrated, evaluated and tested, thus improving our ability to deliver a safe, efficient and innovative system.
Systems Engineering is an established practice capable of delivering technically complex systems, and Systems Engineers swear by the V lifecycle model, which shows the logical relationship between the different System Engineering activities.
In the V model, time and system maturity proceed from left to right:
In each stage of the system life cycle, the Systems Engineering process iterates to ensure that a concept or design is feasible, and that the stakeholders remain supportive of the solution as it evolves.
The process is applied sequentially, one level at a time, adding additional detail and definition with each level of development.
1. Concept Development
The concept development is the first step of the life cycle of the AVP project. This stage explores concepts, identifies stakeholders’ needs and proposes viable solutions.
Requirements definitions are the key to success in the design and development of any complex system. Our Systems Engineers have carefully elicited requirements from all the project stakeholders to ensure the final product meets everyone’s needs, from technical feasibility to budget considerations and testability.
The total set of requirements encompasses the functional, performance, and non-functional requirements and the architectural constraints.
3. System Architecture
The architecture of a system is its fundamental structure. The purpose of drawing an architecture is to define a comprehensive solution based on principles, concepts, and properties logically related and consistent with each other. The architecture defines the system boundary and functions, from which more detailed system requirements can be derived.
step of the cycle is focused on developing:
At the start of the AVP project, both of these architectures were drawn. The system level architecture refers more to the overall system and its operating environment. The functionalities that enable the AVP technology are mapped on the functional architecture, based on the Autoware ROS and showing modules such as perception, sensor fusion, path planning, and others.
4. System design and development
Once the initial opportunities and constraints have been identified in the steps above, it is time to create and describe in detail the AVP system to satisfy the identified needs.
While the Parkopedia Engineers work on developing the AVP system in accordance with the ConOps, Requirements and System Architecture, the Systems Engineers at the CPC focus on the functional system analysis and producing the Safety Case for the AVP testing.
This lifecycle step expands on ISO 26262, which addresses the functional safety in road vehicles. The analysis of the system level safety is carried through system-level functional analysis, such as the Failure Mode and Effect Analysis (FMEA) and the Hazard Risk Assessment (HARA) , which results in providing the system safety goals (translated as the safety requirements for the system). These form the system safety argument for the Safety Plan for the AVP project. This step is crucial to ensure the risk is minimised if and when the system fails. Every malfunction leads to a warning and degradation strategy and the safety driver should be able to gain control of the vehicle.
Note: the system analysis also involves reviewing all of the previous deliverables as the design is being developed.
5. System Integration
The AVP system is made of different system elements provided by different parties: – The vehicle is provided by StreetDrone – The software, application and maps are provided by Parkopedia – The localisation system is developed by the University of Surrey
This step ensures that all of the disparate system elements integrate to create an effective whole and allows the different teams to work in parallel confident that all of the pieces fit and work together. The interface management activity relate to the steps taken to integrate the different architecture elements:
Reviewing and refining the previously mentioned deliverables (ConOps, System Architecture, Requirements, FMEA, HARA)
Assessing the criteria and methods for Requirements verification and validation (Analysis/Inspection/Test/Demonstration)
The Interface Control Document (ICD), which describes the integration strategy of the AVP system. This is a live document and is continuously updated until the end of the system development and testing.
6. Test & Evaluation
Prior to driving in the desired environment, the vehicle has to demonstrate that it is capable of safely navigating and responding to diverse situations and scenarios, such as emerging pedestrians, vehicles, obstacles, etc. By combining simulation and testing in a real environment (open field track and car parks), the AVP system can be validated.
This step focuses on ensuring that the developed system is operationally acceptable and that the responsibility for the effective, efficient, and safe operations of the system is transferred to the owner.
Can the system be tested, demonstrated, inspected, or analysed to show that it satisfies requirements?
Are the requirements stated precisely to facilitate specification of system test success criteria and requirements
Testing will always be performed with a highly trained safety driver, who will monitor the vehicle behavior at all times and ready to take over in the event of a failure. An engineer will monitor the path on a HMI, so that the safety driver is not distracted. The system must be able to detect and respond to a wide range of static obstacles in the testing environment.
It examines the evidence required to show that acceptable safety has been achieved for any given testing activity, and how that evidence is to be collected, such that when all the evidence is taken together, there is a convincing argument that the project as a whole is safe.
It summarises the evidence collected prior to commencing Autonomous Valet Parking testing in accordance with the Safety Plan. There will be a Safety Case for each test phase: testing in a controlled environment (open field track), testing in car parks and the demonstration.
This document sets out the safety procedures that all participants are required to follow during testing of the autonomous control system and ensures the requirements and procedures are read and understood by all involved in the trial, and adhered to at all times. The Risk Assessment for the trial is also included in this document. Similarly to the Safety Case, each testing phase will be prepared with a RAMS.
After testing, the requirements are reviewed and given a pass/fail mark depending on the acceptance criteria set. In both cases, a justification needs to be provided.
Once again, the previous deliverables are reviewed and refined as required.
7.Operation and maintenance
The AVP system has now been tested in different test environments and settings. The system is ready to be deployed, and as part of this activity, several demonstrations are organised as evidence of the operational effectiveness of the deployed AVP system.
The Systems Engineers carry the below activities, verifying and validating the concept one last time:
Safety Case close out
Verification and Validation close out
Review and refine as required previous deliverables
The self-driving car industry is a fast-moving and uncertain environment, which makes safety the number one priority. This is also why the AVP project consortium is encouraging the industry to work together, collaborate, share use-cases and lessons learnt. Self-driving cars have the potential to ensure safety of passengers, reduce road congestion, drive smarter and in a more sustainable way. By implementing a robust Systems Engineering process, the self-driving car industry can mitigate risks and drive society safely into the future.
Safety is the highest priority at the Autonomous Valet Parking project. As we seek to demonstrate that Parkopedia’s maps are suitable for localisation and navigation within covered car parks, our safety case must ensure the safety of the people, vehicles and infrastructure inside our test vehicle and without. There have been some highly publicised incidents involving other organisations’ autonomous vehicles over the past year or two, so what is the AVP project doing around safety?
Our Safety Case involves a combination of System Safety and Operational Safety, to achieve the required assurance levels around our activities. System Safety covers any and all aspects of the system (hardware, software or both) that contribute to the safety case and is the focus of this post. Operational Safety is all the operational decisions taken to ensure that we are conducting the tests safely and you can read more about that here.
Our test vehicle, a StreetDrone.ONE is a converted Twizy, and comes with ultrasound range sensors. There are eight Neobotix ultrasound sensors in total, three to the front, three to the rear, and one in each door looking sideways. The front sensors are slightly fanned out, and in the rear the outer two are corner mounted. The signals from each sensor are gathered together by a Neobotix board and this publishes the range data as an automotive industry canbus signal. These can be monitored and action taken when a significant measurement is made.
The sensors create a “virtual safety cage”. This virtual safety cage can be imagined as an invisible cuboid around the StreetDrone.ONE, slightly wider and longer than the car itself. If anything, be it a car, pedestrian or wall, intrudes inside this cage, the car should stop immediately, thus acting as a belt and braces to the perception and navigation parts of the AI driving the car.
The video below shows a demonstration of the drive-by-wire system of the StreetDrone at 14 mph. The brake is applied by the actuators at the very beginning of the 3 metres wide white strip. Based on a 14mph start speed, we calculated the braking distance to be 4.5 metres. This is obviously an approximation, and the actual braking distance and time depends on many factors including brake disk wear and tear.
Remembering our high-school equations of motion:
Where “v” is the initial velocity, “u” the final velocity, “a” is the acceleration and “s” the distance.
Now we can rearrange this to obtain the acceleration, remembering in our case the final velocity is zero:
These numbers match well with the similar experiments carried out by StreetDrone using IMU data, which have shown peak deceleration of 0.67g and an average deceleration of 0.46g.
The maximum range of the Neobotix ultrasound sensor is 1.5 metres, so we could do the above calculation in reverse and calculate the maximum safe speed. Allowing for a safety buffer of 0.5m:
The distance “s” is 1.0 metres, the acceleration “a” is 4.3m/s^2, and again the final velocity “u” is 0 m/s.
The present AVP plan calls for a maximum speed within car parks of 5 mph which is well within the safety margin calculated above. The next step now is to process the data we’ve captured from the ultrasonic sensors and to develop the software that will automatically apply the maximum brake whenever the virtual safety cage is breached.
Exciting times: this week marks the start of testing for our StreetDrone autonomous vehicle, as we build towards an Autonomous Valet Parking demonstrator.
This first testing phase will be in a controlled environment to minimise risk. For that reason, we’ve chosen Turweston Flight Centre, which has previously been used by our friends at StreetDrone who have done some of their testing there.
In accordance with our project Safety Plan and the Safety Case for this phase of the project, we’ve also been busy collecting safety evidence prior to starting the live tests. These documents will be made publicly available as part of our goal to be as transparent as possible, and for those who wish to use them as a starting point for their own safety case.
For this phase, our safety evidence documents are:
One of the key objectives of this Autonomous Valet Parking project is to demonstrate our autonomous vehicle parking itself in a covered car park. The Transport Systems Catapult is responsible for the safety work package which ensures that all activities undertaken during the project are done in a systematic and safe manner. One of the important deliverables to ensure safety is the Risk Assessment and Method Statement (RAMS).
The RAMS document generally includes:
1. An overview of the project and key objectives to provide the reader with a background of the project
2. The activity being assessed, including:
Roles and responsibilities
Limits of the operation and trial details (route planned, scenarios, vehicle specifications, time of day, limits, weather, specifications)
Legal considerations such as vehicle insurance and laws
Training achieved (eg. driver training on the StreetDrone.ONE vehicle, taking over manually)
3. A risk assessment listing hazards, consequences, mitigation methods and detailing who might be harmed. Following the ISO 26262 standard, a hazard analysis and risk assessment is required in order to determine the criticality of a system.
The risk analysis is focused on:
Risks related to the ongoing operation of the vehicle
Risks related to the operation of external factors that affect current operation
Risks arising from the new equipment that may affect the safety of the vehicle or other
The method statement part describes in a logical sequence how a task will be carried out in a safe manner. It includes all the risks identified and the measures needed to control those risks.
The purpose of the method statement is to ensure that:
The trial is carried in a structured, controlled and safe manner
The hazards and associated risks are understood and mitigated
While the ultimate goal of the project is to demonstrate Autonomous Valet Parking, we will build up to this demonstration through smaller manageable steps and a separate RAMS will be produced at each stage: