Embedded systems are ubiquitous, found in everything from pacemakers and autonomous vehicles to home assistants and industrial machinery. These hidden computers are often designed to operate with minimal human oversight, placing a significant ethical responsibility on their designers. Ethics in embedded system design goes beyond creating functional products; it involves safeguarding lives, protecting privacy, ensuring transparency, and considering the environmental impact.
Read on to explore the key ethical considerations in embedded system design, illustrated with real-world examples.
Safety & Reliability – lesson learned from Boeing
Perhaps the most critical ethical concern is ensuring the safety and reliability of systems. When an embedded system fails, the consequences can be catastrophic.
Consider the infamous case of the Boeing 737 MAX crashes in 2018 and 2019, where flaws in the MCAS embedded control system contributed to two fatal accidents, killing 346 people. Investigations revealed insufficient testing, poor system documentation, and a lack of pilot training related to the software behaviour.
Research highlights the gravity of such issues. A study by Leveson (2011) on complex system failures stresses that software design flaws, especially in embedded safety-critical systems, are often not simple bugs but systemic problems due to inadequate safety engineering processes.
When it comes to critical applications, it’s smart to use formal verification methods, test rigorously under extreme conditions, and build in layered fail-safes to make sure a small issue doesn’t lead to a full system failure.
Security & Privacy
As embedded systems connect more devices to networks, known collectively as the Internet of Things (IoT), security and privacy concerns become a major focus. Poorly secured devices can serve as entry points for attackers, creating serious risks.
One clear example is the 2015 “Jeep Hack,” when researchers Charlie Miller and Chris Valasek remotely took control of a Jeep Cherokee by exploiting vulnerabilities in its embedded systems connected to cellular networks. They were able to manipulate the vehicle’s steering, brakes, and transmission, exposing serious flaws in how these systems were secured. The fallout was significant: Chrysler ended up recalling 1.4 million vehicles to fix the vulnerabilities, sending a strong message to the automotive industry about the risks of connected technologies.
But, the risks extend beyond cars. As an example, wearable health devices collect sensitive personal data. A 2019 study published in the Journal of Medical Internet Research found that 79 percent of top-selling health apps shared user data with third parties, often without proper disclosure or consent. This kind of data exposure not only threatens personal privacy but can also be exploited for insurance discrimination, identity theft, and other malicious purposes.
These challenges can be addressed through a multi-layered security strategy. Designers and developers should implement end-to-end encryption to protect data both in transit and at rest. Strong authentication protocols are needed to verify the identity of devices and users. Regular firmware updates are necessary to patch newly discovered vulnerabilities and reduce the window of opportunity for attackers. In addition, applying strict data minimization practices, where only the necessary data is collected, can significantly lower the potential impact of a breach.
Transparency & Accountability
Users often do not understand how embedded systems make decisions that affect them.
Autonomous vehicles provide a vivid example. When a self-driving Uber vehicle struck and killed a pedestrian in 2018 in Arizona, investigations revealed that the embedded perception system misclassified the pedestrian and decided no action was necessary. The system’s black-box nature made it difficult to immediately understand what went wrong.
This incident highlights a broader issue across many industries: the opacity of decision-making processes in intelligent systems. Without clear visibility into how and why a system behaves in a particular way, users and regulators alike struggle to assess the safety, fairness, and reliability of these technologies.
Recognising this need, ethical guidelines such as those proposed by the IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems (2019) strongly emphasise the importance of explainability. Systems should be designed in such a way that their decision-making processes can be inspected, interpreted, and understood by human operators. Explainability not only promotes accountability but also builds trust, providing assurance that systems behave in ways that are consistent with user expectations and broader societal values.
To move towards this goal, engineers and designers should build systems with comprehensive logging mechanisms that record key operational data. They should also provide clear and accessible documentation that explains how systems are intended to function, including any assumptions, limitations, and decision thresholds. Where feasible, systems should include decision-explanation features that allow users or auditors to review why a particular action was taken, especially in high-risk or safety-critical contexts.
Accessibility & Inclusion
An ethically designed embedded system should be accessible to users with a wide range of abilities. Yet, many embedded devices unintentionally exclude people with disabilities.
For instance, touchscreen-only interfaces in appliances, like smart ovens, can be unusable for visually impaired individuals unless designed with assistive features. The Web Content Accessibility Guidelines (WCAG) are often cited in software design, but similar principles should guide embedded systems as well.
Environmental Impact
Embedded systems designers also face the ethical challenge of minimising environmental harm.
Electronic waste (e-waste) is a major global issue. The Global E-Waste Monitor (2020) reports that the world generated a record 53.6 million metric tons of e-waste in 2019, with embedded electronics in appliances and consumer devices contributing significantly.
Designers can help by using recyclable materials, minimising power consumption with energy-efficient microcontrollers, and designing for easy disassembly to facilitate recycling.
Long-Term Support and Maintenance
Planned obsolescence – intentionally designing products with a limited useful life – is both unethical and wasteful. For embedded systems, failing to provide long-term support can introduce security vulnerabilities and leave users stranded.
In healthcare, for example, abandoning support for older diagnostic devices forces hospitals to choose between expensive upgrades or risking patient care with insecure devices. Without updates, these devices may miss security patches, exposing sensitive patient data and increasing the risk of errors.
Many healthcare institutions operate on tight budgets, making upgrades challenging. The ethical responsibility of manufacturers is to ensure that products continue to function safely throughout their lifecycle. Planned obsolescence in critical devices compromises patient safety and the integrity of healthcare systems.
The issue is not limited to healthcare. In the consumer market, companies entering liquidation can leave customers with expensive, unsupported devices. A recent example is Fisker, the electric vehicle start-up that faced severe financial difficulties. As the company moved towards insolvency, many customers were left with high-end electric vehicles that quickly became non-functional “paperweights” due to the loss of software support and spare parts. This case highlights a broader ethical concern: when companies fail to ensure continuity of service and support, users bear the cost, both financially and practically.
In industries like transportation, energy, and telecommunications, long-term support is essential for maintaining secure, reliable systems. By prioritizing ongoing maintenance, manufacturers demonstrate a commitment to quality and security, fostering trust in services that rely on embedded systems.
Roedan’s approach to designing embedded systems
At Roedan, we prioritise transparency, data protection, and human-centred design in the development of embedded systems. Our commitment to clear documentation, secure data practices, and explainable decision-making ensures that our technologies are safe, reliable, and aligned with societal values.
We believe that ethical considerations should be embedded from the outset to build trust and accountability.
Interested in developing ethical technologies? Contact us at sales@roedan.com or use this page to schedule a meeting with our leadership team.