search

gwu-cs-iot / collaboration

Spring '20 IoT - systems and security class. This is the collaborative half of the class.
https://www2.seas.gwu.edu/~gparmer/classes/2020-01-01-Internet-of-Things-Systems-Security.html
MIT License
14 stars 26 forks source link

Paper discussion 9a: A Large-Scale Analysis of the Security of Embedded Firmwares #71

Open zacharied opened 4 years ago

zacharied commented 4 years ago

Please add your feedback and reviews below.

lrshpak commented 4 years ago

Reviewer: Lily Shpak Review Type: Comprehension

Main Problem Being Solved

As embedded systems become more prevalent, the security risks associated with them are also increasing. Since these systems are becoming more widely used, this means that companies are quickly trying to develop software to run on these systems. This segment of software is called firmware, and it thought of to be software that is specific to hardware.

Main Contributions

This paper did an analysis of lots of different types of firmware. They extracted the image for each firmware and ran tests on the image to see if they could find any vulnerabilities. The authors did this analysis because they think that the firmware is the main cause for vulnerabilities in embedded systems.

Questions

  1. The author mentions that different devices may use the same firmware, I thought firmware is specific to the hardware, so how can the same firmware work with different devices?
  2. How do the authors analyze the security of a firmware, if all the firmwares are different?
  3. The authors say that unpacking firmware is an ethical issue because attackers will know the vulnerabilities for the firmware, but isn't that the point of doing this analysis?
grahamschock commented 4 years ago

Reviewer: Graham Schock Review Type: Comprehensive

Problem Being Solved New enabling technology has led to embedded systems to be more common in society. These devices re present in everything from printers to medical implants. However, due to their importance in everyday human life, security becomes a key concern. This concern is exasperated by the fact that the firmware market is very complex and it is hard to update firmware when a researcher comes out with a patch.

Main contributions

This paper did a large scale data analysis where they collected firmware files from online. They then tested these for known vulnerabilities. Through this they show the challenges of doing such a large analysis. They also detail a methodology on how to design distributed static analysis for firmware. Based on their findings they show that firmware is very vulnerable.

  1. The paper mentions firmware update sites. How do we update firmware? I thought that it is baked into hardware. https://techterms.com/definition/firmware

  2. What are ways to actually solve the security issue? It seems the issue is that there is too much complexity in the firmware market. Do we need more regulation?

  3. What does static analysis actually look like? The paper says that they use static analysis to identify vulnerabilities. What are they actually looking at?

hjaensch7 commented 4 years ago

Reviewer: Henry Jaensch

Review Type: Comprehension Review

Problem Being Solved

The proliferation of embedded devices is happening rapidly. Many of these devices are insecure but due to their high volume it's hard to keep track of all of these vulnerabilities.

Main Contributions

This paper unpacked firmware for a large amount of devices and analyzed them to discover bugs/vulnerabilities. This paper known bugs to discover similar ones in other products. The primary contribution exists as a web service that is capable of unpacking embedded device firmware and statically analyzing it.

Questions

  1. Could this tool become apart of a build process at a large scale? Would a company be able to employ this analysis tech to confirm the security of it's firmware updates and patches?

  2. How does this static analysis work. Once the firmware is unpacked properly other than the correlation engine what is programmatically being done to identify bugs?

nikorev commented 4 years ago

Reviewer: Niko Reveliotis Review Type: Critical

Problem Being Solved

Firmware is continuing to grow in complexity as more and more devices expand their capabilities. With this complexity opens the window up for exploits within these images. Many consumer devices (such as routers) utilize firmware, leaving millions of devices at risk. This paper attempts to develop a way to analyze the security of these firmware images in a way which is scalable.

Main Contributions

Three Questions

  1. Why has firmware continued to grow as devices have become more robust, versus a creating a more robust OS?
  2. What are the tradeoffs to using firmware versus a "low-level" OS?
  3. I'm still not entirely sure how the distributed platform glue code in Figure 2 plays into their platform. I kind of understand the concept of glue code, but specifically what does it mean in terms of their platform?

Three Critiques 1) The hash cracking technique utilizes CUDA-enabled GPUs (which is great), but what is the AMD alternative? A lot of these papers leverege on features exclusive to a single platform. 2) Although I understand the limitations of running a dynamic approach (different attacks/tests depending on the firmware), it is important to address the possible exploits that could've fell through the static tests used. 3) This is a bit weak of a critique, but the paper doesn't emplore scaling of their testing across multiple machines on a distributed network. They briefly touch upon testing on a distributed network with multiple machines as something to test, but in this paper they don't officially include it.

Others commented 4 years ago

Reviewer: Gregor Peach

Review Type: Comprehension

Problem Being Solved

There is an impression that IoT devices are insecure, but the evidence is mostly anecdotal. One of the most security critical pieces of software on devices is their firmware. Firmware is backed into the hardware, so it's difficult to change or update (without physical access or powerful hardware mechanisms). That means if it's compromised, you may as well just throw the whole device away--there is nothing to be done.

Main Contributions

In this paper, the authors document how the gathered a ton of firmware, and then analyzed it. It's honestly amazing how much firmware/software they gathered. A major contribution seems to be their procedure for creating this dataset. After gathering the data, they ran informational and security analysis on it. They identified several, classes of pervasive bad practices that a

Three Questions

pcodes commented 4 years ago

Reviewer: Pat Cody Review Type: Critical

Problem Being Solved

Although due to anecdotal evidence embedded systems have a bad reputation for security, it is difficult to verify these claims on a large scale. Performing thorough security experiments can be prohibitively expensive and time-consuming.

Main Contributions

This paper attempted to mass-analyze various firmwares for embedded systems, and in the process created a framework for more easily doing so. Their system has several components, including an ingestion program to mass-collect firmware, worker nodes for doing analysis, and a correlation engine for producing more advanced results. They discuss many of the security flaws they encounter, and highlight several with case studies.

Questions

Critiques

huachuan commented 4 years ago

Reviewer: Huachuan Wang Review Type: Critique

Overview

This paper presents a large-scale analysis of firmware images. They leverage this large-scale analysis to bring new insights on the security of embedded devices and to underline and addressed detail several vital challenges in future research. They also show the main benefits of looking at many different methods at the same time and of linking their results with other large-scale datasets.

Contribution This paper shows the advantages of performing large-scale analysis of firmware images and describe the main challenges associated with this activity. It proposes a framework to perform firmware collection, filtering, unpacking and analysis on a large scale. It implemented several efficient static techniques. It presents a correlation technique that allows propagating vulnerability information to similar firmware images.

Questions

  1. What is the fuzzy hash, why it will rise computational challenges?

  2. The Backdoor is set on purpose or not? When the backdoor problem is reported, would it be very easy for the developer of the system to find it out?

Critique

This paper has listed many challenges at the beginning of the paper, however, not provide a very reliable way to solve each challenge. For example, in order to identify the firmware image, they have to do it manually. 34% +- 8% is already a big range, I doubt this estimation gives a 95% reliable measure of the useful data in the sample.

mjhegarty commented 4 years ago

Reviewer: Mikey Hegarty Review Type: Critical

Problem

Embedded systems are becoming more and more common and all have unique firmware on them. Firmware can have very large vulnerabilities on them that can go unnoticed if attention is not brought to them. Devices can share vulnerabilities but the shear number of firmware out there + the non-systematic formatting of the files + the no common database for the files makes it difficult to examine a large amount of them.

Main contributions

The authors design a framework for testing the different images for vulnerabilities and finding patterns where vulnerabilities found in one specific framework might also be in another. The impressive part of their contribution in my opinion is the automatic parser that crawls the web to find firmware and the unpacking of those firmware. They then perform static analysis on these firmware to look for vulnerabilities and to try and crack the password hashes in it.

Questions

  1. I guess one broad question I have is what can be done with all of this information? I understand the implications of having the information for a wifi-router but for some IoT devices its a little unclear what can be done with security vulnerabilities without having physical access to the device itself.
  2. I am not super familiar with the different between static analysis and dynamic analysis. Is there any chance that some of the static vulnerabilities that they found require dynamic analysis to see for sure if they work or not?
  3. Do companies really have the word "backdoor" in their backdoor strings that isn't just leftover from testing and is unusable? That seems absolutely ridiculous to ship a product with that still in there.

Critiques

  1. They acknowledge this in the challenges section but I feel like there is definitely a bias on the types of embedded systems used in the study due to the nature of crawling the web for firmware packages. I would guess that there is a much bigger and more public library of more general robotics and embedded systems images over some of the specifics that are now emerging in new markets such as medical devices.
  2. In terms of the numbers reported for their results I feel like there could be a large amount of overlap between very similar releases v 1.1.2 vs 1.1.3 as well as outdated and no longer used firmware that the automatic crawler might have picked up that could conflate their results a little bit.
  3. Honestly it is hard to really understand how good the results are without knowing the full context for them. It would have been nice to have some sort of statistics of how many devices had exploits of the 26,275 checked. There aren't any graphs representing results either.
tuhinadasgupta commented 4 years ago

Reviewer: Tuhina Dasgupta Review Type: Critical

Problem:
Embedded systems are notoriously bad at security, but there's a question as to whether that reputation is deserved. The claim is hard to investigate as it's costly but the paper does make the point that the firmware is the most critical piece of IoT systems.

Contributions:
The paper analyzed a large quantity of firmware for embedded systems. Through this process, they developed a framework for analysis that can be used in the future. Additionally, hey highlighted security flaws found through case studies.

Questions:

  1. Can the framework for analysis be expanded upon to be more customizable/ support the testing of more features?
  2. Beyond analyzing the security of the system, can the authors provide possible solutions?

Critiques: -No suggestions to fix identified issues -No discussion of tradeoff/alternatives to their hash cracking technique

gkahl commented 4 years ago

Reviewer: Greg Kahl

Review Type: Critical

Problem

Embedded IoT systems are becoming more pervasive in our society and the uncertainty of their security is concerning. It has been difficult to analyze the security of these systems, especially the firmware which is baked into the systems. There hasn't been careful analysis and evaluation of the firmware on these systems, which needs to be done in order to verify the security of these embedded systems.

Contributions

This paper acquires, compares, and analyzes security threats in available firmware for IoT embedded systems. They develop a vulnerability detection firmware which then correlates found insecurities to the other firmware in the database to determine which firmware and devices running the firmware are vulnerable.

Questions

1 - How does identifying these security risks help the currently deployed devices/how are we able to fix these problems in the firmware? Or is this more of a warning to help the vendors with future implementations. 2 - What exactly does it mean to build the images as root? They made it sound like that was something you could choose to do or not to do and that confused me a bit. 3 - When they were bruteforcing the hashed passwords found in the passwd and shadow files, was that just to determine if they were using weak passwords or was the fact that they were solved the security issue?

Critiques